This paper is in the following e-collection/theme issue:

Digital Health Reviews (1174) Quality/Credibility of eHealth Information and Trust Issues (485)

Published on in Vol 24, No 4 (2022): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/36463, first published .
Consumers’ Evaluation of Web-Based Health Information Quality: Meta-analysis

Consumers’ Evaluation of Web-Based Health Information Quality: Meta-analysis

Consumers’ Evaluation of Web-Based Health Information Quality: Meta-analysis

Authors of this article:

Yan Zhang1, 2 Author Orcid Image ;   Yeolib Kim3 Author Orcid Image
Article Authors Cited by (9) Tweetations (4) Metrics

    Review

    1School of Information, The University of Texas at Austin, Austin, TX, United States

    2Center for Health Communication, Moody College of Communication and Dell Medical School, The University of Texas at Austin, Austin, TX, United States

    3Graduate School of Technology & Innovation Management, School of Business Administration, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea

    *all authors contributed equally

    Corresponding Author:

    Yeolib Kim, PhD

    Graduate School of Technology & Innovation Management

    School of Business Administration

    Ulsan National Institute of Science and Technology

    50 UNIST-gil

    Ulsan, 44919

    Republic of Korea

    Phone: 82 52 217 3189

    Email: yeolib.kim@unist.ac.kr


    Background: The internet has become a major source of health information for general consumers. Web-based health information quality varies widely across websites and applications. It is critical to understand the factors that shape consumers’ evaluation of web-based health information quality and the role that it plays in their appraisal and use of health information and information systems.

    Objective: This paper aimed to identify the antecedents and consequences of consumers’ evaluation of web-based health information quality as a means to consolidate the related research stream and to inform future studies on web-based health information quality.

    Methods: We systematically searched 10 databases, examined reference lists, and conducted manual searches. Empirical studies that investigated consumers’ evaluation of web-based health information quality, credibility, or trust and their respective relationships with antecedents or consequences were included.

    Results: We included 147 studies reported in 136 papers in the analysis. Among the antecedents of web-based health information quality, system navigability (ρ=0.56), aesthetics (ρ=0.49), and ease of understanding (ρ=0.49) had the strongest relationships with web-based health information quality. The strongest consequences of web-based health information quality were consumers’ intentions to use health information systems (ρ=0.58) and satisfaction with health information (ρ=0.46). Web-based health information quality relationships were moderated by numerous cultural dimensions, research designs, and publication moderators.

    Conclusions: Consumers largely rely on peripheral cues and less on cues that require more information processing (eg, content comprehensiveness) to determine web-based health information quality. Surprisingly, the relationships between individual differences and web-based health information quality are trivial. Web-based health information quality has stronger effects on cognitive appraisals and behavioral intentions than on behavior. Despite efforts to include various moderators, a substantial amount of variance is still unexplained, indicating a need to study additional moderators. This meta-analysis provides broad and consistent evidence for web-based health information quality relationships that have been fractured and incongruent in empirical studies.

    J Med Internet Res 2022;24(4):e36463

    doi:10.2196/36463

    Keywords

    online health information; information quality; credibility; trust; consumer health information behavior; meta-analysis 


    The internet has become a major source of health information for general consumers. However, health information quality (IQ) varies widely across websites and web applications, and the overall quality is concerning [1,2]. Low-quality information conveys incomplete, inaccurate, or outdated knowledge, which may lead users to form erroneous health beliefs and cause negative, or even detrimental, health outcomes. Owing to the immense ramifications, web-based health IQ has attracted continued attention from researchers, health care professionals, and consumers alike.

    The IQ construct has been defined in a disparate fashion. Some researchers have taken an objective view, defining IQ in relation to currently accepted medical guidelines [3]. Others recognized that the evaluation of IQ is contingent on users’ tasks, goals, and value judgments [4-6] and defined IQ, from a subjective view, as users’ perceptions of IQ [7] or “fitness for use” [8]. For the purpose of this review, we adopted the view of IQ in the study by McKinney et al [7] and defined web-based health IQ as users’ perceptions of the quality of health information on the internet. In the internet context, two other concepts share this notion: credibility and trust. Credibility is often defined as perceived IQ, whereas trust denotes users’ willingness to trust web-based information [9].

    Some researchers have differentiated these 3 concepts. For instance, some view IQ as a dimension of credibility or a factor that influences credibility judgment [10], whereas others view credibility as a major dimension of IQ [11]. Some view IQ [12-14] or credibility [15] as antecedents of trust, whereas others view trustworthiness as a major dimension of credibility [16]. Despite these differences, the 3 concepts are intertwined. In the literature on consumers’ web-based health information seeking, they all, to some degree, refer to consumers’ perceived quality of web-based health information [17,18]. To achieve comprehensive coverage of the literature, we included studies that used any of the 3 terms to refer to health consumers’ perceptions of web-based health IQ.

    Systematic reviews concerning IQ, trust, and credibility of web-based health information have recently been published. Sun et al [19] identified the criteria and indicators that consumers use to evaluate web-based health IQ. Sbaffi and Rowley [18] identified factors that affect consumers’ trust in and the perceived credibility of web-based health information. Kim [20] identified antecedents of trust in web-based health information. On a related note, Diviani et al [17] examined the relationship between health literacy and consumer evaluation of web-based health information. These reviews provide a comprehensive view of how consumers evaluate web-based health IQ and outline categories of antecedents of web-based health IQ, such as individual factors (eg, sociodemographic and health status), source factors (eg, reputation), content factors (eg, relevance and usefulness), and design factors (eg, layout and ease of use).

    However, these reviews have several limitations. First, a plethora of antecedents of web-based health IQ was identified; however, few syntheses and comparisons were performed, resulting in a rather murky view of the most influential antecedents and how they affect web-based health IQ evaluation. Second, little effort was made to amalgamate the consequences of web-based health IQ. Third, little effort was made to explain inconsistent results across studies. For example, health literacy (and education levels and other skill-based proxies for health literacy) had a significant positive effect on perceived web-based health IQ and trust in some studies [21-24] but a negative [25-27] or insignificant [28-30] effect in others. These inconsistent results indicate that web-based health IQ relationships may be moderated by contextual factors [31].

    To further enhance our knowledge of the existence, nature, and magnitude of web-based health IQ relationships and elucidate the conceptual and practical significance of the concept [32], we performed a meta-analysis to address the following research questions: (1) what antecedents and consequences are relevant to consumers’ evaluations of web-based health IQ, and (2) what moderators intervene in web-based health IQ relationships?


    Search Strategy

    A systematic search of the literature published since 2000 was performed in July 2020 on 10 databases (eg, PubMed, CINAHL, and PsycINFO), using the search query health information AND (credibility OR quality OR reliability OR trust) AND (online OR Internet OR web) within the title, abstract, and keyword fields of these databases. In addition, we tracked the references of the included papers using Google Scholar. To reduce publication bias, we also searched the ProQuest dissertation and thesis database and reviewed the proceedings of several related conferences.

    Inclusion Criteria, Exclusion Criteria, and Screening

    The studies included in this review were empirical studies that reported effect sizes for web-based health IQ relationships. Studies were excluded if they met the following criteria: (1) focused on health care providers, (2) used qualitative research methods, (3) studied patients’ or consumers’ perceptions of the quality of information from noninternet sources (eg, health care providers and newspapers), (4) were not independent samples, (5) did not report effect sizes, (6) only reported significant results, and (7) were not in English.

    Unique records resulting from the search were screened against the inclusion and exclusion criteria. First, two reviewers (YZ and SS) independently reviewed the titles and abstracts of the 100 randomly selected records. The full text was retrieved and perused when a decision could not be reached based on the title and abstract. The intercoder agreement was moderate (Cohen κ=0.51). Discrepancies were discussed, and we clarified the inclusion and exclusion criteria. Then, the 2 coders independently coded another 50 randomly selected records. The intercoder agreement reached 0.71. Discrepancies were discussed and resolved again. SS then screened the rest of the records. The screening was purposely kept broad to avoid missing relevant studies. The overall process resulted in 273 papers. YK reviewed the full text of these papers and further excluded 142. Relevant papers from related conference proceedings were added, resulting in a final sample of 136 papers, which reported 147 studies. The paper screening and identification procedures are illustrated in Figure 1. The list of studies is reported in Multimedia Appendix 1.

    ‎
    Figure 1. Paper screening and identification procedure.
    View this figure

    Data Extraction and Meta-analytic Approach

    One of the reviewers (YK) extracted and coded the following data from the included studies using Microsoft Excel:

    1. Basic paper features: title, publication outlet, author, publication year, and publication type (journal and nonjournal)
    2. Research design: stimulus type (specific vs general), technology context (social media vs nonsocial media), sample size, sample clinical status (patient vs nonpatient), sample type (student vs nonstudent), operationalization (quality vs credibility vs trust), sample year, number of instrument items, measurement reliability, sample country, sample culture dimensions, and study methods (survey vs experiment); the values for cultural dimensions were obtained by inputting the sample country into the website of Hofstede [33]
    3. Antecedents and consequences: antecedent and consequence variables (when authors of the included papers used different terms to describe the same or similar concept, the terms were grouped under a preferred name; eg, the construct of direct experience with cancer in the study by Feng and Yang [34] and the construct of perceived severity of mental health in the study by McKinley and Ruppel [35] were coded as health experience and beliefs; constructs were categorized as antecedents or consequences as in the original studies), reliability scores when available, and effect sizes for specific antecedents and consequences (ie, correlations, odds ratio, β, chi-square, F statistic, and t statistic; the latter 4 were subsequently converted into correlations using formulas [36-38]).

    YZ reviewed the coded data against the original full-text papers to ensure accuracy and consistency. The interrater agreement (Cohen κ) reached 0.93 for basic paper features and research design, 0.87 for grouping concepts, and 0.91 for effect sizes. Disagreements were discussed and resolved. Interested readers can contact the authors to obtain the meta-analysis database.

    Following the methods by Hunter and Schmidt [36], this meta-analysis used a random-effects model to analyze correlations (rs). Weighted mean correlations (ρ or main effects) were computed by correcting for measurement and sampling errors. Reliabilities from each study were used to correct measurement errors. In studies that did not report a reliability value, the mean reliability (Multimedia Appendix 2) was used as the substitute. Reliability was set at 1.00 for variables assumed to have no measurement error (eg, gender, age, education, income, and race). Sample sizes were used to correct for sampling errors. Various supporting statistics such as the 95% CI, 90% credibility interval, Q statistic, I2 statistic, and Begg test were computed in addition to ρs. Heterogeneity was detected if the Q statistic was significant (P<.05), the I2 was >75%, or the 90% credibility interval was wide. The Begg test [39] exposes where publication bias exists in the meta-analysis via funnel plot asymmetry, whereby P<.05 implicates publication bias.

    Informed by prior meta-analyses on relevant topics [40-42] and the characteristics of the included studies, we examined three categories of moderators—cultural, research design, and publication—and the operationalization of web-based health IQ (quality, credibility, and trust), resulting in a total of 13 factors (Table 1).

    All moderators were categorical; thus, subgroup analyses using a random-effects model [43] were conducted to calculate the mean ρs. QM, an omnibus test, was calculated to statistically compare subgroup means. Antecedents and consequences with a sufficient number of observations (≥20) were analyzed against moderators. Those without sufficient observations were combined into composite variables for the analysis based on conceptual similarities. The metafor package [44] in R was used to analyze the main and moderating effects.

    Table 1. Moderators for web-based health information quality relationships.
    ModeratorDefinition and operationalization
    Sample cultureThis refers to the culture that sample participants belong to. It is operationalized by 5 cultural dimensions outlined in the cultural dimension theory by Hofstede [45].

    		Individualism versus collectivismIndividualism is “a preference for a loosely-knit social framework” where people are supposed to take care of only themselves or their close family members [46]. Collectivism represents a preference for in-group loyalties. People in a collectivistic society must unconditionally be in service to other in-group members to show their loyalty [47].

    		Power distanceThis refers to the degree to which “the less powerful members of a society accept and expect that power is distributed unequally” [46]. In a society with high power distance, people “accept a hierarchical order in which everybody has a place and which needs no further justification.” In a society with low power distance, people strive to “equalize the distribution of power and demand justification for inequalities of power” [46].

    		Uncertainty avoidanceThis expresses “the degree to which members of a society feel uncomfortable with uncertainty and ambiguity.” Societies with strong uncertainty avoidance are “intolerant of unorthodox behavior and ideas” [46], whereas societies with weak uncertainty avoidance exhibit a more relaxed attitude [48].

    		Long-term versus short-term orientationA society with a long-term orientation fosters virtues oriented toward future rewards, in particular, perseverance and thrift [48]. A society with a short-term orientation “prefers to maintain time-honored traditions and norms while viewing societal change with suspicion” [46].

    		Indulgence versus restraintIndulgence stands for a society’s tendency to allow “relatively free gratification of basic and natural human desires related to enjoying life and having fun.” Restraint stands for “a society that suppresses gratification of needs and reregulate it by strict social norms.” [46,48].
    Research designThis refers to the study’s research methods to address research problems, including research settings, data collection, measurement, and the analysis of data.

    		Technology contextThis refers to the internet technology platforms where a study situates their examination of web-based health information quality. The technology context was categorized into social media (eg, web-based health communities, Twitter, and Facebook) and non–social media (ie, general health websites).

    		Sample typeThis refers to whether a sample comprises students or nonstudents.

    		Sample clinical statusThis refers to people who assume to have no specific conditions or patients who have been diagnosed with particular conditions.

    		Study methodsThis refers to the research methods that a study used to collect data. Two specific research methods were frequently used and thus coded for this meta-analysis: survey and experiment.

    		Stimulus typeThis refers to the stimuli used in the included studies. Two types of stimuli were identified: general and specific. General stimuli are web-based health information in general (without specifications of information source and content). Specific stimuli are specific health information or health information systems (eg, a specific health website or a specific health message).
    Publication

    		Publication outletThis refers to the venue where a study was published. Two types of publication outlets were defined: journal and nonjournal (including conference proceedings and theses and dissertations).

    		TimeThis refers to when a study was published. Two periods were defined—before 2014 and in or after 2014—by applying the median split on the publication year.
    Operationalization of web-based health information qualityThis refers to the three focal concepts included in the analysis: web-based health information quality, credibility, and trust in web-based health information.

    Basic Characteristics of the Included Papers

    The 136 papers included 109 (80.1%) journal articles, 20 (14.7%) conference papers, and 7 (5.2%) theses and dissertations. The publication years ranged from 2000 to 2020, with 75% (102/136) of the papers published after 2010. The health domains covered included both general and specific health topics (eg, schizophrenia, cancer, HIV, and prescription medications).

    The included papers reported 147 independent studies. Sample sizes ranged from 34 to 8586 (median 252); 67.3% (99/147) of samples involved nonstudents, 32.7% (48/147) involved students, 8.8% (13/147) of samples were patients, and 91.2% (134/147) were nonpatients. Among the 133 samples that reported countries (15 countries), 76 (57.1%) were from the United States, followed by 16 (12%) from China, 10 (7.5%) from Korea, 8 (6%) from Germany, and 5 (3.8%) from Australia.

    Antecedents and Consequences of Web-Based Health IQ

    Table 2 presents 18 antecedents and 8 consequences of web-based health IQ with at least 6 observations. Those with the number of samples <6 were not included in the analysis as the results tend to be less generalizable [41]. The antecedents fell into four categories: individual difference, source, content, and design. The consequences fell into three categories: cognitive appraisals, behavioral intentions, and behaviors.

    Table 3 presents the main effects of the antecedents and consequences. Using the Cohen criteria [49] for judging the magnitude of correlation effect sizes, the design factor—navigability—was most strongly related to web-based health IQ (ρ=0.56), followed by the other design factor—aesthetics (ρ=0.49)—and a content factor—ease of understanding (ρ=0.49). Four other factors—source trustworthiness (ρ=0.28), health knowledge (ρ=0.15), internet experience (ρ=0.13), and social endorsement (ρ=0.10)—showed significant but weak relationships with web-based health IQ.

    On the basis of the Begg test, which takes into account publication bias (Begg P=.02), and using the trim-and-fill method [50] with 10 imputed studies on the right side of the funnel plot, age had a significant association with web-based health IQ (ρ=0.27; 95% CI 0.06-0.48; Q=3753.86). Thus, the age and web-based health IQ relationship changed from nonsignificant to significant, with individuals who were older rating the web-based health IQ higher than those who were younger. The remaining factors were not significantly related to web-based health IQ.

    Regarding consequences, the web-based health IQ exerted the strongest effect on intentions to use health information systems (ρ=0.58). Its relationship with intentions to use health information was also significant but not as strong (ρ=0.37). Web-based health IQ’s relationships with cognitive appraisal factors were mostly moderate, with the effect size for satisfaction being the largest (ρ=0.46). Web-based health IQ was moderately related to health information seeking (ρ=0.30) and did not have a significant relationship with health information use.

    Across the results of the main effects, Q statistics were substantial, indicating that the effect size distribution was heterogeneous and that some variables other than subject-level sampling and measurement errors contributed to the effect size variances [51]. Confirming the Q statistics, the I2 statistics indicated wide dispersion. The credibility interval for all relationships was wide, further implying that the effect size distribution was heterogeneous.

    Table 2. Antecedents and consequences of web-based health information quality with at least 6 observations.
    VariableDefinition
    Antecedents

    		Individual differences


    		GenderThe gender of the participants included in study samples (female=1 and male=0)


    		AgeThe age of the participants included in study samples


    		EducationThe education levels of the participants included in study samples


    		IncomeThe income of the participants included in study samples


    		RaceThe race of the participants included in study samples (White=1 and non-White=0)


    		Internet experienceAn individual’s experience with using the internet, as manifested in aspects such as the length or frequency of use and the use of the range of web-based services [52]


    		Personal involvementAn individual’s perceived personal relevance of the web-based health information [53]


    		Perceived health statusIndividuals’ self-assessment of the status of their overall personal physical and mental health [54,55]


    		Condition experience and beliefsAn individual’s experience with a health condition, perceived risk for developing the condition, and perceived severity of the condition


    		Health literacyIndividuals’ ability to obtain health information from both electronic and nonelectronic sources and their ability to process, understand, and apply the obtained health information to solve health problems and make appropriate health decisions [56,57]


    		Health knowledgeIndividuals’ knowledge about their health problems and the care for the problems [58]

    		Source-related factors


    		Source trustworthinessThe extent to which an individual believes that a specific web-based health information provider has attributes (eg, reputation) that are beneficial to the consumer [14]


    		Source expertiseThe extent to which the source or the author of a message, webpage, or website is perceived to be capable of making correct assertions [59]

    		Content factors


    		Ease of understandingWhether the provided information is easy to understand (eg, in everyday language) and informative to users [60,61]


    		Social endorsementEndorsements from other users of a website and could be manifested in forms ranging from sharing, commenting, and rating to liking [62]


    		Content comprehensivenessWhether information provided is comprehensive, providing users with comparatively complete information (eg, necessary information to establish a medical claim, statistics, references, testimonials, source and author information, and user support information) [63,64]

    		Design factors


    		NavigabilityWhether a website has clear navigation menus and effective hyperlinks and whether the information is easy to access by searching or browsing [65,66]


    		AestheticsThe visual design of a website, including the structural features such as typography, images, color, and aesthetics (eg, whether the website is professional and appealing) [67]
    Consequences

    		Cognitive appraisals


    		AttitudesIndividuals’ evaluations of and feelings about health websites or web-based health information [64,68]


    		Perceived usefulnessThe degree to which consumers believe that using health information on the internet would enhance their health-related activities [47]


    		Perceived health benefitsThe perceived level of rewards or risks that people have about the consequences of using or acting on web-based health information [14,65]


    		Satisfaction with health informationIndividuals’ satisfaction with health websites or web-based health information

    		Behavioral intentions


    		Intentions to use health informationIndividuals’ intentions or willingness to use web-based health information to make health decisions, manage health problems, or inform health behaviors


    		Intentions to use health information systemsIndividuals’ intentions or willingness to use web-based health information systems to seek health information

    		Behavior


    		Health information seekingIndividuals’ use of web-based or offline sources to find health-related information, which is manifested in aspects such as the types of information sought and the frequency and intensity of health information seeking


    		Use health informationIndividuals’ use or application of health information (from web-based or offline sources) to make health decisions, manage health problems, or inform health behaviors
    Table 3. Antecedents and consequences of quality of web-based health information.
    FactorsSamples, nSample size, Nraρb, mean (SD)95% CI90% CVcQdI2 (%)eBegg P valuef
    Antecedents

    		Individual differences


    		Gender (female)2520,1010.040.04 (0.13)–0.04 to 0.13–0.17 to 0.25216.29g86.45.22


    		Age2023,4630.040.04 (0.13)–0.11 to 0.20–0.18 to 0.26834.84g97.15.02


    		Education1516,8740.050.05 (0.17)–0.09 to 0.20–0.23 to 0.33332.32g94.29.70


    		Internet experience1462350.120.13 (0.15)0.01 to 0.24–0.11 to 0.37186.18g91.99.75


    		Personal involvement1341710.110.13 (0.22)–0.05 to 0.31–0.23 to 0.49465.37g97.08.31


    		Perceived health status926,2070.040.04 (0.21)–0.12 to 0.19–0.31 to 0.39715.44g98.55.92


    		Income918,1770.050.05 (0.10)–0.08 to 0.19–0.11 to 0.21270.04g95.65.61


    		Race (White)914,162–0.07–0.08 (0.26)–0.52 to 0.37–0.51 to 0.352609.14g99.56.08


    		Condition experience and beliefs777720.050.05 (0.10)–0.01 to 0.11–0.12 to 0.2218.74g48.88.99


    		Health literacy636610.180.22 (0.28)–0.01 to 0.45–0.25 to 0.69298.22g97.83.27


    		Health knowledge627970.130.15 (0.11)0.07 to 0.22–0.03 to 0.3318.26g63.64.72

    		Source-related factors


    		Source trustworthiness1741540.250.28 (0.25)0.10 to 0.45–0.14 to 0.70950.38g97.78.66


    		Source expertise1359880.170.20 (0.27)–0.08 to 0.49–0.25 to 0.65649.35g97.66.44

    		Content-related factors


    		Ease of understanding1439810.410.49 (0.28)0.35 to 0.630.03 to 0.95698.28g97.90.47


    		Social endorsement722670.090.10 (0.17)0.00 to 0.19–0.18 to 0.3832.03g77.57.14


    		Content comprehensiveness713730.180.21 (0.30)–0.11 to 0.54–0.29 to 0.71549.04g97.57.56

    		Design-related factors


    		Navigability1230990.470.56 (0.33)0.44 to 0.670.02 to 1.00436.08g96.85.21


    		Aesthetics1143070.400.49 (0.26)0.30 to 0.680.06 to 0.92434.41g95.99.06
    Consequences

    		Cognitive appraisals


    		Attitudes1439340.380.43 (0.24)0.36 to 0.500.04 to 0.821871.08g88.04.75


    		Perceived usefulness1011,1100.250.29 (0.23)0.08 to 0.50–0.09 to 0.67557.55g97.78.60


    		Perceived health benefits962920.320.37 (0.16)0.26 to 0.490.10 to 0.64189.17g94.86.61


    		Satisfaction with health information933340.410.46 (0.29)0.41 to 0.51–0.01 to 0.931699.85g84.31.36

    		Behavioral intentions


    		Intentions to use health information1776630.320.37 (0.33)0.32 to 0.43–0.18 to 0.924162.85g99.23.66


    		Intentions to use health information systems816140.490.58 (0.24)0.43 to 0.720.19 to 0.97290.34g95.73.55

    		Behavior


    		Health information seeking1826,2590.250.30 (0.28)0.15 to 0.46–0.16 to 0.7612,308.96g99.59.08


    		Health information use1515,0210.210.25 (0.28)–0.00 to 0.50–0.21 to 0.711083.33g98.36.77

    aWeighted mean correlation.

    bCorrected weighted mean correlation and SD of ρ.

    c90% credibility interval.

    dHeterogeneity statistic.

    ePercentage of variation across studies that is because of heterogeneity.

    fThe Begg test for funnel plot asymmetry.

    gP<.01.

    Moderators of Web-Based Health IQ Relationships

    Substantial heterogeneity calls for moderator analyses to explain the variance. The examined moderators included culture, research design, publication factors, and one operationalization-related moderator—the focal variable. The analysis was performed on web-based health IQ’s relationships with eight factors: two individual factors—gender and age—enabled by adequate sample numbers and six composite factors—source, content, design, cognitive appraisals, behavioral intentions, and behavior—formed by combining lower-level antecedents and consequences to offer adequate observations for the analysis. For the moderator analysis involving age and web-based health IQ, we did not include the 10 imputed studies, given that incorporating simulated data can distort the subgroup comparison. Table 4 presents the subgroup mean values and QM statistics. Other relevant statistics (95% CI, 90% credibility interval, QE, and R2) can be found in Multimedia Appendices 3-10. All moderators were significantly related to the effect size of at least one web-based health IQ relationship examined; 6 moderators significantly affected ≥3 relationships. The following interpretations focused on subgroups with significant differences.

    Culture moderated the three antecedents of web-based health IQ: gender, age, and source. Females in individualistic (ρ=0.06 vs ρ=–0.11), low power distance (ρ=0.06 vs ρ=–0.02), and high uncertainty avoidance (ρ=0.08 vs ρ=–0.03) cultures rated web-based health IQ higher than males. Older individuals in low uncertainty avoidance (ρ=0.21 vs ρ=–0.07) and indulgence cultures (ρ=0.19 vs ρ=–0.05) rated web-based health IQ higher. Individuals with high uncertainty avoidance (ρ=0.37 vs ρ=0.20), long-term orientation (ρ=0.32 vs ρ=0.19), and restraint (ρ=0.37 vs ρ=0.17) cultures exhibited a stronger source and web-based health IQ relationship.

    Culture moderated two consequences of web-based health IQ: cognitive appraisals and behavioral intentions. Individuals in long-term cultures had higher cognitive appraisals of web-based health IQ (ρ=0.40 vs ρ=0.27). Individuals with low uncertainty avoidance (ρ=0.59 vs ρ=0.41), short-term orientation (ρ=0.80 vs ρ=0.43), and indulgence cultures (ρ=0.60 vs ρ=0.43) had higher behavioral intentions as a result of the web-based health IQ than individuals in their respective counterpart cultures.

    Research design moderated two antecedents of web-based health IQ: gender and content. Women rated the web-based health IQ higher in studies using the survey method (ρ=0.06 vs ρ=–0.06), non–social media technology context (ρ=0.06 vs ρ=–0.10), and nonpatient samples (ρ=0.09 vs 0.00). The content and web-based health IQ relationships were stronger in studies using the survey method (ρ=0.51 vs ρ=0.21), general stimuli (ρ=0.73 vs ρ=0.30), and nonstudent samples (ρ=0.43 vs ρ=0.24).

    Research design moderated three consequences of web-based health IQ: cognitive appraisals, behavioral intentions, and behavior. Studies using specific stimuli (ρ=0.48 vs ρ=0.27) and nonstudent samples (ρ=0.35 vs ρ=0.26) produced larger effect sizes for the web-based health IQ and cognitive appraisals relationship. Studies using specific stimuli (ρ=0.54 vs ρ=0.32), social media context (ρ=0.45 vs ρ=0.39), and student samples (ρ=0.53 vs ρ=0.39) reported higher behavioral intentions. Student samples also produced a larger effect size for the web-based health IQ and behavior relationship (ρ=0.66 vs ρ=0.20).

    Publication factors moderated the gender and web-based health IQ relationship. Journal articles (ρ=0.06 vs ρ=–0.08) and papers published before 2014 (ρ=0.08 vs ρ=–0.01) reported larger effect sizes than their respective counterparts. Publication year moderated web-based health IQ and cognitive appraisals and web-based health IQ and behavioral intentions, with recent publications (2014 and after) reporting lower cognitive appraisals (ρ=0.32 vs ρ=0.37) but higher behavioral intentions (ρ=0.55 vs ρ=0.25).

    The three focal variables—quality, credibility, and trust—produced significant differences in 2 web-based health IQ relationships. The quality subgroup reported a stronger design and web-based health IQ relationship than the credibility subgroup (ρ=0.58 vs ρ=0.33). The omnibus test for comparing the focal variables in the web-based health IQ and behavioral intentions was significant (QM=30.50; P<.01). Post hoc tests revealed that the significant difference was because of the trust group being higher than the quality group (QM=13.63; P<.01) and the trust group being higher than the credibility group (QM=26.85; P<.01).

    Table 4. Influence of moderators on quality of web-based health information relationships.
    ModeratorsGender (female=1)AgeSource-related factorsContent-related factorsDesign-related factorsCognitive appraisalsBehavioral intentionsBehavior
    Culture

    		Individualism versus collectivism, QMa10.50b0.323.200.030.241.882.711.67


    		Individualism, mean (k; N)c0.06 (18; 17,745)0.05 (12; 19,834)0.22 (21; 7645)0.37 (20; 5491)0.39 (9; 2127)0.31 (23; 15,533)0.43 (13; 4684)0.27 (20; 33,688)


    		Collectivism, mean (k; N)–0.11 (4; 1242)0.03 (5; 2542)0.44 (6; 1460)0.46 (4; 468)0.57 (6; 1917)0.39 (11; 4008)0.58 (7; 1977)0.37 (9; 5515)

    		Power distance, QM3.85d0.073.200.030.241.882.711.67


    		High, mean (k; N)–0.02 (5; 1901)0.01 (6; 3201)0.44 (6; 1460)0.46 (4; 468)0.57 (6; 1917)0.39 (11; 4008)0.58 (7; 1977)0.37 (9; 5515)


    		Low, mean (k; N)0.06 (17; 17,086)0.05 (11; 19,175)0.22 (21; 7645)0.37 (20; 5491)0.39 (9; 2127)0.31 (23; 15,533)0.43 (13; 4684)0.27 (20; 33,688)

    		Uncertainty avoidance, QM6.78b7.37b6.25d0.130.251.23109.01b2.02


    		High, mean (k; N)0.08 (10; 13,180)–0.07 (8; 12,795)0.37 (14; 3022)0.45 (11; 2177)0.61 (4; 969)0.31 (17; 7284)0.41 (9; 4344)0.42 (12; 14,139)


    		Low, mean (k; N)–0.03 (12; 5807)0.21 (9; 9581)0.20 (13; 6083)0.33 (13; 3782)0.43 (11; 3075)0.33 (17; 12,257)0.59 (11; 2317)0.21 (17; 25,064)

    		Orientation, QM0.572.937.21b0.520.023.88d457.96b.08


    		Long-term, mean (k; N)0.07 (15; 14,013)–0.05 (13; 14,411)0.32 (19; 4457)0.48 (15; 3551)0.52 (9; 2556)0.40 (23; 7935)0.43 (15; 5761)0.29 (17; 13,738)


    		Short-term, mean (k; N)–0.01 (7; 4974)0.22 (4; 7965)0.19 (8; 4648)0.22 (9; 2408)0.39 (6; 1488)0.27 (11; 11,606)0.80 (5; 900)0.28 (12; 25,465)

    		Indulgence versus restraint, QM0.003.86d11.47b0.130.040.37165.28b0.92


    		Indulgence, mean (k; N)0.00 (10; 6213)0.19 (6; 8974)0.17 (9; 5223)0.34 (11; 3582)0.41 (7; 1862)0.31 (13; 11,190)0.60 (8; 1704)0.20 (14; 22,776)


    		Restraint, mean (k; N)0.07 (12; 12,774)–0.05 (11; 13,402)0.37 (18; 3882)0.43 (12; 2307)0.53 (8; 2182)0.38 (20; 6945)0.43 (12; 4957)0.29 (12; 11,502)
    Methods

    		Study method, QM5.39d0.010.579.55b2.460.493.39N/Ae


    		Survey, mean (k; N)0.06 (17; 17,929)0.04 (13; 21,681)0.25 (7; 4884)0.51 (10; 3271)0.56 (14; 5795)0.34 (27; 21,314)0.40 (22; 8870)—f


    		Experiment, mean (k; N)–0.06 (8; 2172)–0.01 (7; 1782)0.22 (23; 5258)0.21 (17; 3772)0.26 (6; 897)0.41 (10; 1702)0.46 (3; 407)N/A

    		Stimulus type, QM2.130.051.319.53b0.494.74a100.36b1.04


    		General, mean (k; N)0.06 (13; 16,088)0.05 (11; 20,648)0.35 (4; 1321)0.73 (5; 822)0.60 (9; 3963)0.27 (18; 15,477)0.32 (11; 5688)0.29 (25; 35,528)


    		Specific, mean (k; N)–0.01 (12; 4013)–0.02 (9; 2815)0.22 (26; 8821)0.30 (22; 6221)0.39 (11; 2729)0.48 (19; 7539)0.54 (14; 3589)0.27 (6; 3983)

    		Technology context, QM4.55d0.063.330.72N/AN/A5.18dN/A


    		Social media, mean (k; N)–0.10 (4; 2092)0.02 (3; 1397)0.32 (10; 2011)0.24 (10; 2493)——0.45 (6; 1107)—


    		Non–social media, mean (k; N)0.06 (21; 18,009)0.04 (17; 22,066)0.21 (20; 8131)0.41 (17; 4550)N/AN/A0.39 (19; 8170)N/A

    		Sample clinical status, QM7.18bN/AN/AN/AN/AN/A0.570.04


    		Nonpatients, mean (k; N)0.09 (4; 9478)—————0.24 (3; 1646)0.11 (6; 4192)


    		Patients, mean (k; N)0.00 (21; 10,623)N/AN/AN/AN/AN/A0.44 (22; 7631)0.31 (25; 35,319)

    		Sample type, QM0.000.030.005.63d.049.20b4.69d3.71d


    		Students, mean (k; N)0.01 (7; 1978)–0.03 (3; 594)0.22 (13; 2959)0.24 (13; 2947)0.50 (4; 841)0.26 (8; 1693)0.53 (6; 1133)0.66 (9; 7008)


    		Nonstudents, mean (k; N)0.05 (18; 18,123)0.04 (17; 22,869)0.24 (17; 7183)0.43 (14; 4096)0.52 (16; 5851)0.35 (29; 21,323)0.39 (19; 8144)0.20 (22; 32,503)
    Publication

    		Outlet, QM6.41d0.020.180.732.850.41N/A0.01


    		Journal, mean (k; N)0.06 (17; 18,204)0.05 (12; 21,288)0.24 (20; 7528)0.38 (19; 5426)0.50 (13; 4751)0.33 (30; 21,454)—0.29 (26; 38,726)


    		Nonjournal, mean (k; N)–0.08 (8; 1897)–0.01 (8; 2175)0.21 (10; 2614)0.26 (8; 1617)0.54 (7; 1941)0.45 (7; 1562)N/A0.22 (5; 785)

    		Year, QM5.26d0.150.000.010.216.42d146.34b0.00


    		Before 2014, mean (k; N)0.08 (9; 12,145)0.07 (7; 16,892)0.26 (13; 5463)0.33 (8; 1684)0.52 (12; 4380)0.37 (21; 14,357)0.25 (10; 4430)0.27 (15; 24,416)


    		2014 and after, mean (k; N)–0.01 (16; 7956)–0.02 (13; 6571)0.20 (17; 4679)0.36 (19; 5359)0.52 (8; 2312)0.32 (16; 8659)0.55 (15; 4847)0.31 (16; 15,095)
    Focal variable, QM0.070.940.092.199.34b3.5930.50b4.12


    		Quality, mean (k; N)—0.04 (3; 737)0.30 (8; 1899)0.31 (6; 1632)0.58 (10; 4815)0.45 (11; 4510)0.40 (7; 3355)0.40 (6; 4095)


    		Credibility, mean (k; N)0.02 (9; 2867)0.22 (8; 7801)0.20 (18; 4859)0.29 (16; 3943)0.33 (9; 1704)0.26 (17; 9301)0.31 (8; 1577)0.19 (7; 7341)


    		Trust, mean (k; N)0.06 (9; 14,572)–0.05 (8; 14,290)0.25 (4; 3384)0.57 (5; 1468)—0.36 (9; 9205)0.45 (10; 4345)0.30 (17; 27,889)

    aOmnibus test comparing group means.

    bP<.01.

    cCell entries show subgroup means (weighted mean correlation corrected for measurement unreliability); each parenthesis contains k (number of samples) and N (total sample size).

    dP<.05.

    eN/A: not applicable; insufficient effect sizes for subgroup comparison.

    fNot available.


    Using a comprehensive meta-analytic approach, this study analyzed antecedents and consequences of consumer web-based health IQ evaluations and contextual factors that moderate the relationships based on 147 independent studies. The major findings are discussed in the following sections.

    Web-Based Health IQ Antecedents

    Consistent with systematic reviews of consumer web-based health information evaluation behavior [18,19], we identified four major categories of antecedents of web-based health IQ: individual, source, content, and design factors. Furthermore, we revealed the magnitude of the antecedents’ effect. We found that among the 18 antecedents examined, navigability (design) was the strongest predictor of web-based health IQ, followed by ease of understanding (content) and aesthetics (design). Four factors had significant but weak relationships with web-based health IQ: source trustworthiness (source), health knowledge (individual), internet experience (individual), and social endorsement (content). Age (individual) was significantly related to web-based health IQ after correcting for publication bias. However, this result needs to be viewed with caution as imputed data were generated to obtain this result. The remaining 10 antecedents were not substantially related to web-based health IQ evaluation.

    These results suggest that consumers rely prominently on peripheral cues (eg, navigability, aesthetics, and ease of understanding) and less on systematic cues (eg, content comprehensiveness) to evaluate web-based health IQ. This is consistent with the Fogg et al [69,70] findings from large-scale surveys that website design look and ease of use (including navigability) were the most prominent influencers of website credibility, exerting stronger impacts than source expertise and trustworthiness. According to dual processing models of information processing and assessment, such as the Elaboration Likelihood Model and Heuristic-Systematic Model [71,72], these results can be attributed to consumers’ lack of motivation and/or ability to evaluate web-based health IQ [73,74]. However, the results were not conclusive. First, it is possible that theoretically significant motivational and ability factors, such as personal involvement and source expertise [71,72], did not show a significant direct impact on web-based health IQ in this research because their relationships were moderated by contextual factors, which were not analyzed because of insufficient observations. Second, other theoretically and/or empirically significant influencers of web-based health IQ that are closely related to systematic information processing, such as augment strength [14] and content consistency [75,76], were not analyzed because of insufficient observations; thus, their effects were not accounted for in this research. More research is needed to elucidate the antecedent and web-based health IQ relationships.

    Consequences

    Web-based health IQ was significantly related to all the consequences identified in the research, except for health information use. The effect of web-based health IQ on behavioral intentions (particularly intentions to use health information systems) was the strongest, followed by cognitive appraisal factors (particularly satisfaction with health information). The relationship of web-based health IQ with health information–seeking behaviors was moderate, consistent with the findings of another meta-analysis of credibility and health information seeking [77].

    The information system success model posits that IQ predicts users’ intention to use or use of and satisfaction with an information system [78,79]. The model of information adoption posits that IQ determines users’ attitudes toward information (ie, usefulness) [80,81]. Empirical research in information systems has provided strong support for the IQ-satisfaction relationship [40], whereas support for the IQ-use relationship has been mixed [79]. Our meta-analyses of web-based health IQ consequences are largely consistent with these findings, suggesting that web-based health IQ is important for consumers’ intentions to use and satisfaction with web-based health information systems and information and information-seeking behavior. The 2 aforementioned models, although primarily developed and tested in organizational or individual work settings, are applicable in the context of consumers’ web-based health information seeking.

    Moderators

    Culture moderated three antecedent and web-based health IQ relationships (ie, age, gender, and source) and two web-based health IQ and consequence relationships (ie, cognitive appraisals and behavioral intentions), demonstrating itself as an important factor shaping both web-based health IQ evaluation and its consequences. However, few empirical studies have directly examined the culture and web-based health IQ relationships. Future studies should fill this gap, which is critical for informing the design of health information systems and policies that serve different cultural groups in and across nations.

    Research design factors moderated two antecedent and web-based health IQ (ie, gender and content) and all 3 consequence and web-based health IQ relationships, reinforcing the importance of careful research design in studying web-based health IQ. It is worth noting that sample type and stimulus type affected the greatest number of relationships, with student samples and studies using general stimuli reporting stronger content and web-based health IQ relationships and having lower cognitive appraisals but stronger behavioral intentions (and stronger behavior for the student samples). The results caution the use of student samples and general stimuli when studying web-based health IQ relationships. The clinical status of the sample moderated the gender and web-based health IQ relationship. It may moderate more relationships for patients’ personal involvement [72]; however, it remains inconclusive because of insufficient observations.

    Limited publication venue bias was observed as the publication outlet moderated only the gender and web-based health IQ relationship. As a proxy to detect how web-based health IQ relationships have fluctuated over time, the publication year moderated three relationships—gender and web-based health IQ, web-based health IQ and cognitive appraisals, and web-based health IQ and behavioral intentions—revealing that individuals’ cognitive appraisals of web-based health IQ lessened; however, intentions to act on the information increased over time. It is plausible that consumers are becoming more critical as arbiters of web-based health information; however, they are also becoming more receptive to web-based health information and information systems.

    The focal variables (credibility, trust, and quality) moderated two relationships—design and web-based health IQ and web-based health IQ and behavioral intentions—out of the 8 relationships examined, indicating that some theoretical and/or methodological issues exist that promulgate this effect size disparity. Studies using quality identified a larger effect size than studies that used credibility in the design and web-based health IQ relationship. This can be attributed to the fact that studies that examined the relationship viewed quality as intrinsic merit of information (eg, accuracy, argument strength, consistency, and comprehensiveness) [14,82] and credibility as perceived reliability or trustworthiness of information [64]. In such a case, we speculate that consumers had more difficulty determining IQ than credibility [83]; thus, they need to rely more on design factors to form IQ perceptions. For the web-based health IQ and behavioral intentions relationship, studies using trust produced the largest effect size, followed by studies using quality and credibility, indicating that trust most strongly predicts behavioral intentions, followed by quality and credibility. This may be because studies on web-based health IQ and behavioral intentions were more likely to consider risk and gain assessment as part of the trust formation process [84-86], such that trust showed a higher predictive power for behavioral intentions [12,14].

    Limitations

    As with all meta-analysis studies, the main effects of a small number of observations or small sample size (eg, race or health knowledge with web-based health IQ relationships) should be interpreted with caution. Insufficient observations also limit moderator analyses, whereby moderator analyses of some theoretically or practically important relationships (eg, race, personal involvement, and health literacy with web-based health IQ relationships) were not performed. Relatedly, some antecedents and consequences were combined to form high-level categories to enable moderator analyses, which inevitably masks how some important specific relationships (eg, web-based health IQ and use of health information) might be affected by moderators.

    In terms of moderator analysis, consistent with prior meta-analysis findings, student-based results were biased [79], and survey-based results produced larger effect sizes than experience-based results [41]. The most noteworthy finding concerning moderator analyses was that the three conceptualizations of web-based health IQ (ie, quality, credibility, and trust) moderated two out of the eight relationships examined (ie, design and web-based health IQ and web-based health IQ and behavior intentions), suggesting that despite a significant conceptual overlap, theoretical and/or operationalization differences exist among the 3 constructs. This result should be interpreted in light of the fact that we took a phenomenological approach, adopting the authors’ conceptualizations of web-based health IQ (quality, credibility, and trust). A detailed examination of the definitions and measures of these constructs is warranted to elucidate the differences among the concepts. A preliminary examination of the included papers revealed that not many studies provided explicit definitions of the constructs and that measures of the same construct varied, with many articles not including specific and complete measures. These observations call for future empirical studies to offer clearer definitions of the constructs and complete measures to enable a fair assessment of these concepts for future literature synthesis.

    Despite attempts to apply various moderators to explain the variance across web-based health IQ relationships, substantial variance remained. Future research should prudently select additional moderators to explain this variance. For example, health topics merit investigation as an important contextual factor with theoretical significance for studying information-seeking behavior [87,88]. Website type also merits investigation in light of recent findings that it influences how consumers apply content, design, and source factors to evaluate web-based health IQ [89,90].

    Conclusions

    On the basis of a meta-analysis of 147 empirical studies, our study confirmed that consumers’ evaluation of web-based health IQ significantly affects their cognitive appraisals of web-based health information, intentions to use web-based information systems and information, and information-seeking behavior, suggesting the important role that web-based health IQ plays in promoting health information seeking. The study also confirmed that consumers’ evaluation of web-based health IQ is shaped by source, content, design, and individual factors, with the most influential factors being design, particularly navigability and aesthetics, and ease of understanding of content. Many individual factors, such as gender, race, education, personal involvement, and health literacy, did not show significant relationships with web-based health IQ. However, moderator analyses and the residual variance after the analyses suggest that these relationships may be moderated by numerous methodological and nonmethodological moderators. Patient empowerment and active participation in health care require individuals to have equal access to high-quality health information. More studies are needed to elucidate individual factors and web-based health IQ relationships to address potential information access disparities among different user groups.

    Acknowledgments

    The authors would like to thank Stephanie Tomasky and Shijie Song for their assistance in the article search and screening processes.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    List of studies included in sample.

    DOCX File , 59 KB
    ‎
    Multimedia Appendix 2

    Mean reliabilities.

    DOCX File , 13 KB
    ‎
    Multimedia Appendix 3

    Influence of moderators on the relationship between gender and web-based health information quality.

    DOCX File , 18 KB
    ‎
    Multimedia Appendix 4

    Influence of moderators on the relationship between age and web-based health information quality.

    DOCX File , 18 KB
    ‎
    Multimedia Appendix 5

    Influence of moderators on the relationship between source-related factors and web-based health information quality.

    DOCX File , 18 KB
    ‎
    Multimedia Appendix 6

    Influence of moderators on the relationship between content-related factors and web-based health information quality.

    DOCX File , 18 KB
    ‎
    Multimedia Appendix 7

    Influence of moderators on the relationship between design-related factors and web-based health information quality.

    DOCX File , 17 KB
    ‎
    Multimedia Appendix 8

    Influence of moderators on the relationship between web-based health information quality and cognitive appraisals.

    DOCX File , 18 KB
    ‎
    Multimedia Appendix 9

    Influence of moderators on the relationship between web-based health information quality and behavioral intentions.

    DOCX File , 18 KB
    ‎
    Multimedia Appendix 10

    Influence of moderators on the relationship between web-based health information quality and behavior.

    DOCX File , 18 KB
    1. Eysenbach G, Powell J, Kuss O, Sa ER. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA 2002 May;287(20):2691-2700. [CrossRef] [Medline]
    2. Zhang Y, Sun Y, Xie B. Quality of health information for consumers on the web: a systematic review of indicators, criteria, tools, and evaluation results. J Assn Inf Sci Tec 2015 Apr 29;66(10):2071-2084. [CrossRef]
    3. Tsunehara ST, Gaster B. A critical assessment of diabetes information on the internet. Diabetes Educ 2002;28(2):186-195. [CrossRef]
    4. Eppler MJ. Managing Information Quality: Increasing the Value of Information in Knowledge-intensive Products and Processes. 2nd edition. Berlin, Germany: Springer; 2006.
    5. Hilligoss B, Rieh SY. Developing a unifying framework of credibility assessment: construct, heuristics, and interaction in context. Inf Process Manag 2008 Jul;44(4):1467-1484. [CrossRef]
    6. Taylor RS. Value-Added Processes in Information Systems. Norwood, NJ, USA: Ablex Publishing Corporation; 1986.
    7. McKinney V, Yoon K, Zahedi FM. The measurement of web-customer satisfaction: an expectation and disconfirmation approach. Inf Syst Res 2002 Sep 1;13(3):296-315. [CrossRef]
    8. Wang RY, Strong DM. Beyond accuracy: what data quality means to data consumers. J Manag Inf Syst 1996;12(4):5-33. [CrossRef]
    9. Tseng S, Fogg BJ. Credibility and computing technology. Commun ACM 1999 May;42(5):39-44. [CrossRef]
    10. Sundar SS. Exploring receivers' criteria for perception of print and online news. Journal Mass Commun Q 1999;76(2):373-386. [CrossRef]
    11. Rieh SY, Danielson DR. Credibility: a multidisciplinary framework. Ann Rev Info Sci Tech 2007;41(1):307-364. [CrossRef]
    12. Harris PR, Sillence E, Briggs P. Perceived threat and corroboration: key factors that improve a predictive model of trust in internet-based health information and advice. J Med Internet Res 2011 Jul 27;13(3):e51 [FREE Full text] [CrossRef] [Medline]
    13. Song J, Zahedi FM. Trust in health infomediaries. Decis Support Syst 2007 Mar;43(2):390-407. [CrossRef]
    14. Yi MY, Yoon JJ, Davis JM, Lee T. Untangling the antecedents of initial trust in Web-based health information: the roles of argument quality, source expertise, and user perceptions of information quality and risk. Decis Support Syst 2013 Apr;55(1):284-295. [CrossRef]
    15. Corritore CL, Wiedenbeck S, Kracher B, Marble RP. Online trust and health information websites. Int J Technol Hum Interact 2012 Oct;8(4):92-115. [CrossRef]
    16. Freeman KS, Spyridakis JH. An examination of factors that affect the credibility of online health information. Tech Commun 2004 Jun;51(2):239-263 [FREE Full text]
    17. Diviani N, van den Putte B, Giani S, van Weert JC. Low health literacy and evaluation of online health information: a systematic review of the literature. J Med Internet Res 2015 May 07;17(5):e112 [FREE Full text] [CrossRef] [Medline]
    18. Sbaffi L, Rowley J. Trust and credibility in Web-based health information: a review and agenda for future research. J Med Internet Res 2017 Jun 19;19(6):e218 [FREE Full text] [CrossRef] [Medline]
    19. Sun Y, Zhang Y, Gwizdka J, Trace CB. Consumer evaluation of the quality of online health information: systematic literature review of relevant criteria and indicators. J Med Internet Res 2019 May 02;21(5):e12522 [FREE Full text] [CrossRef] [Medline]
    20. Kim Y. Trust in health information websites: a systematic literature review on the antecedents of trust. Health Informatics J 2016 Jun;22(2):355-369 [FREE Full text] [CrossRef] [Medline]
    21. Helft PR, Eckles RE, Johnson-Calley CS, Daugherty CK. Use of the internet to obtain cancer information among cancer patients at an urban county hospital. J Clin Oncol 2005 Aug 01;23(22):4954-4962. [CrossRef] [Medline]
    22. Hesse BW, Nelson DE, Kreps GL, Croyle RT, Arora NK, Rimer BK, et al. Trust and sources of health information: the impact of the Internet and its implications for health care providers: findings from the first Health Information National Trends Survey. Arch Intern Med 2005 Dec;165(22):2618-2624. [CrossRef] [Medline]
    23. Marrie RA, Salter AR, Tyry T, Fox RJ, Cutter GR. Preferred sources of health information in persons with multiple sclerosis: degree of trust and information sought. J Med Internet Res 2013 Mar 17;15(4):e67 [FREE Full text] [CrossRef] [Medline]
    24. Neter E, Brainin E. eHealth literacy: extending the digital divide to the realm of health information. J Med Internet Res 2012 Jan 27;14(1):e19 [FREE Full text] [CrossRef] [Medline]
    25. Bernhardt JM, Felter EM. Online pediatric information seeking among mothers of young children: results from a qualitative study using focus groups. J Med Internet Res 2004 Mar 01;6(1):e7 [FREE Full text] [CrossRef] [Medline]
    26. Lawson R, Forbes S, Williams J. Patterns of trust in sources of health information. N Z Med J 2011 Jan 21;124(1328):98-105. [Medline]
    27. Nguyen GT, Bellamy SL. Cancer information seeking preferences and experiences: disparities between Asian Americans and Whites in the Health Information National Trends Survey (HINTS). J Health Commun 2006;11 Suppl 1:173-180. [CrossRef] [Medline]
    28. Murray E, Lo B, Pollack L, Donelan K, Catania J, White M, et al. The impact of health information on the internet on the physician-patient relationship: patient perceptions. Arch Intern Med 2003 Jul 28;163(14):1727-1734. [CrossRef] [Medline]
    29. Yan YY. Online health information seeking behavior in Hong Kong: an exploratory study. J Med Syst 2010 Apr;34(2):147-153. [CrossRef] [Medline]
    30. Ye Y. Correlates of consumer trust in online health information: findings from the health information national trends survey. J Health Commun 2011 Jan;16(1):34-49. [CrossRef] [Medline]
    31. Beldad A, de Jong M, Steehouder M. How shall I trust the faceless and the intangible? A literature review on the antecedents of online trust. Comput Human Behav 2010 Sep;26(5):857-869. [CrossRef]
    32. Hong W, Chan FK, Thong JY, Chasalow LC, Dhillon G. A framework and guidelines for context-specific theorizing in information systems research. Inf Syst Res 2014 Mar;25(1):111-136. [CrossRef]
    33. Compare countries. Hofstede Insights.   URL: https://www.hofstede-insights.com/fi/product/compare-countries/ [accessed 2022-02-27]
    34. Feng H, Yang Y. A model of cancer-related health information seeking on the internet. China Media Res 2007 Jul;3(3):14-24.
    35. McKinley CJ, Ruppel EK. Exploring how perceived threat and self-efficacy contribute to college students’ use and perceptions of online mental health resources. Comput Human Behav 2014 May;34:101-109. [CrossRef]
    36. Hunter JE, Schmidt FL. Methods of Meta-Analysis: Correcting Error and Bias in Research Findings. 2nd edition. Thousand Oaks, CA, USA: SAGE Publications; Jun 3, 2004.
    37. Peterson RA, Brown SP. On the use of beta coefficients in meta-analysis. J Appl Psychol 2005 Jan;90(1):175-181. [CrossRef] [Medline]
    38. Bonett DG. Transforming odds ratios into correlations for meta-analytic research. Am Psychol 2007 Apr;62(3):254-255. [CrossRef] [Medline]
    39. Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics 1994 Dec;50(4):1088-1101. [Medline]
    40. Ghasemaghaei M, Hassanein K. Online information quality and consumer satisfaction: the moderating roles of contextual factors – a meta-analysis. Inf Manag 2015 Dec;52(8):965-981. [CrossRef]
    41. Kim Y, Peterson RA. A meta-analysis of online trust relationships in e-commerce. J Interact Market 2017 May;38:44-54. [CrossRef]
    42. Ma TJ, Atkin D. User generated content and credibility evaluation of online health information: a meta analytic study. Telematics Informatics 2017 Aug;34(5):472-486. [CrossRef]
    43. Treweek S, Oxman AD, Alderson P, Bossuyt PM, Brandt L, Brożek J, DECIDE Consortium. Developing and evaluating communication strategies to support informed decisions and practice based on evidence (DECIDE): protocol and preliminary results. Implement Sci 2013 Jan 09;8:6 [FREE Full text] [CrossRef] [Medline]
    44. Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Soft 2010;36(3):1-48. [CrossRef]
    45. Hofstede G, Hofstede GJ, Minkov M. Cultures and Organizations: Software of the Mind. 3rd edition. New York, NY, USA: McGraw-Hill; May 3, 2010.
    46. National culture. Hofstede Insights.   URL: https://hi.hofstede-insights.com/national-culture [accessed 2021-05-03]
    47. Yun EK, Park HA. Consumers' disease information-seeking behaviour on the Internet in Korea. J Clin Nurs 2010 Oct;19(19-20):2860-2868. [CrossRef] [Medline]
    48. Hofstede G. Culture's consequences: comparing values, behaviors, institutions, and organizations across nations. 2nd edition. Thousand Oaks, CA, USA: Sage Publications; 2001.
    49. Cohen J. A power primer. Psychol Bull 1992 Jul;112(1):155-159. [CrossRef] [Medline]
    50. Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000 Jun;56(2):455-463. [CrossRef] [Medline]
    51. Lipsey MW, Wilson DB. Practical meta-analysis. Thousand Oaks, CA, USA: Sage Publications; 2001.
    52. Rains SA. Seeking health information in the information age: the role of internet self-efficacy. Western J Commun 2008 Mar 11;72(1):1-18. [CrossRef]
    53. Zaichkowsky JL. Conceptualizing involvement. J Advert 1986 Jun;15(2):4-34. [CrossRef]
    54. Madans JH. Health surveys. In: Smelser NJ, Baltes PB, editors. International Encyclopedia of the Social & Behavioral Sciences. 2nd edition. Amsterdam, The Netherlands: Elsevier; 2015:725-730.
    55. Gould SJ. Consumer attitudes toward health and health care: a differential perspective. J Consum Affairs 1988;22(1):96-118 [FREE Full text] [CrossRef]
    56. Institute of Medicine, Board on Neuroscience and Behavioral Health, Committee on Health Literacy. In: Nielsen-Bohlman L, Panzer AM, Kindig DA, editors. Health Literacy: A Prescription to End Confusion. Washington, DC, USA: National Academies Press; 2004.
    57. Norman CD, Skinner HA. eHealth literacy: essential skills for consumer health in a networked world. J Med Internet Res 2006 Jun 16;8(2):e9 [FREE Full text] [CrossRef] [Medline]
    58. Wilson EV, Lankton NK. Modeling patients' acceptance of provider-delivered e-health. J Am Med Inform Assoc 2004;11(4):241-248 [FREE Full text] [CrossRef] [Medline]
    59. Pornpitakpan C. The persuasiveness of source credibility: a critical review of five decades' evidence. J Appl Social Pyschol 2004 Feb;34(2):243-281. [CrossRef]
    60. Thon FM, Jucks R. Believing in expertise: how authors' credentials and language use influence the credibility of online health information. Health Commun 2017 Jul;32(7):828-836. [CrossRef] [Medline]
    61. Yang F, Li C. The color of gender stereotyping: the congruity effect of topic, color, and gender on health messages’ persuasiveness in cyberspace. Comput Human Behav 2016 Nov;64:299-307. [CrossRef]
    62. Borah P, Xiao X. The importance of 'Likes': the interplay of message framing, source, and social endorsement on credibility perceptions of health information on Facebook. J Health Commun 2018;23(4):399-411. [CrossRef] [Medline]
    63. Dutta-Bergman MJ. The impact of completeness and web use motivation on the credibility of e-health information. J Commun 2004 Jun;54(2):253-269. [CrossRef]
    64. Rains SA, Karmikel CD. Health information-seeking and perceptions of website credibility: examining Web-use orientation, message characteristics, and structural features of websites. Comput Human Behav 2009 Mar;25(2):544-553. [CrossRef]
    65. Cao W, Zhang X, Xu K, Wang Y. Modeling online health information-seeking behavior in China: the roles of source characteristics, reward assessment, and internet self-efficacy. Health Commun 2016 Sep;31(9):1105-1114. [CrossRef] [Medline]
    66. Shim M, Jo HS. What quality factors matter in enhancing the perceived benefits of online health information sites? Application of the updated DeLone and McLean Information Systems Success Model. Int J Med Inform 2020 May;137:104093. [CrossRef] [Medline]
    67. Robins D, Holmes J, Stansbury M. Consumer health information on the Web: the relationship of visual design and perceptions of credibility. J Am Soc Inf Sci 2010;61(1):13-29. [CrossRef]
    68. Dillard JP. Persuasion past and present: attitudes aren't what they used to be. Commun Monographs 2009 Jun 02;60(1):90-97. [CrossRef]
    69. Fogg BJ, Soohoo C, Danielson DR, Marable L, Stanford J, Tauber ER. How do users evaluate the credibility of Web sites?: a study with over 2,500 participants. In: Proceedings of the 2003 conference on Designing for user experiences. 2003 Jun Presented at: DUX '03; Jun 6 - 7, 2003; San Francisco, CA, USA p. 1-15. [CrossRef]
    70. Fogg BJ, Marshall J, Laraki O, Osipovich A, Varma C, Fang N, et al. What makes Web sites credible?: a report on a large quantitative study. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2001 Mar Presented at: CHI '01; Mar 31- Apr 5, 2001; Seattle, WA, USA p. 61-68. [CrossRef]
    71. Chaiken S. Heuristic versus systematic information processing and the use of source versus message cues in persuasion. J Pers Soc Psychol 1980 Nov;39(5):752-766. [CrossRef]
    72. Petty RE, Cacioppo JT. The effects of involvement on responses to argument quantity and quality: central and peripheral routes to persuasion. J Pers Soc Psychol 1984 Jan;46(1):69-81. [CrossRef]
    73. Metzger MJ. Making sense of credibility on the Web: models for evaluating online information and recommendations for future research. J Am Soc Inf Sci 2007 Nov;58(13):2078-2091. [CrossRef]
    74. Wathen CN, Burkell J. Believe it or not: factors influencing credibility on the Web. J Am Soc Inf Sci 2002;53(2):134-144 [FREE Full text] [CrossRef]
    75. Lederman R, Fan H, Smith S, Chang S. Who can you trust? Credibility assessment in online health forums. Health Policy Technol 2014 Mar;3(1):13-25. [CrossRef]
    76. Sillence E, Blythe JM, Briggs P, Moss M. A revised model of trust in Internet-based health information and advice: cross-sectional questionnaire study. J Med Internet Res 2019 Nov 11;21(11):e11125 [FREE Full text] [CrossRef] [Medline]
    77. Chang CC, Huang MH. Antecedents predicting health information seeking: a systematic review and meta-analysis. Int J Inf Manag 2020 Oct;54:102115. [CrossRef]
    78. DeLone WH, McLean ER. Information systems success: the quest for the dependent variable. Inf Syst Res 1992 Mar;3(1):60-95. [CrossRef]
    79. Petter S, DeLone W, McLean E. Measuring information systems success: models, dimensions, measures, and interrelationships. Eur J Inf Syst 2008;17(3):236-263. [CrossRef]
    80. Erkan I, Evans C. The influence of eWOM in social media on consumers’ purchase intentions: an extended approach to information adoption. Comput Human Behav 2016 Aug;61:47-55. [CrossRef]
    81. Sussman SW, Siegal WS. Informational influence in organizations: an integrated approach to knowledge adoption. Inf Syst Res 2003 Mar;14(1):47-65. [CrossRef]
    82. Purcarea VL, Gheorghe IR, Petrescu CM. Credibility elements of eWOM messages in the context of health care services. A Romanian perspective. J Med Life 2013 Sep 15;6(3):254-259 [FREE Full text] [Medline]
    83. Park JH, Chung HE, Yoo WS. Is the Internet a primary source for consumer information search?: group comparison for channel choices. J Retail Consum Serv 2009 Mar;16(2):92-99. [CrossRef]
    84. Hether HJ, Murphy ST, Valente TW. It's better to give than to receive: the role of social support, trust, and participation on health-related social networking sites. J Health Commun 2014 Dec 25;19(12):1424-1439. [CrossRef] [Medline]
    85. Lee SY, Choi J, Noh GY. Factors influencing health-related internet activities and their outcomes. J Health Commun 2016 Nov;21(11):1179-1186. [CrossRef] [Medline]
    86. Liu S, Zhang R, Lu X. The impact of individuals' attitudes toward health websites on their perceived quality of health information: an empirical study. Telemed J E Health 2019 Nov;25(11):1099-1107. [CrossRef] [Medline]
    87. Marchionini G. Exploratory search: from finding to understanding. Commun ACM 2006 Apr 01;49(4):41-46. [CrossRef]
    88. Vakkari P. Task-based information searching. Ann Rev Info Sci Tech 2003;37(1):413-464. [CrossRef]
    89. Chang YS, Zhang Y, Gwizdka J. The effects of information source and eHealth literacy on consumer health information credibility evaluation behavior. Comput Human Behav 2021 Feb;115:106629. [CrossRef]
    90. Liu J, Song S, Zhang Y. Linguistic features and consumer credibility judgment of online health information. University of Illinois. 2021.   URL: https://www.ideals.illinois.edu/handle/2142/109674 [accessed 2021-05-07]

    ‎
    IQ: information quality

    Edited by A Mavragani; submitted 15.01.22; peer-reviewed by DJ Lim, E Sillence, S Song; comments to author 05.02.22; accepted 15.03.22; published 28.04.22

    Copyright

    ©Yan Zhang, Yeolib Kim. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 28.04.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.