This paper is in the following e-collection/theme issue:

Consumer & Patient Education and Shared-Decision Making (679) Quality/Credibility of eHealth Information and Trust Issues (531) Participatory Medicine & E-Patients (773)

Published on in Vol 12, No 2 (2010): Apr-Jun

An International Comparison of Web-based Reporting About Health Care Quality: Content Analysis

An International Comparison of Web-based Reporting About Health Care Quality: Content Analysis

An International Comparison of Web-based Reporting About Health Care Quality: Content Analysis

Authors of this article:

Olga C. Damman1 ;   Ylva K.A. van den Hengel2 ;   A. Jeanne M. van Loon2 ;   Jany Rademakers1 Author Orcid Image
Article Authors Cited by (29) Tweetations Metrics

    Original Paper

    1NIVEL (Netherlands Institute for Health Services Research), Utrecht, the Netherlands

    2RIVM (National Institute for Public Health and the Environment), Bilthoven, the Netherlands

    Corresponding Author:

    Olga C Damman, MSc

    NIVEL (Netherlands Institute for Health Services Research)

    P.O. Box 1568, 3500 BN

    Utrecht

    the Netherlands

    Phone: 31 30 272 97 00

    Fax:31 30 272 97 29

    Email: o.damman@nivel.nl


    Background: On more and more websites, consumers are provided with public reports about health care. This move toward provision of more comparative information has resulted in different information types being published that often contain contradictory information.

    Objective: The objective was to assess the current state of the art in the presentation of online comparative health care information and to compare how the integration of different information types is dealt with on websites. The content analysis was performed in order to provide website managers and Internet researchers with a resource of knowledge about presentation formats being applied internationally.

    Methods: A Web search was used to identify websites that contained comparative health care information. The websites were systematically examined to assess how three different types of information (provider characteristics and services, performance indicators, and health care user experience) were presented to consumers. Furthermore, a short survey was disseminated to the reviewed websites to assess how the presentation formats were selected.

    Results: We reviewed 42 websites from the following countries: Australia, Canada, Denmark, Germany, Ireland, the Netherlands, Norway, the United Kingdom, the United States, and Sweden. We found the most common ways to integrate different information types were the two extreme options: no integration at all (on 36% of the websites) and high levels of integration in single tables on 41% of the websites). Nearly 70% of the websites offered drill down paths to more detailed information. Diverse presentation approaches were used to display comparative health care information on the Internet. Numbers were used on the majority of websites (88%) to display comparative information.

    Conclusions: Currently, approaches to the presentation of comparative health care information do not seem to be systematically selected. It seems important, however, that website managers become aware of the complexities inherent in comparative information when they release information on the Web. Important complexities to pay attention to are the use of numbers, the display of contradictory information, and the extent of variation among attributes and attribute levels. As for the integration of different information types, it remains unclear which presentation approaches are preferable. Our study provides a good starting point for Internet research to further address the question of how different types of information can be more effectively presented to consumers.

    J Med Internet Res 2010;12(2):e8

    doi:10.2196/jmir.1191

    Keywords

    Consumer health information; information display; decision making; Internet; international comparison; content analysis 


    Public reporting of comparative health care information has become increasingly important in several countries. Comparative health care information is information by which consumers can make explicit comparisons between the performances of health care providers or health plans in order to make an informed choice. In the United States and the United Kingdom, efforts to make this kind of information publicly available have been ongoing for about fifteen years. The aims are to increase public accountability and to support consumer choice in health care and indirectly to improve the quality of health services. Health care policy in the Netherlands currently focuses on transparency as well: health care consumers are encouraged to make use of public comparative information about health care services and quality [1,2]. Dutch consumers have been provided with public reports of health care information in newspapers and magazines since the late 1990s. In addition, comparative health care information has been published on the Internet in the Netherlands for the past few years.

    The number of websites containing comparative health care reports is rapidly growing worldwide. This number will continue to rise given the increased tendency of many health care systems to become publicly accountable and to use market-based approaches. In addition, existing websites likely will offer more types of information as well as information about different health care sectors in order to support health care consumers’ decision making. When we look at various websites, no standard approaches for presenting information seem to emerge. Carlisle [3] examined ten American websites and concluded that “each is unique in presentation of grades and how the grades are tabulated.” However, based on laboratory studies on human decision making, it is known that information presentation formats influence consumers’ responses [4,5]. Therefore, it is necessary to reflect on and learn from the presentation approaches used in different countries within the rapidly growing movement of public health care reporting.

    In fact, presentation formats of comparative health care information have been steadily gaining attention. Poor information presentation is frequently cited in the literature as one reason that this kind of information is rarely used by consumers [6-8]. Despite several years of international experience, there is little evidence that health care reports support consumer decision making [9-11]. Many researchers have suggested that the information presented is too complex for consumers and is not adjusted to consumers’ cognitive processing and decision making strategies. In a recent review, Fung and colleagues [8] concluded that “[d]espite its theoretical appeal, making public reporting work requires successfully addressing several challenges, most notably designing and implementing a reporting system appropriate for its purpose.”

    One of the difficulties that consumers may face is the large amount of information on Web pages, which is often overwhelming [12]. It is known that consumers can only process a few “chunks” of information simultaneously [13] and are easily overloaded by information [14,15]. Consumers’ attitudes toward the amount of information on websites are somewhat mixed: higher numbers of features on websites have been associated with both positive [16,17] and negative [18] attitudes. In this context, the number of different types of information and the structure in which these are presented are important [19].

    With respect to public comparative health care information, van Loon and Tolboom [20] defined three different information types. The first type is information about the characteristics and services provided by individual health care providers and health care facilities. This is factual information about providers’ names, addresses, and the geographic region in which health care is provided as well as information about the type of provider (eg, academic or non-academic hospital), provider specialty, available facilities, provider’s religion, costs of services, and waiting times. The second information type is information about quality of health care based on performance indicators, usually derived from existing provider registrations (ie, administrative records) or registrations required by governments and established for public reporting purposes. These concern medical and health care performance information based on relatively factual information relating to a particular health facility such as the number of patients with pressure wounds or the number of operations of a particular type. The third information type is quality information based on health care user experience. Like the second information type, this information type concerns health care performance. However, in this case, the data are derived from patient surveys. For example, patients or clients are surveyed about their experiences with the treatment in the hospital or about their satisfaction with the food or privacy in the nursing home. Within each of the three types of information, several subtypes can be distinguished as well, such as general quality indicators and more specific underlying aspects of care.

    Using different information types and various indicators to make a decision is known to be a difficult cognitive process [5,21]. Moreover, as the amount of information on a Web page increases, a simple information structure combined with high usability is, almost inevitably, not attainable. Apart from the fact that more information types will increase the amount of comparative information, presenting different information types can be complex in itself. For example, it is a complex task for consumers to make a choice when a health care provider performs well on one specific quality aspect but badly on another. It can become even more complex when indicators stemming from different information types are contradictory although they concern the same aspects of care. This can be the case when quality information is drawn from both hospitals’ administrative records and patient surveys. A hospital’s registration may indicate, for example, that patients have the opportunity to participate in the decision for a particular type of anesthetic. This would be reflected by the score “yes” on the quality indicator “patient participation in choice of anesthetic.” Despite this, results of a patient survey may show that patients reported negative experiences concerning participation in decision making. For example, if patients at a particular hospital were more negative compared with patients at other hospitals, the first hospital’s performance would be given a lower rating. The question is how consumers are supposed to deal with these kinds of complexities. We know that consumers may respond differently to information depending on its complexity [22-24]. For example, the number of contradictions in the information increases information complexity, which can affect decision making accuracy [24].

    An additional difficulty might emerge when different information types are presented by different information displays, such as numbers versus stars. It is unknown whether inconsistent information displays further hamper consumers’ ability to process comparative health care information.

    Information display, such as words, numbers and symbols, may be another source of difficulty for consumers. In an early review of McCormack and colleagues [25], concerned largely with offline comparative health care information, the dominant presentation approaches consisted of combinations of text and graphics or text and percentages in a table format. The use of numbers may lead to confusing and overwhelming information display. Consumers may not have an emotional or affective understanding of numbers and the information may therefore be less “evaluable,” a term coined by Hibbard to refer to the ease or precision with which the values of the attributes across alternatives create an affective (good/bad) feeling [21,27]. Hibbard and colleagues demonstrated that visual display in the form of stars facilitated consumers’ comprehension and use of comparative health care information [26,27]. Previous research has also shown that the readability of text is important for consumers [28,29], and that health information on the Internet is often beyond consumers’ reading ability [30,31].

    In short, the large amount and variety of information as well as how the information is presented are important issues pertaining to the publication of comparative health care information. Indeed, these issues have been cited in the literature since the early days of offline and online public reporting [3,25]. In the past decade, the number of different information types has increased, largely due to emerging information technology. Apart from a few reviews of information types presented and presentation formats used [3,19,25], no comprehensive reviews have been conducted recently. More importantly, since more countries are adopting a public reporting system for health care information, it is of interest to document which strategies are applied in countries outside the United States to present different types of health care information. If we want to understand the decision maker’s current health care information environment and be able to simplify it, an up-to-date overview of what consumers are actually confronted with is needed.

    The aim of the present paper was to describe how different types of information are presented on websites containing public comparative health care information. Our primary concern was the structure used to integrate different information types. We further reviewed the drill down paths offered on websites and how information was displayed. Drill down paths are paths that provide options to get more detailed information that may also be used to structure the total amount of available information. Information display can make information more valuable to consumers. Our intention was not to review all of the websites that exist worldwide but rather to provide an overview of the state of the art that can be used as a resource of knowledge for website managers and Internet researchers. Our research question was: “How are different types of Web-based comparative health care information presented worldwide?”


    Search Strategy

    This study was conducted from July to September 2008. Two key Dutch publications on public reporting of health care were used as a starting point to search for websites providing health care information to the public [32,33]. These studies only included countries in which both performance indicators and public reporting have been incorporated in the health care system. We then performed a search using the Web search engine Google for particular terms and their translations in English, German, French, Spanish, Italian, Dutch, Norwegian, Swedish, Danish, and Finnish. The terms chosen were: quality, quality indicators, health care, compare, choose, information, patients, consumers, satisfaction, health plans, hospitals, nursing homes, home care, and mental health care. We included only websites that contained comparative information, that is, information by which consumers can make explicit comparisons between health care providers or health plans. For websites where information for health care providers was presented separately, we reviewed only the comparative information. We chose to do this because, as stated previously, comparative information is intended to facilitate consumer choice in health care.

    Analyses

    We visited the selected websites and assessed the presentation approaches that were used. The following aspects were systematically considered: (1) the health care sector(s) for which information was presented; (2) the types of information presented; (3) the degree of integration of different information types; (4) the drill down paths provided; and (5) the information displays used.

    For types of information, we followed the classification system of Van Loon and Tolboom [20] for public health care information: “A” indicated factual information based on provider characteristics and services; “B” indicated quality information based on performance indicators; and “C” indicated quality information based on health care user experience. The degree to which different information types were displayed in an integrated way was also assessed. In the absence of a ready taxonomy of classifying presentation formats, we classified information integration as: “0” to mean no integration, that is, different information types on different pages; “1” to mean limited integration, that is, different types of information on one page, but no integration in a single table; “2” to mean a medium amount of integration, that is, different information types on one page but clearly separated from each other; and “3” to mean a high level of integration, that is, different information types were presented in a single table. Drill down paths were assessed qualitatively according to the different approaches on the websites; we used no particular classification system. Finally, we reviewed the display of information and focused on the use of words, numbers, bar graphs, and different types of symbols. All analyses and coding activities were performed by two of the authors (OD and YH) independently. They discussed their findings and searched for agreement.

    Besides reviewing the website content, we disseminated a short survey to each website included in this study. This survey contained open and closed questions about which types of information the website presented and how the presentation formats were chosen. The survey was either directly mailed to the website (in case a direct contact address was found on the website) or delivered indirectly by contacting the website through a request form. Respondents could return the completed survey to the researchers by email or by post.


    Search Results

    In total, we found 42 websites in 10 different countries that presented comparative health care information. Table 1 gives a short description of each website. Most websites we identified were in the United States, although we also found a range of websites in the United Kingdom, Germany, and the Netherlands. The aim of most reporting systems was to inform consumers about health care performance and to support consumers’ choices. A few websites were not explicitly designed for consumers, but because these websites were intended to increase public accountability and were accessible for consumers, we included them in the current study.

    Table 1. Brief descriptions of reviewed websites
    Country and WebsiteaURL (Archived WebCite URL)bDescription
     
     
    Australia
    1. Your Hospitalshttp://www.health.vic.gov.au/yourhospitals
    (http://www.webcitation.org/5clVd3AEQ)    		Initiative of the Consumer Participation and Information Program. The aim is to provide information to patients, caregivers, and health care professionals. The information is generated by the Department of Health, its funded agencies, and special interest groups.
     
    Canada
    2. Hospital Reporthttp://www.hospitalreport.ca
    (http://www.webcitation.org/5clVfMnoX)    		Initiative of the HHRC (Hospital Report Research Collaborative). The aims are to increase public accountability and to improve quality of care.
     
    Denmark
    3. Sundhedhttp://www.sundhed.dk
    (http://www.webcitation.org/5clVsUhI3)    		Initiative of the Danish Ministry of Health. The reporting system ‘Sundhedkvalitet’ is managed by the National Board of Health. The aim is to support consumers in their health care choices.
     
    Germany
    4. Weisse Listehttp://www.weisse-liste.de
    (http://www.webcitation.org/5clW384wA)    		Initiative of the Bertelsmann Stiftung in collaboration with patient associations and scientific partners. The aims are to empower consumers and to support them in their health care choices.
    5. Klinik Führer Rhein-Ruhrhttp://www.kliniken-rhein-ruhr.de
    (http://www.webcitation.org/5clW4zeOf)    		Initiative of the Initiativkreis Ruhrgebiet Verwaltungs-GmbH (a collaborative of hospitals) in collaboration with scientific partners. The aim is to support consumers in their health care choices. The information is generated from the hospitals and from patient surveys.
    6. Klinikführer
    Rheinland    		http://www.klinikfuehrer-rheinland.de
    (http://www.webcitation.org/5clWCp5WX)    		Initiative of the Krankenhauszweckverband Köln, Bonn, und Region (KHVZ) (a collaborative of hospitals). The aim is to support consumers in their health care choices. The information is generated from the hospitals by the KHVZ.
    7. Hamburger
    Krankenhaus-spiegel    		http://www.hamburger-krankenhausspiegel.de
    (http://www.webcitation.org/5clWR10vD)    		Initiative of 25 hospitals in collaboration with other partners. The aim is to support consumers in their health care choices, and to stimulate providers’ quality improvement initiatives. The information is generated from the hospitals by independent audit parties.
    8.Klinikbe-
    wertungen    		http://www.klinikbewertungen.de
    (http://www.webcitation.org/5clWVbRiG)    		Initiative of MedizInfo, which is an Internetportal about health and health care. The aim is to provide an independent online forum about consumers’ experiences in order to help consumers in their health care choices. A second aim is to stimulate providers’ quality improvement initiatives. The information is generated from consumers’ reports on the forum.
     
    Ireland
    9. Health Information and Quality Authorityhttp://www.hiqa.ie
    (http://www.webcitation.org/5clZNb8km)    		Initiative of the Health Information and Quality Authority (part of the government’s health reform program). The aims are to monitor quality of care on a set of standards and to stimulate improvement initiatives. A third aim is to help consumers in their health care choices.
     
    The Netherlands
    10. kiesBeterhttp://www.kiesBeter.nl
    (http://www.webcitation.org/5clT0whdn)    		Initiative of the Ministry of Health and managed by the National Institute for Public Health and the Environment (RIVM) in collaboration with patient associations, health care providers, and scientific partners. The aim is to provide an independent portal for all questions from the public about health and health care. One particular aim is to support consumers in their health care choices.
    11. Independer
    Gezondheids-zorg    		http://www.independer.nl
    (http://www.webcitation.org/5clU1lwM3)    		Initiative of Independer.nl in collaboration with other parties. The aim is to increase transparency and to support consumers in their health care choices. The information is generated by the external parties, Mediquest and Zorgweb.
    12. Zorgkiezerhttp://www.zorgkiezer.nl
    (http://www.webcitation.org/5clU6C5MZ)    		Initiative of DGN Publishers (Internet company) in collaboration with health care providers and health insurance companies. The aim is to help consumers and health care professionals in their choices. The information is generated by the website editors.
    13. Zorgbelanghttp://www.zorgbelang-nederland.nl
    (http://www.webcitation.org/5clUFSXlV)    		Initiative of Zorgbelang Nederland (association of local organizations advocating health care consumers’ interests) in collaboration with patient associations and other parties. The aim is to provide the public with information about health care.
    14. Agis Zorggidshttp://www.agisweb.nl
    (http://www.webcitation.org/5clUPVzGY)    		Initiative of health insurer Agis. The aim is to inform the insured about their options in health care (concerning contracted providers) and to provide public accountability for the activities of Agis. The information is generated by external parties.
    15. Menzis
    behandelwijzer    		http://www.menzis.nl
    (http://www.webcitation.org/5clUWhkfA)    		Initiative of health insurer Menzis. The aim is to support the insured in their health care choices (concerning contracted providers). The information is generated by the health purchase department and by external parties.
    16. VGZ Zorggids -Vergelijk en kieshttp://www.vgz.nl
    (http://www.webcitation.org/5clUcadC0)    		Initiative of health insurer VGZ. The aim is to support the insured in their health care choices (concerning contracted providers). The information is generated by external parties.
    17. CZ Ziekenhuisver-gelijkerhttp://www.cz.nl
    (http://www.webcitation.org/5clUhnHi0)    		Initiative of health insurer CZ. The aim is to support the insured in their health care choices (concerning contracted providers). The information is generated by external parties.
    18. AD Ziekenhuisver-gelijkerhttp://www.ad.nl/ziekenhuistop100
    (http://www.webcitation.org/5clUkUGcj)    		Initiative of the daily paper Algemeen Dagblad (AD), in collaboration with health care professionals and medical associations. The aim is to inform the public about hospital performances. The information is generated by the paper: hospitals are asked to provide the information.
    19. Elsevier Beste Ziekenhuizenhttp://www.elsevier.nl/artimg/200709/besteziekenhuizen.pdf
    (http://www.webcitation.org/5clUqCXSH)    		Initiative of the weekly magazine, Elsevier, in collaboration with health care professionals, managers, and researchers. The aim is to inform the public about hospital performance concerning current questions in health care.
    20. Vaatpatienthttp://www.vaatpatient.nl
    (http://www.webcitation.org/5clUrNSEI)    		Initiative of the Vereniging van Vaatpatienten (VVVP) (vascular disease patient association). The aim is to support patients in their health care choices. The information is generated by external parties. The VVVP provides quality marks based on the information.
     
    Norway
    21. Fritt Sykehusvalg Norgehttp://www.frittsykehusvalg.no
    (http://www.webcitation.org/5clV91h05)    		Initiative of the Norwegian Ministry of Health in collaboration with patient advisors. The aim is to empower consumers and to support consumers and health care professionals in their choices. In addition, the aim is to stimulate competition and quality improvement.
     
    United Kingdom
    22. Dr. Fosterhttp://www.drfoster.co.uk
    (http://www.webcitation.org/5clV0liGA)    		Private initiative in collaboration with the Information Centre for Health and Social Care, health service organizations, and local authorities. The aims are to inform consumers and health care professionals about the options in health care, and to support consumers in their health care choices. In addition, the aim is to stimulate quality improvement initiatives. The information is generated from a number of external sources.
    23. NHS choiceshttp://www.nhsdirect.nhs.uk
    (http://www.webcitation.org/5clVBVDFE)    		Initiative of the NHS (National Health Services), in collaboration with the National Library for Health, the Information Centre for Health and Social Care, the Health care Commission and other parties. The aim is to support consumers in their decisions about health and health care.
    24. Human Fertilisation and Embryology
    Authority, clinics guide    		http://www.hfea.gov.uk
    (http://www.webcitation.org/5clVXndpE)    		Initiative of the Human Fertilisation and Embryology Authority (HFEA). The aims are to inform consumers about the options in health care and to support them in their health care choices. The information is generated by the HFEA and provided by the clinics.
    25. British Association of Aesthetic Plastic
    Surgeons    		http://www.baaps.org.uk
    (http://www.webcitation.org/5clVJK6mK)    		Initiative of the British Association of Aesthetic Plastic Surgeons. The aims are to inform the public about the practice and quality of plastic surgery and to support consumers in their health care choices.
    26. Private Healthcare UKhttp://www.privatehealth.co.uk
    (http://www.webcitation.org/5clVKQVND)    		Initiative of Intuition Communication Ltd (a commercial organization). The aims are to inform consumers about options in private health care and to support them in their health care choices.
     
    United States
     
    27. Hospital Comparehttp://www.hospitalcompare.hhs.gov
    (http://www.webcitation.org/5clZSIjyU)    		Initiative of the US Department of Health and Human Services (HHS). Hospital Compare is a collaboration of the Centers for Medicare and Medicaid Services (CMS), the Department of Health and Human Services, and members of the Hospital Quality Alliance (HQA). The aim is to support consumers in their health care choices. The information is provided by the health care providers.
    28. The Leapfrog Grouphttp://www.leapfroggroup.org
    (http://www.webcitation.org/5clZVjgKj)    		Initiative of the Leapfrog Group (a collaboration of employers). The aim is to stimulate transparency and access to information in order to support health purchasers and consumers in their choices. In addition, the aim is to stimulate quality improvement initiatives. The information is provided by the health care providers.
    29. The Patient Advocatehttp://www.opa.ca.gov/report_card
    (http://www.webcitation.org/5clZX6PqW)    		Initiative of the Office of the Patient Advocate (OPA) in collaboration with the Department of Managed Health Care. The aim is to inform health care consumers about their rights and about the options in health care (patient empowerment). In addition, aims are to stimulate health care transparency and to support health care purchasers and consumers in their choices. The information is generated from a number of external sources.
    30. Nursing Home
    Compare    		http://www.medicare.gov/NHcompare
    (http://www.webcitation.org/5clZpsiiX)    		Initiative of Medicare. The aims are to inform the public about nursing home options in Medicare and to support consumers in their choices. The information is generated by external parties and/or provided by the nursing homes.
    31. Home Health care Comparehttp://www.medicare.gov/HHcompare
    (http://www.webcitation.org/5clZlSuKF)    		Initiative of Medicare. The aims are to inform the public about home health care options in Medicare and to support consumers in their choices. The information is generated by external parties and/or provided by the home health care providers.
    32. Dialysis Facility Comparehttp://www.medicare.gov/dialysis
    (http://www.webcitation.org/5clZvYdSV)    		Initiative of Medicare. The aims are to inform the public about chronic kidney disease and dialysis, about dialysis facility options in Medicare, and to support consumers in their choices. The information is generated by external parties and/or provided by the facilities.
    33. Medicare Options Comparehttp://www.medicare.gov/MPPF
    (http://www.webcitation.org/5cla2KXFv)    		Initiative of Medicare. The aims are to inform the public about health plans options in Medicare and to support consumers in their choices. The information is generated by external parties and / or provided by the plans.
    34. U Compare
    Healthcare    		http://www.ucomparehealthcare.com
    (http://www.webcitation.org/5cla68Ljp)    		Initiative of About, Inc (part of the New York Times Company). The aim is to support consumers in their health care choices. The information is generated from a number of external federal sources
    35. California Nursing Home Searchhttp://www.calnhs.org
    (http://www.webcitation.org/5claA0qhi)    		Initiative of the California Health care Foundation in collaboration with the Department of Social and Behavioral Sciences of the University of California. The aim is to inform the public about the options in health care. The information is generated from a number of external state and federal sources.
    36. NCQAhttp://www.ncqa.org
    (http://www.webcitation.org/5claG27UP)    		Initiative of the National Committee for Quality Assurance (NCQA). The aim is to stimulate transparency and quality improvement initiatives. In addition, the aim is to support consumers in their health care decisions. The information (based on a set of standardized measures) is generated by the NCQA.
    37. US News Healthhttp://health.usnews.com/sections/health
    (http://www.webcitation.org/5claM7ca0)    		Initiative of the US News magazine (which also includes a weekly digital magazine). The aim is to inform the public about performance of hospitals (America’s best hospitals) and about health plans (America’s best health plans). The information is generated by the magazine’s editors.
    38. AHD.comhttp://www.ahd.com
    (http://www.webcitation.org/5claNNKMz)    		Initiative of the American Hospital Directory, Inc. (a private company). The aim is to inform subscribers about performances of hospitals. The information is generated by the company and extracted form a number of external sources.
    39. Health Care Choiceshttp://www.healthcarechoices.com
    (http://www.webcitation.org/5claTftLr)    		Initiative of Health Care Choices (HCC) which is a not-for-profit corporation. The aims are to inform the public about the health care system and to support health care purchasers and consumers in their choices.
    40. Quality Checkhttp://www.qualitycheck.org
    (http://www.webcitation.org/5claYPkbV)    		Initiative of the Joint Commission on Accreditation of Health care Organizations (JCAHQ), which is a non-for-profit organization. The aim is to support consumers in their health care choices. The information is provided by the health care providers to the Joint Commission.
    41. PHC4http://www.phc4.org
    (http://www.webcitation.org/5ndQxiDQX)    		Initiative of the Pennsylvania’s Health Care Cost Containment Council. The aim is to increase transparency and competition between health care providers. The information is generated from hospitals and health plans by the Council.
    Sweden
    42. Aldre-guidenhttp://www.socialstyrelsen.se/aldreguiden
    (http://www.webcitation.org/5cladHprj)    		Initiative of Socialstyrelsen (a governmental organization of the Ministry of Health). The aims are to inform consumers about the options in elderly care and to support their choices. In addition, the aim is to stimulate quality improvement initiatives. The information is provided by local authorities.

    aDescription based on website content in September 2008.

    bBecause website content and presentation formats change over time, the URLs have been archived: the URLs within brackets can be used to view the information on the home page.

    Most websites contained, in one way or another, both summary and more detailed information. Summary information was usually presented in tabular formats using rows to display providers and columns to display attributes (see Figure 1). Tables with a display configured differently (ie, providers in columns and attributes in rows) were also common but this configuration was not typically used in summary tables. However, tables configured in this way were frequently found to present more detailed comparative information pertaining to the specific providers selected. Although some summary tables presented many different attributes, in most cases only a limited number of attributes (about 3 to 7) was displayed. In some summary tables, the main attributes were divided into subattributes. Another frequently used method was to allow the consumer to determine the amount of information to be presented in a table.

    ‎
    Figure 1. Example of a typical tabular format displaying providers in rows and attributes in columns
    View this figure

    Information Characteristics

    Table 2 provides an overview of the information characteristics on the reviewed websites.

    Table 2. Reviewed websites and their information characteristics
    WebsiteaHealth Care SectorTypes of
    Informationb    		Classification of Integration
    Formatc    		Drill Down PathsInformation DisplayRationale for Presentation
    Formatsd
    1. Your HospitalsHospitalsB, C2No drill down paths, reports downloaded as PDF filesWords; numbers-
    2. Hospital ReportHospitalsB, C0No drill down paths, reports downloaded as PDF filesNumbers-
    3. SundhedHospitalsA, B, C0 (separate pages for different types of information); 2 (different types in one table by
    consumer choice)    		Drill down paths to same information per providerNumbers; stars (5); capitals-
    4. Weisse ListeHospitals (will include nursing homes and
    rehabilitation
    facilities in near future)    		A, B, C1Drill down paths to more specific information per hospitalWords; numbers; horizontal bars; round icons (favorites)D, E, F
    5. Klinik Führer Rhein-RuhrHospitalsA, B, C3Drill down paths to more specific information per hospitalWords; numbers; thermometers-
    6. Klinikführer RheinlandHospitalsA, B3Drill down paths to more specific information per hospitalWords; numbers; traffic lights (3 colors); horizontal barsD, E, F
    7. Hamburger Krankenhaus-spiegelHospitalsA, B0No drill down pathsNumbers; horizontal bars-
    8.Klinikbe-
    wertungen    		HospitalsA, C (anecdotal information)3Drill down paths to specific evaluations of patientsNumbers; stars (6); words-
    9. Health Information and QualityHospitalsB-No drill down paths, reports downloaded as PDF filesWords in different colors (= symbols)-
    10. kiesBeterHospitals, nursing homes, home care, outpatient mental health care, care for the handicapped,
    primary care, palliative care, health plans    		A, B, C0 and 2 (depending on health care sector); 3 (summary information)Drill down paths to more detailed informationWords; numbers; capital letters; stars (3); stars (5); horizontal bars (1)D, E, F
    11. Independer
    Gezondheids-zorg    		Hospitals, home care, primary care, physiotherapy, health plansA, B, C3Drill down paths to more specific information per providerWords, numbers, stars (4), stars (5), round icons (colored), coins, horizontal barsD, E, G
    12. ZorgkiezerHospitals, health plansA, B3Drill down paths to more specific information per providerWords, numbers, stars (5), checkmarks-
    13. ZorgbelangNursing homes, home care, care for the handicapped, outpatient mental health careA (links to websites with B and C)-No drill down pathsWords-
    14. Agis ZorggidsContracted hospitalsA ,C0No drill down pathsWords, round icons (3)E, F
    15. Menzis behandelwijzerContracted hospitalsA, B, C3No drill down pathsWords, numbers, stars (4), round icons (with certain degree of filling), plus icons (3)F, G
    16. VGZ Zorggids -Vergelijk en kiesContracted hospitals and other providersA, B, C3No drill down pathsWords, numbers, squares (4)-
    17. CZ Ziekenhuisver-gelijkerContracted hospitalsA, B, C1No drill down pathsWords, numbers, stars (4), stars (5)-
    18. AD Ziekenhuisver-gelijkerHospitalsB, C2Drill down paths to more specific information per providerWords, numbers-
    19. Elsevier Beste
    Ziekenhuizen    		HospitalsA, B3No drill down paths, reports downloaded as PDF filesRound icons (5, colored), horizontal bars-
    20. VaatpatientHospitalsA, B2Drill down paths to more specific information per providerNumbers, checkmarks-
    21. Fritt Sykehusvalg NorgeHospitalsA, B, C1Drill down paths to somewhat more detailed quality informationNumbers, words, symbols (-, +, 0)-
    22. Dr. FosterHospitals, specialized clinics, complementary practitionersA, B1 (with exception of distance)Drill down paths to more specific information per hospital; selection options to obtain more detailed informationWords,numbers, horizontal bars, stars (5), squares (3)-
     
    23. NHS choicesHospitalsA, B, C3 (summary information); 1 (detailed information)Drill down paths to more detailed information; drill down paths to more specific information per providerWords, numbers, round icons with words, stars (3), horizontal bars, squares (5)-
    24. Human Fertilisation and Embryology
    Authority, clinics guide    		Specialized clinicsA, B0No drill down pathsWords, numbers, horizontal bars, triangles (1)-
    25. British Association of Aesthetic Plastic SurgeonsPlastic surgeonsA, B2Drill down paths to more specific information per providerWords, numbers, stars (1)-
    26. Private Healthcare UKHospitals, doctors, GP’s, nursing homes, cosmetic surgery, dental care, health plansA, B, C (anecdotal information)1Drill down paths to more specific information per providerWords, numbers, ribbons (1)D
     
    27. Hospital CompareHospitalsA, B, C3 (summary information); 2 (after selection of hospitals)Drill down paths to hospital location on mapWords, numbers, horizontal bars (1)-
     
    28. The Leapfrog GroupHospitalsB-Drill down paths to more specific information per providerVertical bars (4), horizontal bars (1)-
    29. The Patient AdvocateMedical groups, hospitals, health plansB, C3 (summary information); 0 (detailed information)Drill down paths to more detailed informationnumbers, stars (4), horizontal bars (1), round icons with words and colors (5)-
    30. Nursing Home CompareNursing homesA, B0 (summary information); 2 (detailed information)Drill down paths to more specific information per provider; drill down paths to provider location on map; drill down paths to visual display in bar graphsWords, numbers, cubes in bar (4), horizontal bars-
    31. Home Health care CompareHome careA, B0 (summary information); 2 (detailed information)Drill down paths to visual display in bar graphsWords, numbers, checkmarks, horizontal bars-
    32. Dialysis Facility CompareSpecialized centraA, B0 (summary information); 1 (detailed information)Drill down paths to more specific information per provider; drill down paths to more detailed quality information; drill down paths to provider location on mapWords, numbers, horizontal bars, checkmarks-
    33. Medicare Options CompareHealth plansA, B, C0 (summary information); 3 (detailed information)Drill down paths to more specific information per health planWords, numbers, stars (5)-
    34. U Compare HealthcareDoctors, hospitals, nursing homes, health plans, mammography centers; fertility clinicsA, B1Drill down paths to more specific information per providerWords, numbers, vertical bars (1), plus icons (1), checkmarks-
     
    35. California Nursing Home SearchNursing homes, home care, hospicesA, B3 (summary information); 1 (detailed information)Drill down paths to more specific information per providerWords, numbers, stars (3)D
    36. NCQADoctors, health plansA, B, C3 (summary and detailed information)Drill down paths to more detailed quality information; drill down paths to more specific information per providerWords, numbers, stars (4), horizontal bars, certification symbols (1)-
    37. US News HealthHospitals, health plansB, C0 (hospitals); 3 (health plans)Drill down paths to more detailed information; drill down paths to more specific information per providerWords, numbers, round colored icons (5), stars (5)-
    38. AHD.comHospitalsA, (B and C only when for members)0 (summary and detailed information)Drill down paths to more specific information per providerWords, numbers, colored parts-
    39. Health Care ChoicesHospitals (and doctors for pay)A, links to B-No drill down pathsWords, numbers-
     
    40. Quality CheckHospitals, nursing homes, home care, outpatient mental health careA, B1 (summary information); 0 (detailed information)Drill down paths to more specific information per provider; drill down paths to more detailed informationWords, numbers, checkmarks, certification symbols (1), round icons (3)-
    41. PHC4Hospitals, health plansA, B, C3 (hospitals); 0 (health plans)No drill down paths, reports downloaded as PDF filesNumbers, round icons (3)F
    42. Aldre-
    guiden    		Care for the elderlyB-Drill down paths to more detailed quality informationNumbers, vertical barsD, E, F

    aDescription based on website content in September 2008. Website content and presentation formats change over time. Therefore, the URLs have been archived (see Table 1).

    bThis classification is based on Van Loon and Tolboom [20]: A = Factual information based on provider characteristics and services; B = Quality information based on performance indicators; C = Quality information based on health care user experience

    c0 = no integration of different types of information (different types of information on different pages); 1 = limited integration of different types of information (different types of information can be selected and viewed on one page, but no integration in one table on one page); 2 = quite amount of integration of different types of information (different types on one page, but clearly separated from each other); 3 = high level of integration of different types of information (different types of information presented in one table, with or without action of the consumer).

    dD = test(s) of different formats; E = existing scientific knowledge; F = expert opinion; G = other rationale

    Health Care Sectors

    On 32 of the 42 websites (76%), information about hospitals was presented. Although in recent years more information has become available in other health care sectors, such as nursing homes and home care (found on 10 websites; 24%), and health plans (found on 10 websites; 24%), hospital information clearly had the largest share on the Internet. Information about health plans was found mainly on US websites. Reporting systems containing information on several health care sectors were found mainly on websites from the United Kingdom and the United States.

    Types of Information

    The most common type of information found on the reviewed websites was quality information based on performance indicators (found on 37 websites; 88%). Information on health care providers’ characteristics and services was also common (found on 34 websites; 81%); this information was usually presented for each provider separately. In these cases, we did not further evaluate the information. Quality information based on health care users’ experiences was found on a little more than half (found on 22 websites; 52%) of the reviewed websites.

    Integration of Different Information Types

    The degree of integration of different information types was most often classified as type 0 (no integration of different types of information; different types of information on different pages). This type of information integration was found on 15 websites (36%). Type 3 (high level of integration of different information types; different types of information presented in one table) was found on 17 websites (41%). The two integration structures falling in between these extremes were less often found: type 1 on 10 websites (24%) and type 2 on 9 websites (21%), respectively. Concerning type 1 and type 2 integration, many different options were used to separate the information types. For example, separate tab pages, menu bars, white spaces, bold headlines, and colors to distinguish between different information types were displayed. In some cases, different information displays were used at the same time.

    Examples of all four classifications are shown in Figures 2 to 5. Figure 2 is an example of type 0 integration (no integration of different types of comparative information). The example is from the PHC4 website in the United States. In this example, information on health care user experience is displayed, but information based on performance indicators can be found elsewhere on the website. Figure 3 is an example of type 1 integration (limited integration of different types of comparative information). The example is from the Fritt Sykehusvalg website in Norway. In this example, different information types can be selected on the displayed tab pages, but are not displayed in a single table simultaneously. Figure 4 is an example of type 2 integration (a medium amount of integration of different types of comparative information). This example is from the kiesBeter website in the Netherlands. Different information types on one page are presented in separate blocks. Figure 5 is an example of type 3 integration (high integration of different types of comparative information). This example is from the Kliniken Rhein Ruhr website in Germany. Different information types are integrated in a single table.

    ‎
    Figure 2. Example of type 0 integration (no integration of different types of comparative information)
    View this figure

    ‎
    Figure 3. Example of type 1 integration (limited integration of different types of comparative information)
    View this figure

    ‎
    Figure 4. Example of type 2 integration (medium amount of integration of different types of comparative information)
    View this figure

    ‎
    Figure 5. Example of type 3 integration (high integration of different types of comparative information)
    View this figure

    Drill Down Paths

    A considerable number of websites (29; 69%) provided drill down paths to more specific information. The most common types of drill down paths were paths to more specific information per provider (on 21 websites; 50%) and paths to more detailed (underlying) information (on 11 websites; 26%). The information per provider to which a Web page was linked usually consisted of very specific information listed on a single Web page. Concerning more detailed comparative information, the degree to which more specific information was provided differed across websites. Figure 6 shows an example of more detailed information available after drilling down. The example is from the website US News Health. In this example, more detailed information can be found by clicking on “more detail.”

    Information Display

    To display comparative health care information, numbers (37; 88%) and words (32; 76%) were most commonly used. Most often, information about provider characteristics and services was presented by using words and numbers only. Graphical formats and symbols were frequently applied as well, usually to present quality information. The most frequently applied symbols were stars (on 15 websites; 36%; see Figures 1 and 4) and round icons (on 10 websites; 24%; see Figure 2). The numbers of stars, round icons and other symbols differed both across and within websites: five, four, and three symbols were most frequently found. Furthermore, it was quite common (on 18 websites; 43%) to use bar charts to present quality information.

    Rationale for Presentation Formats

    In total, 10 of the 42 websites (24%) returned a completed survey. Of these 10, the most common rationales for the presentation formats used were expert opinion and tests with consumers and/or other stakeholders (both found on 7 websites; 70% of the responding websites) (see Table 2).

    ‎
    Figure 6. Example of drill down path to more detailed information
    View this figure


    Principal Results

    We reviewed 42 websites providing public comparative health care information and analyzed the presentation approaches of different information types. The general conclusion is that a wide variety of presentation approaches are used on Web-based reporting systems, in particular with respect to the integration of different information types and the information display. The two extreme options to integrate different information types were most often found: providing no integrated information at all and presenting a high level of integration in a single table. Between these two extremes, different options to either separate or integrate the information types were applied. Although different presentation formats were found, some standard elements emerged as well. On most websites, for example, tabular formats were used that presented providers in rows and indicators in columns. The majority of information was provided hierarchically, with options to get an overall sense of performance provided first, and options to get more detailed information provided subsequently. This format seemed necessary to manage the total amount of available information.

    Study Limitations

    Our study was intended to provide an impression of existing presentation approaches of comparative health care information. Clearly, not all aspects related to information presentation have been systematically reviewed. Although it is beyond the scope of the current study, it is important to keep in mind that the quality of the information itself has not been assessed. Websites may vary on the quality of the information collected and presented. We believe, however, that the current study results provide insight into the state of the art concerning the presentation of comparative health care information in the late 2000s. Our study might be limited by the fact that the search strategies were performed solely by native speakers of Dutch. The number of websites per country might be biased toward including more Dutch websites. And, in general, the number of websites found per country may be partly influenced by each author’s mastery of the different languages included in the search. We only captured Western websites, and the results should therefore be interpreted as only representative of Western websites. Another limitation is the fact that the response rate of the survey was very low. Because of this low response rate, we had limited insight into how information was tested and what consumers’ reactions were. From the returned surveys, it appeared that consulting experts and tests with consumers were important methods to select presentation formats. It is unknown whether these methods are representative of those used for development of the other websites included in the study.

    Conclusions

    Regarding the usefulness of comparative information for consumers, several results related to the reviewed presentation formats are worth discussing further.

    First, the standard use of tabular formats to structure the information is important. On the investigated websites, the use of rows for providers and columns for attributes was the typical format for displaying summary information, whereas the opposite display format was used for more detailed information (after selection or drill down paths). It would be relevant to determine whether it makes a difference for consumers to see either providers or attributes in rows. It is known that consumers use both holistic processing (providers first) and dimensional processing (attributes first) with a slight preference for the latter [34]. Swait and Adamowicz [23] argued that the more complex information is, the simpler the heuristics that are used, which results in readers focusing more on alternatives (providers) than on attributes. From these findings we conclude that it is not the direction of the information display that is particularly important, but rather the information complexity in the table. Given the fact that most consumers will probably view only summary information, these tables should thus contain graspable numbers of providers and attributes. Otherwise, consumers will not concentrate on the attribute information even though this is the information that has been provided to support their decisions.

    A second important aspect to consider is the variety of information display options found on websites. Words as well as numbers were frequently used to present comparative information. It is striking that numbers were displayed on so many websites although it is known that consumers have difficulty evaluating them [21]. As recently demonstrated by Peters and colleagues [35], numbers do not have evaluative meaning to consumers. On a large number of the websites, however, information was presented using symbols. Hibbard and colleagues [21,27] argued that visual cues such as stars increase the evaluability of information, because these cues help consumers sort providers into categories of better and worse. Furthermore, symbols might more easily attract attention compared with numbers and words, similar to pictorial information [36,37]. Pictures seem to promote a more holistic and integrative strategy to process information than do words [38]. However, when there is text-symbol incongruity, symbols may decrease message comprehension, especially among consumers having low literacy [39]. In an experiment by So and Smith [24], symbols (smiley faces) added to tabular information did not facilitate consumers’ decision accuracy. Future research on comparative health care information should include similar experiments and examine the impact of symbols. The use of stars,which were frequently found on the reviewed websites, may be an effective presentation format of comparative health care information. More research is needed to confirm this.

    Third, attention should be focused on the integrated presentation of different information types. To our knowledge, there are no studies that examined the effects of integration levels of different information types. Hence, we cannot make scientifically based inferences about how the different degrees of integration found on the websites included either support or impede consumer decision making. Compared with the 1996 review of McCormack et al [25], who analyzed the content of comparative health care information, it is important that more “objective” performance indicators are dominant in the current review (included in 88% of the reviewed websites). In the findings of McCormack et al, such performance indicators were included in 10 out of 24 (24%) reporting systems, all in combination with health care user experience data. Despite the lack of evidence for consumer reactions, some arguments about the advantages and disadvantages of integrating information types can be made. One important benefit of a high level of integration is that all information can be viewed in an overview at the same time. This may contribute to a sense of clarity and to better coping with a large amount of information. A drawback is that such an overview cannot take up too much space on Web pages, and that the chance that a page will contain contradictory information increases. In addition, more specific information will be lost or difficult to find for consumers, and the flexibility to apply different search strategies diminishes. The opposite of no or very limited integration can, however, also bring about negative consequences. For example, consumers may not see a large part of the information at all or may fail to notice important information elements. In addition, consumers may need to undertake many steps in the process of viewing information, although it is known that consumers prefer to see information on one Web page [40]. An approach advocated by Harris-Kojetin et al [19] is to help consumers to think about their own priorities in the major dimensions of health care. This approach using self-selection menus could be applied to assess whether consumers are more focused on technical outcomes of health care or more focused on aspects related to trust in health care. The fact that these two health care consumer profiles can be distinguished among different patient groups [41] may be used as argument for low levels of integration of different information types. However, the approach of self-testing consumer preferences assumes that consumers have stable preferences, although we know that consumers often construct these preferences while viewing information [42]. All in all, the issue of integrating different information types remains an important topic for further discussion and, importantly, for future research on health care information. In our opinion, a certain level of integration is necessary to prevent consumers overlooking important information or getting stuck in too many decision steps.

    A fourth topic for further discussion is the role of contradictory information, which appears to be inherent in comparative health care information. As stated, a higher integration of different information types increases the chance that contradictory information must be processed. It is usually assumed that conflicting information increases task complexity. Psychological theories such as cognitive dissonance theory [43] suggest that when people meet aspects of their decision environment that are incompatible with each other, they attempt to reestablish consistency by transforming some of the incompatible elements. The activities associated with this restoring process are known to demand elaboration [44], and will probably lead to distress as well. Individuals tend to avoid conflict or to avoid choosing at all when choices become more complex [14,45-48]. In addition, there is a higher chance individuals will use simpler choice heuristics [23]. At this time, it remains unclear how to deal with the issue of contradictory information. It is important that future studies search for comprehensible presentation formats that facilitate correct processing of contradictory information. Meanwhile, website managers should be careful not to present information that includes many contradictory elements.

    Finally, we want to address the large amount of information we found on websites. It is known that today’s consumers are often overloaded with information. Different effects of information overload have been described in the literature. Importantly, a large amount of information can lead to low quality of consumers’ choices [14] and to less purchasing [49]. Lurie [50] showed that the amount of information that needs to be processed not only depends on the number of alternatives and attributes in a choice set, but also on the number of attribute levels and the distribution of attribute levels across alternatives. To control the amount of information on websites, it seems necessary to provide only limited numbers of providers and attributes to consumers, as was already suggested concerning information complexity. When a large variety of attribute levels are shown, Web designers and research staff should note the increasing complexity and search for alternative options to display information. Drill down paths can be used to layer information and to comprehensibly provide a large amount of information, as was done on many websites reviewed in this study. Furthermore, it may be necessary to inform consumers on the home page about the amount of information that can be viewed on the website. Consumers will then be better prepared and perhaps less discouraged when they attempt to access the information. Future research should focus on the amount of information that consumers are able and willing to process.

    With the current descriptive study, we have shed some light on the decision environment of health care consumers in a period of market-based, consumer choice-driven health care sectors. We believe that more transparency about the effectiveness of the chosen formats on websites is greatly needed; currently it is largely unclear which rationales are used to select them. Evidence-based quality criteria for presentation approaches should be formulated, and future research can assess how different websites meet these criteria. Moreover, research is needed on other aspects of the decision environment, such as consumers’ considerations and motivations to achieve a (good) decision and their decision strategies. Consumers highly motivated to search for good performance might be less distressed by complex information presentation than people who do not care to actively choose health care in any case. More generally, the design of websites should be linked to theoretical models of consumer decision making and communication technology. In our opinion, it is a challenge for Internet research to create more manageable comparative health care information that is actually used by consumers. Current presentation approaches on websites do not seem to be systematically selected. Website managers should not just release data on the web, but instead should become aware of the many complexities inherent in the comparative information they are providing.

    Acknowledgments

    This project received financial support from the Dutch Ministry of Health, Welfare and Sport.

    Conflicts of Interest

    Ylva van den Hengel and Jeanne van Loon work for the National Institute for Public Health and the Environment (RIVM), which manages the Dutch website kiesBeter (www.kiesBeter.nl), one of the websites reviewed in this paper.

    1. Maarse H, Meulen RT. Consumer choice in Dutch health insurance after reform. Health Care Anal 2006 Mar;14(1):37-49. [Medline]
    2. Lako CJ, Rosenau P. Demand-driven care and hospital choice. Dutch health policy toward demand-driven care: results from a survey into hospital choice. Health Care Anal 2009 Mar;17(1):20-35. [Medline] [CrossRef]
    3. Carlisle RT. Internet report cards on quality: what exists and the evidence on impact. W V Med J 2007;103(4):17-21. [Medline]
    4. Bettman JR, Kakkar P. Effects of information presentation format on consumer information acquisition strategies. J Consum Res 1977;3(4):233-240.
    5. Payne JW, Bettman JR, Johnson EJ. The Adaptive Decision Maker. New York, NY: Cambridge University Press; 1993.
    6. Vaiana ME, McGlynn EA. What cognitive science tells us about the design of reports for consumers. Med Care Res Rev 2002 Mar;59(1):3-35. [Medline]
    7. Hibbard JH. Engaging health care consumers to improve the quality of care. Med Care 2003 Jan;41(1 Suppl):I61-I70. [Medline]
    8. Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008 Jan 15;148(2):111-123 [FREE Full text] [Medline]
    9. Marshall MN, Shekelle PG, Leatherman S, Brook RH. The public release of performance data: what do we expect to gain? A review of the evidence. JAMA 2000 Apr 12;283(14):1866-1874 [FREE Full text] [Medline]
    10. Farley DO, Short PF, Elliott MN, Kanouse DE, Brown JA, Hays RD. Effects of CAHPS health plan performance information on plan choices by New Jersey Medicaid beneficiaries. Health Serv Res 2002 Aug;37(4):985-1007 [FREE Full text] [Medline]
    11. Farley DO, Elliott MN, Short PF, Damiano P, Kanouse DE, Hays RD. Effect of CAHPS performance information on health plan choices by Iowa Medicaid beneficiaries. Med Care Res Rev 2002 Sep;59(3):319-336. [Medline]
    12. Peters E, Dieckmann N, Dixon A, Hibbard JH, Mertz CK. Less is more in presenting quality information to consumers. Med Care Res Rev 2007 Apr;64(2):169-190. [Medline] [CrossRef]
    13. Slovic P. Towards understanding and improving decisions. In: Howell WC, Fleishman EA, editors. Human Performance and Productivity, Volume 2: Information Processing and Decision Making. Hillsdale, NJ: Erlbaum; 1982:157-183.
    14. Keller KL, Staelin R. Effects of quality and quantity of information on decision effectiveness. J Consum Res 1987;14(2):200-213.
    15. Hibbard JH, Jewett JJ, Engelmann S, Tusler M. Can Medicare beneficiaries make informed choices? Health Aff (Millwood) 1998;17(6):181-193 [FREE Full text] [Medline]
    16. Wells WD, Chen Q. The dimensions of commercial cyberspace. Journal of Interactive Advertising 2000;1(1).
    17. Coyle JR, Thorson E. The effects of progressive levels of interactivity and vividness in web marketing sites. J Advert 2001;30(3):65-77.
    18. McMillan SJ, Hwang JS, Lee G. Effects of structural and perceptual factors on attitudes toward the website. J Advert Res 2003;43:400-409.
    19. Harris-Kojetin LD, McCormack LA, Jaël EF, Sangl JA, Garfinkel SA. Creating more effective health plan quality reports for consumers: lessons from a synthesis of qualitative testing. Health Serv Res 2001 Jul;36(3):447-476 [FREE Full text] [Medline]
    20. Van Loon AJM, Tolboom RAL. Ontwerprapport kiesBeter.nl 2006-2007. (Design report kiesBeter.nl 2006-2007). Bilthoven, Netherlands: Rijksinsituut voor Volksgezondheid en Milieu (RIVM); 2005.
    21. Hibbard JH, Peters E. Supporting informed consumer health care decisions: data presentation approaches that facilitate the use of information in choice. Annu Rev Public Health 2003;24:413-433. [Medline] [CrossRef]
    22. Wright P. Writing technical information. In: Rothkopf EZ, editor. Review of Research in Education, Volume 14. Washington, DC: American Educational Research Association; 1987:327-385.
    23. Swait J, Adamowicz W. The influence of task complexity on consumer choice: a latent class model of decision strategy switching. J Consum Res 2001;28:135-148.
    24. So S, Smith M. Multivariate decision accuracy and the presentation of accounting information. Accounting Forum 2004;28:283-305.
    25. McCormack LA, Garfinkel SA, Schnaier JA, Lee AJ, Sangl JA. Consumer information development and use. Health Care Financ Rev 1996;18(1):15-30. [Medline]
    26. Hibbard JH, Peters E, Slovic P, Finucane ML, Tusler M. Making health care quality reports easier to use. Jt Comm J Qual Improv 2001 Nov;27(11):591-604. [Medline]
    27. Hibbard JH, Slovic P, Peters E, Finucane ML. Strategies for reporting health plan performance information to consumers: evidence from controlled studies. Health Serv Res 2002 Apr;37(2):291-313 [FREE Full text] [Medline]
    28. Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ 2002 Mar 9;324(7337):573-577 [FREE Full text] [Medline]
    29. Keselman A, Tse T, Crowell J, Browne A, Ngo L, Zeng Q. Assessing consumer health vocabulary familiarity: an exploratory study. J Med Internet Res 2007;9(1):e5 [FREE Full text] [Medline] [CrossRef]
    30. Berland GK, Elliott MN, Morales LS, Algazy JI, Kravitz RL, Broder MS, et al. Health information on the Internet: accessibility, quality, and readability in English and Spanish. JAMA 2001;285(20):2612-2621 [FREE Full text] [Medline]
    31. Eysenbach G, Powell J, Kuss O, Sa ER. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA 2002;287(20):2691-2700 [FREE Full text] [Medline]
    32. Groenewoud AS, Kreuger L, Huijsman R. Keuzevrijheid, keuzemogelijkheden en keuze-ondersteuning in de gezondheidszorg. Een verkennende, internationale studie ter ondersteuning van een reflectie op de ingeslagen weg. (Freedom of choice, choice options and choice support in healthcare. A prospective, international study to support a reflection on the chosen path). Rotterdam, Netherlands: EUR/iBMG; 2006.
    33. Lugtenberg M, Westert GP. Kwaliteit van de gezondheidszorg en keuze-informatie voor burgers. (Quality of healthcare and information for the public). Tilburg, Netherlands: Tranzo; 2007.
    34. Russo JE, Dosher BA. Strategies for multiattribute binary choice. J Exp Psychol Learn Mem Cogn 1983 Oct;9(4):676-696. [Medline]
    35. Peters E, Dieckmann NF, Västfjäll D, Mertz CK, Slovic P, Hibbard JH. Bringing meaning to numbers: the impact of evaluative categories on decisions. J Exp Psychol Appl 2009 Sep;15(3):213-227. [Medline] [CrossRef]
    36. Mitchell AA. The effects of visual and emotional advertising: an information-processing approach. In: Percy L, Woodside A, editors. Advertising and Consumer Psychology. Lexington, MA: Lexington Books; 1983:197-217.
    37. Jae H, Delvecchio D. Decision-making by low-literacy consumers in the presence of point-of-purchase information. J Consum Aff 2004;38(2):342-354.
    38. Holbrook MB, Moore WL. Feature interactions in consumer judgments of verbal versus pictorial presentations. J Consum Res 1981;8(1):103-113.
    39. Jae H, Delvecchio DS, Cowles D. Picture-text incongruency in print advertisements among low-and high literacy consumers. J Consum Aff 2008;42(3):439-451.
    40. Gerteis M, Gerteis JS, Newman D, Koepke C. Testing consumers' comprehension of quality measures using alternative reporting formats. Health Care Financ Rev 2007;28(3):31-45. [Medline]
    41. Groenewoud AS. It’s your choice! A study of search and selection processes, and the use of performance indicators in different patient groups [dissertation]. Rotterdam, Netherlands: Erasmus University; 2008.
    42. Slovic P. The construction of preference. Am Psychol 1995;50:364-371.
    43. Festinger L. A Theory of Cognitive Dissonance. Stanford, CA: Stanford University Press; 1957.
    44. Houston MJ, Childers TL, Heckler SE. Picture-word consistency and the elaborative processing of advertisements. J Mark Res 1987;24(4):359-369.
    45. Tverksy A, Shafir E. Choice under conflict: the dynamics of deferred decision. Psychol Sci 1992;3(6):358-361.
    46. Dhar R. Consumer preference for a no-choice option. J Consum Res 1997;24:215-231.
    47. Dhar R. Context and task effects on choice deferral. Marketing Letters 1997;8:119-130.
    48. Zhang Y, Mittal V. Decision difficulty: effects of procedural and outcome accountability. J Consum Res 2005;32(3):465-472.
    49. Iyengar SS, Lepper MR. When choice is demotivating: can one desire too much of a good thing? J Pers Soc Psychol 2000 Dec;79(6):995-1006. [Medline]
    50. Lurie NH. Decision making in information rich environments: the role of information structure. J Consum Res 2004;30(4):473-486.

    Edited by G Eysenbach; submitted 30.12.08; peer-reviewed by C Fung; comments to author 22.07.09; revised version received 04.11.09; accepted 01.12.09; published 13.04.10

    Copyright

    © Olga C Damman, Ylva KA van den Hengel, A Jeanne M van Loon, Jany Rademakers. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 13.04.2010.  

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.