This paper is in the following e-collection/theme issue:

Physician and Health Services Rating by Consumers (126) Participatory Medicine & E-Patients (758) Quality Improvement (223)

Published on in Vol 22, No 2 (2020): February

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/13830, first published .
What Patients Value in Physicians: Analyzing Drivers of Patient Satisfaction Using Physician-Rating Website Data

What Patients Value in Physicians: Analyzing Drivers of Patient Satisfaction Using Physician-Rating Website Data

What Patients Value in Physicians: Analyzing Drivers of Patient Satisfaction Using Physician-Rating Website Data

Authors of this article:

Sonja Bidmon1 Author Orcid Image ;   Ossama Elshiewy2 Author Orcid Image ;   Ralf Terlutter1 Author Orcid Image ;   Yasemin Boztug2 Author Orcid Image
Article Authors Cited by (26) Tweetations (9) Metrics

    Original Paper

    1Department of Marketing and International Management, Alpen-Adria-Universitaet Klagenfurt, Klagenfurt am Woerthersee, Austria

    2Department of Business Administration, Marketing and Consumer Behavior, University of Goettingen, Goettingen, Germany

    Corresponding Author:

    Sonja Bidmon, PhD

    Department of Marketing and International Management

    Alpen-Adria-Universitaet Klagenfurt

    Universitaetsstrasse 65-67

    Klagenfurt am Woerthersee, 9020

    Austria

    Phone: 43 463 2700 4048

    Fax:43 463 2700 994048

    Email: sonja.bidmon@aau.at


    Background: Customer-oriented health care management and patient satisfaction have become important for physicians to attract patients in an increasingly competitive environment. Satisfaction influences patients’ choice of physician and leads to higher patient retention and higher willingness to engage in positive word of mouth. In addition, higher satisfaction has positive effects on patients’ willingness to follow the advice given by the physician. In recent years, physician-rating websites (PRWs) have emerged in the health care sector and are increasingly used by patients. Patients’ usage includes either posting an evaluation to provide feedback to others about their own experience with a physician or reading evaluations of other patients before choosing a physician. The emergence of PRWs offers new avenues to analyze patient satisfaction and its key drivers. PRW data enable both satisfaction analyses and implications on the level of the individual physician as well as satisfaction analyses and implications on an overall level.

    Objective: This study aimed to identify linear and nonlinear effects of patients’ perceived quality of physician appointment service attributes on the overall evaluation measures that are published on PRWs.

    Methods: We analyzed large-scale survey data from a German PRW containing 84,680 surveys of patients rating a total of 7038 physicians on 24 service attributes and 4 overall evaluation measures. Elasticities are estimated from regression models with perceived attribute quality as explanatory variables and overall evaluation measures as dependent variables. Depending on the magnitude of the elasticity, service attributes are classified into 3 categories: attributes with diminishing, constant, or increasing returns to overall evaluation.

    Results: The proposed approach revealed new insights into what patients value when visiting physicians and what they take for granted. Improvements in the physicians’ pleasantness and friendliness have increasing returns to the publicly available overall evaluation (b=1.26). The practices’ cleanliness (b=1.05) and the communication behavior of a physician during a visit (b level between .97 and 1.03) have constant returns. Indiscretion in the waiting rooms, extended waiting times, and a lack of modernity of the medical equipment (b level between .46 and .59) have the strongest diminishing returns to overall evaluation.

    Conclusions: The categorization of the service attributes supports physicians in identifying potential for improvements and prioritizing resource allocation to improve the publicly available overall evaluation ratings on PRWs. Thus, the study contributes to patient-centered health care management and, furthermore, promotes the utility of PRWs through large-scale data analysis.

    J Med Internet Res 2020;22(2):e13830

    doi:10.2196/13830

    Keywords

    online physician ratings; patient satisfaction; multiattribute models; health care management 


    Background

    Patients are taking a more active role in the decision-making process concerning their medical care [1] in the face of a changing patient-physician relationship [2]. With the World Wide Web as an important source of health-related information [3-5], physician-rating websites (PRWs) are on the rise [6-10], offering an “interesting new source of information about quality of care from the patient's perspective” [11]. PRWs offer the possibility to rate a service encounter with a physician on a Web-based platform. Patients can either post an evaluation to provide feedback to others about their own experience with a physician or read evaluations of other patients before choosing a physician [6].

    From a physician’s point of view, PRWs are important because patients’ perceptions of the physician’s service quality are made publicly available. This fact substantially increases the relevance of patient satisfaction to generate positive word of mouth [7,8]. At the same time, patients’ evaluations on PRWs offer directions for the improvement of a physician’s service quality. The ability to identify how and to which extent different service attributes contribute to patients’ overall satisfaction with a physician is of high importance for the physician and health care sector [12].

    The emergence of PRWs offers new avenues to analyze patient satisfaction and its key drivers. Existing studies on key drivers of patient satisfaction are usually based on small sample sizes [13,14]. From a key driver perspective, analyzing data from PRWs allows for large-scale analyses based on a large number of patients and physicians to identify how specific service attributes contribute to overall patient satisfaction and patients’ behavioral intentions. Thus, the main purpose of this paper was to conduct a key driver analysis using a multiattribute model applied to large-scale data from a PRW. The specific knowledge about the relationship between a service attribute and the overall evaluation can direct the stakeholders’ efforts to improve performance and set priorities in satisfaction management [15] and help to properly allocate scarce resources [16]. Hence, the findings from our study will strengthen the understanding of patient satisfaction and contribute to the body of knowledge in health care management.

    Usage and Usefulness of Physician-Rating Websites

    A broad base of literature has been published so far to investigate PRWs by researchers in many countries worldwide, such as Germany [17-22], Great Britain [23-30], Switzerland [31-33], the Netherlands [34], the United States [6,35-40], Canada [41], and China [42-46]. Rothenfluh and Schulz [31] identified and analyzed 143 PRWs in German- and English-speaking countries (12 countries in total). Apart from the delivery of factual information, such as opening hours and directions [47], PRWs invite patients to evaluate their physicians and articulate their experiences based on the quality of care they received during a medical appointment. Patients produce user-generated content in this vein by posting comments and ratings [48]. From the patients’ point of view, PRWs are a convenient method to share information about the medical care they have received [38,49]. Usage of PRWs is on the rise [9,18,24,49-52], although slowly compared with rating websites in other domains of everyday life [19,33].

    Several points of criticism, however, can be addressed toward PRWs. The literature is inconsistent with regard to the link between information revealed on rating sites and quality of care [11,35,40,49,53,54]. Although people may feel challenged to judge a physician’s competence [32,55], they do assess physicians’ competence on PRWs [32].

    Text mining approaches [42,43] have been used to analyze what patients articulate in free-text comments. These methods often only identify the most frequently mentioned aspects in the comments while neglecting entirely those that only a minority of reviewers mentioned [43,56]. In addition, as Hao and Zhang [43] underline, people may refrain from posting negative textual comments because of data privacy concerns. Thus, some PRWs deliberately refrain from free-text responses and instead rely on rating scales as answer options (such as the PRWs used in this study).

    Holliday et al [50] pinpoint that there are mainly 2 different types of PRWs with regard to the data collection: (1) the so-called independent websites (p 626), such as Healthgrades [57], which are run by private companies and nurtured by crowd-sourced data, and (2) PRWs that can be established by health systems, which collect ratings from patients with recent physician or hospital visits (so-called health system websites, [50]). The PRW we use in our study can be classified as a special form of health system website, as it is run by a noncommercial foundation in cooperation with the main national insurance carriers in Germany. This guarantees that only patients covered by the main national insurance carriers are allowed to post reviews, thus guarding against fraud, which is often seen as another general drawback of PRWs [11].

    Information posted on PRWs and especially data in numerical rating scales can be valuable for different stakeholders. Information delivered on PRWs should be of interest not only for patients but also for physicians and other health care providers. Finally, information based on PRW data can help (noncommercial) PRW providers to justify their business model.

    A Key Driver Analysis of Patient Satisfaction Using Physician-Rating Website Data

    Customer-oriented health care management and patient satisfaction have become important for physicians in their attempt to attract patients in an increasingly competitive environment [58,59]. Our research about patient satisfaction draws upon the literature on customer satisfaction [60]. There is a broad consensus in research that customer satisfaction with products or services is determined by comparing the previous expectations with the actual perceived performance of the product or service (the so-called expectancy disconfirmation framework [61]). As patients’ expectations often prove to be latent over time and individuals do not consciously compare expectations and actual perceived performance [62], research often follows a performance-only appraisal when studying satisfaction [62,63]. In our study, we followed this argument by assuming that customers (ie, patients) form their overall attitude toward the service experience (ie, appointment with a physician) based on the perceived service quality without a conscious comparison with expectations [64,65]. Furthermore, following the study by Wilkie and Pessemier [66], we used a multiattribute model [67] and assumed that the overall attitude toward a service is the sum of attitudes toward the different attributes of the service. Our approach also allowed us to take into account linear and nonlinear effects of perceived service quality on overall evaluation [68,69], ie, service attributes can have diminishing, constant, or increasing returns.

    Diminishing returns mean that improvements in perceived attribute quality have a positive impact on overall evaluation but to a decreasing extent. This means that the contribution to an increase in the overall evaluation gets smaller with increasing perceived attribute quality (following a monotonically increasing and concave function). These service attributes are labeled basic factors in the denotation of the 3-factor structure of customer satisfaction [70,71] and are typically taken for granted by patients.

    Constant returns mean that improvements in perceived attribute quality have a positive impact on overall evaluation and that the contribution to the improvements remains the same along the scale of possible levels of perceived attribute quality (following a monotonically increasing and linear function). Service attributes with constant returns are denominated as performance factors [70,71].

    Increasing returns hold that improvements in perceived attribute quality have a positive impact on overall evaluation, but now the contribution to an increase in the overall evaluation expands in size with increasing perceived attribute quality (following a monotonically increasing and convex function). Service attributes with increasing returns are denominated as excitement factors [70,71].

    Beyond satisfaction or any kind of overall attitude, perceived service quality can also influence repeated purchases [72] and induce positive word of mouth [73]. Consequently, we expanded our multiattribute model of patient satisfaction by using 4 different measures of overall evaluation: (1) overall impression of the physician, (2) patients’ experience with the results of medical treatment by the physician, (3) willingness to recommend the physician, and (4) willingness to revisit the physician for medical treatment.

    To the best of our knowledge, no study has conducted a key driver analysis of patient satisfaction using online physician-rating data and thus has taken a comprehensive perspective on the utility of PRWs. Our study aimed to fill this research gap.


    Data Sources and Measures

    In our study, we used the database of the German noncommercial PRW Weisse Liste [74]. This German PRW can be seen as a best practice example with regard to its compliance with quality criteria required for good physician-rating portals according to the German Agency for Quality in Medicine [75]. The purpose of this platform is the online provision of physician ratings in terms of perceived attribute quality by actual patients. To initiate the formation of a large base of ratings, the platform sent out the physician-rating survey through its statutory health insurance partners by mail in several waves until autumn 2013. The target group of the mail survey was a representative sample of patients from 2 of the largest statutory health insurances in Germany, and patients were allowed to fill out the physician-rating survey either online or offline. The offline ratings were then transferred to the Web-based PRW. Hence, data from this period contain physician evaluations that are based on either online ratings or ratings via a postal mail survey (offline). The idea behind surveying online and offline at the same time was to gain additional momentum for the data collection process to quickly reach a broad rating database. This approach thus led to a highly representative sample of patients’ ratings (ie, both online and offline segments were able to participate). To participate, patients had to state their name, health insurance carrier, and insurance number. For data protection reasons, physician ratings and patient data were processed separately for both types of data collection. Patients could rate the same physician several times, but only the most recent rating was used in the evaluation.

    The online and offline surveys were identical. In the survey, patients were asked several questions related to the following service dimensions: office and staff, communication, and medical treatment by the chosen physician. The questions were worded in the form of 24 different statements, which can be answered on a 4-point scale with 1 as strongly disagree to 4 as strongly agree and the option to answer cannot be assessed (the 24 statements are listed in Table 1 under service attributes). In addition to the service attributes, the following 4 measures of overall evaluation were collected in the survey: “What is your overall impression of this physician?” (overall impression), “How would you describe your experience with the results of medical treatment by this physician?” (experience with results), “Would you recommend this physician to your best friend?” (willingness to recommend), and “Would you visit this physician again, if you had to be medically treated?” (willingness to revisit). The 4 measures of overall evaluation were surveyed using a 5-point scale with 1 as bad to 5 as excellent for overall impression and experience with results and with 1 as definitely not to 5 as definitely for willingness to recommend and willingness to revisit. We rescaled the overall evaluations to a 4-point scale for similarity to the measures of perceived attribute quality. Higher values in the ratings are associated with higher satisfaction for the respective service attribute and a more positive assessment of the overall evaluation measure. Summary scores of the overall evaluations were published on the PRW as soon as 5 or more completed surveys were registered for a physician. We used the same criteria of a minimum of 5 completed surveys for inclusion of physicians in our analysis to avoid biased evaluations by small numbers of surveys.

    Table 1. Means and standard deviations of the service attributes and overall evaluations.
    Service attributes and overall evaluationsValues, mean (SD)
    Service attributes

    		The physician has a pleasant and friendly manner3.84 (0.45)

    		The physician listens to me carefully3.77 (0.55)

    		The physician handles my questions, concerns, and fears in an empathetic way3.74 (0.58)

    		The physician’s office is clean and neat3.85 (0.39)

    		The physician indicates clearly how to take prescribed medication3.80 (0.48)

    		The physician explains diagnoses, causes, and treatments so that I understand everything3.72 (0.58)

    		The physician does not hurry during the medical treatment3.68 (0.61)

    		I have the impression that the physician will refer me to a specialist if this is medically necessary3.78 (0.52)

    		The physician explains exactly the benefits and associated risks of proposed medical treatments3.67 (0.62)

    		Personal medical records are handled with confidentiality3.78 (0.48)

    		In case of disease, the physician explains various treatment options3.64 (0.65)

    		The physician involves me in decisions about upcoming examinations and treatments3.66 (0.63)

    		The physician conducts physical examinations of me thoroughly3.63 (0.66)

    		The physician’s office creates a well-organized impression3.70 (0.57)

    		The protection of my privacy is respected in the office3.74 (0.53)

    		The staff makes me feel welcome3.71 (0.56)

    		The physician’s office is nicely decorated3.63 (0.58)

    		Consultation time and absences are clearly communicated3.75 (0.52)

    		The physician regularly enquires about my tolerance of the prescribed medication3.40 (0.81)

    		The period between the first contact and the medical appointment is appropriate3.57 (0.65)

    		The medical equipment in the office creates a modern impression3.37 (0.71)

    		The waiting time before entering the physician’s office is adequate3.38 (0.76)

    		The waiting area offers enough space to maintain a distance from other patients3.33 (0.77)

    		Mentioning the reason for my visit in front of other patients is avoided3.50 (0.74)
    Overall evaluations

    		Overall impression3.11 (0.85)

    		Experience with results2.98 (0.86)

    		Willingness to recommend3.57 (0.82)

    		Willingness to revisit3.80 (0.62)

    In summary, we had access to a representative random sample containing 84,680 surveys of patients rating a total of 7038 general practitioners collected up to September 2014 (the PRW was launched in May 2011). The number of completed surveys for each physician is between 5 as a minimum and 82 as maximum. In the sample, the average number of completed surveys for each physician is 12 (SD 7). In Table 1, we summarize the means and standard deviations of the measures described previously. We treated the answer cannot be assessed as missing values for the service attributes in all subsequent analyses.

    Statistical Analysis

    A number of methods for identification of the 3-factor structure of customer satisfaction have been developed and applied outside health care research (for a review of these methods and their application, see the study by Arbore and Busacca [70]). The most widespread approach is the Penalty-Reward-Contrast Analysis introduced by Brandt [76]. One major criticism of this approach is the necessity to dichotomize the rating scales for the perceived attribute quality. Hereby, dummy variables are used only for low and high values of the measures to assess the nonlinear relationship between the perceived attribute quality and the overall evaluation (basic, performance, and excitement factors). This approach has been criticized because of the loss of information caused by dichotomizing the ends of the scale [77,78], but furthermore has to be linked to underestimation of effect sizes and an increased probability of type 2 errors [79].

    We used log-log regression models for our analyses. This modeling approach draws from econometric models of demand [80]. The slope coefficient from a log-log regression model identifies if an explanatory variable (ie, perceived attribute quality) has diminishing, constant, or increasing returns to a dependent variable (ie, overall evaluation). These 3 types of relationships have the previously described similarity to the 3-factor structure of customer satisfaction (see the study by Matzler and Sauerwein [71] for basic, performance, and excitement factors).

    To empirically identify the 3 different types of response patterns (ie, diminishing, constant, or increasing returns) using the log-log regression model, we first took the natural logarithm (ln) on both sides of a linear equation [80]: ln Y=b0+b1 ln X. Then, we estimated the parameters using ordinary least squares and b1 becomes the elasticity of Y with regard to X (ie, the percentage change in Y caused by a one percentage change in X, see the study by Varian [81]). The log-log regression model arrives at constant elasticities. This means that the magnitude of the elasticity obtained from our model is independent of the magnitudes of Y and X.

    Depending on the magnitude of the parameter estimate b1, we can empirically determine the type of (nonlinear) relationship between X and Y. If b1<1, the functional relationship is concave, and the attribute measured in X has diminishing returns to Y (ie, overall evaluation). If b1=1, the functional relationship is linear, with X having constant returns, and if b1>1, then the functional relationship is convex, where X has increasing returns. Applying the log-log regression model with perceived attribute quality as X and the measures of overall evaluation as Y (both transformed using the natural logarithm) allowed us to classify the service attributes into these 3 categories depending on the magnitude of b1. We used significance testing with H0: b1=1 to support the classification beyond the sole interpretation of the magnitude of b1. The parameter b0 serves as an intercept to account for the baseline level of ln Y (ie, the overall evaluation). We estimated log-log regression models with each service attribute in a single equation to allow for a different starting point of the curve for each X. This enables the functional relationships to be more flexibly positioned within the relationship between the perceived attribute quality and the overall evaluation. As each physician in the underlying database has at least 5 ratings for the service attributes and overall evaluations, we used physician-specific intercepts to further account for unobserved heterogeneity (so-called fixed effects, see Baltagi [82]). Therefore, the different intercepts in our model allowed for a different starting point of each service attribute as well as for each physician.

    Importantly, our proposed approach of a multiattribute model with nonlinear slope coefficients held a number of relevant assumptions. First, following previous research [83,84], we emphasized that the numerical ratings for our evaluation measures have to be assumed at ratio scale level. From this, our approach asked for a specific coding of the ratings to numerical data. The ratings have to be coded with 1 as the lowest possible numerical value and larger values that increase by 1 unit for each larger rating option. This setting is necessary for the data transformation using the natural logarithm in combination with our assumption of ratio scale level for the ratings to arrive at meaningful nonlinear slope coefficients. Any other coding will make the log-log regression model assume that the numerical values of the ratings are not ratio scale level and that there are values below the minimum when fitting the linear or nonlinear slope (eg, when coding 11-14 instead of 1-4, the estimates would take the range from 1 to 14 into account). Consequently, it is important to mention that our elasticities have to be interpreted within the range of X and Y used in our data. As usual in multiattribute models, we also assumed that all slope coefficients are positive and monotonically increasing.

    To test the robustness of the results from our approach, additional calculations were carried out: to show that our empirical findings do not rely on the log-log regression model only, we analyzed the data with 2 alternative approaches (the results of the robustness checks are available on request). In our first robustness check, we estimated elasticities from a standard linear-linear regression model by multiplying the resulting linear slope coefficients with the ratio of X and Y (=b1×[X/Y]) [80,81]. This approach does not result in constant elasticities but rather elasticities as a function of the values of X and Y (although we think that the latter is a less meaningful assumption). To show a comparison between these 2 approaches for elasticities in regression analysis, we computed the average elasticities from such a linear-linear regression model. Comparing these results shows that both approaches lead to the same classification (although there is aggregation bias in the linear-linear model because of using the average value across X and Y ratios). Importantly, the proportion of explained variance (R2) is systematically larger for the log-log regression models (our approach) compared with the linear-linear regression models (alternative approach). This outcome supports the position that our log-log regression model should be preferred for 2 reasons: (1) better fit to the data (in general for a log-log regression model with linear and nonlinear slopes compared with a linear-linear regression model with only linear slopes) and (2) more meaningful assumption because of constant elasticities. In our second robustness check, we employed dummy variable coding (with 1 as baseline) for each service attribute as explanatory variable X. This leads to 3 dummy variables for the rating values 2, 3, and 4. Using the overall evaluations as dependent variable, the slope coefficients of each of these dummy variables (and each service attribute as X) describe the average increase of the dependent variable as a function of the respective rating value compared with the lowest value (=1). Comparing the increase in the 3 slope coefficients for the 3 dummy variables (2 vs 1, 3 vs 1, and 4 vs 1) allowed us to detect diminishing, constant, or increasing returns to scale. If the average of the 2 slope coefficients 2 versus 1 and 4 versus 1 is below, equal, or above the slope coefficient of 3 versus 1, then this service attribute can be classified as having diminishing, constant, or increasing returns to the overall evaluations. Applying this approach led to the same classification as our approach using the log-log regression model. However, the dummy variable approach showed lower R2 compared with the log-log regressions models. In addition, the hypothesis testing cannot be carried out using 1 slope coefficient but has to be combined using 3 slope coefficients. This impedes straightforward hypothesis testing for the 3-factor model of customer satisfaction, which we employed in our research.


    Parameter Estimates and Model Diagnostics

    To assess the relationship between each service attribute and each overall evaluation, we estimated the log-log regression models with each service attribute in a single equation as proposed in the Methods section. This procedure offers 2 further advantages besides providing a different starting point of the curve for each service attribute and each physician. First, because of many service attributes in our setting that are correlated in their ratings, 1 multiple regression equation will produce severe multicollinearity problems. Second, as the answers cannot be assessed are flagged as missing values, using several explanatory variables at the same time will lead to case-wise deletion if just one of the service attributes has a missing value in their evaluation. Therefore, estimating models with each service attribute as a single explanatory variable will furthermore allow usage of all available information because of the pairwise consideration of nonmissing values of perceived attribute quality and the overall evaluations. Tables 2-5 provide the summary statistics of our log-log regression models. We provide the parameter estimates of b1 in Tables 2-5 along with the 95% CI for testing the hypothesis H0: b1=1. Bootstrapping is used for hypothesis testing to avoid biased standard errors because of the large sample size [85]. Therefore, we employed 1000 bootstrap replicates and show the 95% CI of this distribution in Tables 2-5 (bootstrapped [bs] 95% CI).

    Classification of a service attribute depends on 2 determinants: the size of b1 (below 1, around 1, or above 1) and its location with regard to the bs 95% CI. A b1<1 together with a bs 95% CI that does not include 1 classifies the corresponding service attribute as having diminishing returns to the overall evaluation. A b1>1 together with a bs 95% CI that does not include 1 classifies the service attribute as having increasing returns. Estimates of b1 around 1 with a bs 95% CI that includes 1 lead to a classification of the corresponding service attribute as having constant returns to the overall evaluations. Table 2 presents the b1 values for all of the 24 service attributes and the overall evaluation criterion of overall impression and Table 3 for the overall evaluation criterion of experience with results. Table 4 provides the b1 values for all the 24 service attributes and the overall evaluation criterion of willingness to recommend and Table 5 for the overall evaluation criterion of willingness to revisit. We list the parameter estimates and service attributes in Tables 2-5 in descending order of b1 for the models with overall impression as dependent variable. In all of the four tables (Table 2-5) we also present the proportion of explained variance (R2) and the number of observations used for estimation (N).

    Table 2. Parameter estimates and model diagnostics of log-log regression models 1 (sorted by the size of b).
    Service attributesOverall impression
    b (bsa 95% CI)R2Number of observations used, N
    The physician has a pleasant and friendly manner1.26 (1.19-1.34)0.3383,697
    The physician listens to me carefully1.09 (1.05-1.14)0.482,887
    The physician handles my questions, concerns, and fears in an empathetic way1.05 (1.02-1.09)0.4383,509
    The physician’s office is clean and neat1.05 (0.97-1.15)0.1482,924
    The physician indicates clearly how to take prescribed medication1.03 (0.97-1.09)0.2782,697
    The physician explains diagnoses, causes, and treatments so that I understand everything1.01 (0.97-1.05)0.3883,341
    The physician does not hurry during the medical treatment.97 (0.93-1.01)0.3983,393
    I have the impression that the physician will refer me to a specialist if this is medically necessary.93 (0.87-0.99)0.2783,123
    The physician explains exactly the benefits and associated risks of proposed medical treatments.92 (0.88-0.95)0.3981,536
    Personal medical records are handled with confidentiality.88 (0.81-0.96)0.1982,541
    In case of disease, the physician explains various treatment options.87 (0.84-0.91)0.480,961
    The physician involves me in decisions about upcoming examinations and treatments.87 (0.84-0.91)0.3780,343
    The physician conducts physical examinations of me thoroughly.86 (0.82-0.90)0.3882,936
    The physician’s office creates a well-organized impression.85 (0.80-0.91)0.2582,937
    The protection of my privacy is respected in the office.82 (0.75-0.88)0.2182,095
    The staff makes me feel welcome.80 (0.74-0.86)0.2283,254
    The physician’s office is nicely decorated.72 (0.66-0.77)0.1682,928
    Consultation time and absences are clearly communicated.71 (0.66-0.77)0.1581,931
    The physician regularly enquires about my tolerance of the prescribed medication.63 (0.60-0.66)0.3479,255
    The period between the first contact and the medical appointment is appropriate.59 (0.53-0.63)0.1781,183
    The medical equipment in the office creates a modern impression.59 (0.55-0.63)0.2174,618
    The waiting time before entering the physician’s office is adequate.52 (0.48-0.56)0.1882,839
    The waiting area offers enough space to maintain a distance from other patients.46 (0.43-0.50)0.1582,948
    Mentioning the reason for my visit in front of other patients is avoided.46 (0.42-0.50)0.1481,753

    abs: bootstrapped.

    Table 3. Parameter estimates and model diagnostics of log-log regression models 2 (sorted by the size of b for overall impression).
    Service attributesExperience with results
    b (bsa 95% CI)R2Number of observations used, N
    The physician has a pleasant and friendly manner1.18 (1.10-1.26)0.2783,625
    The physician listens to me carefully1.06 (1.01-1.11)0.3582,824
    The physician handles my questions, concerns, and fears in an empathetic way1.02 (0.98-1.07)0.3883,450
    The physician’s office is clean and neat1.03 (0.94-1.13)0.1382,857
    The physician indicates clearly how to take prescribed medication1.04 (0.98-1.10)0.2682,645
    The physician explains diagnoses, causes, and treatments so that I understand everything1.00 (0.95-1.04)0.3683,277
    The physician does not hurry during the medical treatment.95 (0.91-0.99)0.3583,332
    I have the impression that the physician will refer me to a specialist if this is medically necessary.94 (0.89-1.01)0.2683,068
    The physician explains exactly the benefits and associated risks of proposed medical treatments0.92 (0.88-0.96)0.3781,490
    Personal medical records are handled with confidentiality.87 (0.80-0.94)0.1882,483
    In case of disease, the physician explains various treatment options.88 (0.85-0.92)0.3880,922
    The physician involves me in decisions about upcoming examinations and treatments.87 (0.84-0.91)0.3580,298
    The physician conducts physical examinations of me thoroughly.87 (0.84-0.91)0.3782,877
    The physician’s office creates a well-organized impression.83 (0.78-0.89)0.2382,883
    The protection of my privacy is respected in the office.80 (0.74-0.86)0.1982,034
    The staff makes me feel welcome.78 (0.72-0.83)0.283,188
    The physician’s office is nicely decorated.71 (0.65-0.76)0.1582,860
    Consultation time and absences are clearly communicated.72 (0.66-0.78)0.1581,879
    The physician regularly enquires about my tolerance of the prescribed medication.64 (0.61-0.67)0.3479,206
    The period between the first contact and the medical appointment is appropriate.60 (0.55-0.64)0.1681,133
    The medical equipment in the office creates a modern impression.60 (0.56-0.64)0.274,581
    The waiting time before entering the physician’s office is adequate.52 (0.49-0.56)0.1782,779
    The waiting area offers enough space to maintain a distance from other patients.47 (0.43-0.50)0.1482,888
    Mentioning the reason for my visit in front of other patients is avoided.46 (0.42-0.50)0.1381,703

    abs: bootstrapped.

    Table 4. Parameter estimates and model diagnostics of log-log regression models 3 (sorted by the size of b for overall impression).
    Service attributesWillingness to recommend
    b (bsa 95% CI)R2Number of observations used, N
    The physician has a pleasant and friendly manner1.33 (1.25-1.42)0.3783,786
    The physician listens to me carefully1.16 (1.11-1.21)0.4682,978
    The physician handles my questions, concerns, and fears in an empathetic way1.12 (1.08-1.17)0.4983,606
    The physician’s office is clean and neat1.08 (0.99-1.17)0.1583,022
    The physician indicates clearly how to take prescribed medication1.09 (1.03-1.15)0.3182,795
    The physician explains diagnoses, causes, and treatments so that I understand everything1.06 (1.02-1.11)0.4383,445
    The physician does not hurry during the medical treatment1.00 (0.96-1.05)0.4283,480
    I have the impression that the physician will refer me to a specialist if this is medically necessary1.00 (0.94-1.05)0.3183,212
    The physician explains exactly the benefits and associated risks of proposed medical treatments.96 (0.92-1.01)0.4481,633
    Personal medical records are handled with confidentiality.91 (0.83-0.99)0.2182,628
    In case of disease, the physician explains various treatment options.92 (0.89-0.96)0.4581,056
    The physician involves me in decisions about upcoming examinations and treatments.92 (0.88-0.95)0.4280,432
    The physician conducts physical examinations of me thoroughly.90 (0.86-0.94)0.4183,025
    The physician’s office creates a well-organized impression.91 (0.85-0.97)0.2983,033
    The protection of my privacy is respected in the office.85 (0.79-0.91)0.2382,173
    The staff makes me feel welcome.87 (0.81-0.93)0.2683,349
    The physician’s office is nicely decorated.74 (0.68-0.79)0.1783,020
    Consultation time and absences are clearly communicated.74 (0.68-0.81)0.1782,028
    The physician regularly enquires about my tolerance of the prescribed medication.64 (0.61-0.68)0.3779,346
    The period between the first contact and the medical appointment is appropriate.60 (0.55-0.65)0.1881,268
    The medical equipment in the office creates a modern impression.58 (0.53-0.62)0.274,712
    The waiting time before entering the physician’s office is adequate.54 (0.50-0.59)0.282,932
    The waiting area offers enough space to maintain a distance from other patients.47 (0.43-0.51)0.1683,040
    Mentioning the reason for my visit in front of other patients is avoided.47 (0.43-0.51)0.1581,854

    abs: bootstrapped.

    Table 5. Parameter estimates and model diagnostics of log-log regression models 4 (sorted by the size of b for overall impression).
    Service attributesWillingness to revisit
    b (bsa 95% CI)R2Number of observations used, N
    The physician has a pleasant and friendly manner1.03 (0.96-1.10)0.3783,757
    The physician listens to me carefully.87 (0.82-0.92)0.4382,938
    The physician handles my questions, concerns, and fears in an empathetic way.84 (0.79-0.88)0.4683,572
    The physician’s office is clean and neat.77 (0.68-0.85)0.1383,008
    The physician indicates clearly how to take prescribed medication.79 (0.73-0.85)0.2882,767
    The physician explains diagnoses, causes, and treatments so that I understand everything.76 (0.72-0.81)0.3883,421
    The physician does not hurry during the medical treatment.72 (0.67-0.77)0.3683,449
    I have the impression that the physician will refer me to a specialist if this is medically necessary.74 (0.68-0.80)0.2983,198
    The physician explains exactly the benefits and associated risks of proposed medical treatments.68 (0.63-0.73)0.3881,604
    Personal medical records are handled with confidentiality.67 (0.59-0.74)0.1982,622
    In case of disease, the physician explains various treatment options.65 (0.60-0.69)0.3881,032
    The physician involves me in decisions about upcoming examinations and treatments.65 (0.61-0.70)0.3780,410
    The physician conducts physical examinations of me thoroughly.62 (0.57-0.66)0.3383,004
    The physician’s office creates a well-organized impression.65 (0.59-0.71)0.2583,016
    The protection of my privacy is respected in the office.61 (0.55-0.67)0.282,170
    The staff makes me feel welcome.64 (0.58-0.70)0.2483,325
    The physician’s office is nicely decorated.50 (0.45-0.55)0.1483,001
    Consultation time and absences are clearly communicated.52 (0.46-0.58)0.1482,007
    The physician regularly enquires about my tolerance of the prescribed medication.41 (0.38-0.45)0.2779,322
    The period between the first contact and the medical appointment is appropriate.40 (0.36-0.45)0.1481,262
    The medical equipment in the office creates a modern impression.36 (0.32-0.40)0.1474,689
    The waiting time before entering the physician’s office is adequate.35 (0.32-0.40)0.1482,914
    The waiting area offers enough space to maintain a distance from other patients.30 (0.26-0.34)0.1183,035
    Mentioning the reason for my visit in front of other patients is avoided.32 (0.28-0.36)0.1181,826

    abs: bootstrapped.

    Overall Impression and Experience With Results

    On the one hand, for overall impression, the first 3 service attributes in Table 2 show increasing returns, ie, b1>1 together with a bs 95% CI that does not include 1. These are, in descending order of b1, “The physician has a pleasant and friendly manner,” “The physician listens to me carefully,” and “The physician handles my questions, concerns, and fears in an empathetic way.” On the other hand, the service attributes “The physician’s office is clean and neat,” “The physician indicates clearly how to take prescribed medication,” “The physician explains diagnoses, causes, and treatments so that I understand everything,” and “The physician does not hurry during the medical treatment” show constant returns to overall impression as a dependent variable, ie, values of b1 around 1 together with a bs 95% CI that includes 1. All other service attributes have diminishing returns to overall impression, that is, they show values of b1 below 1 together with a bs 95% CI that does not include 1. This pattern is similar for experience with results (see Table 3) as overall evaluation except for “The physician handles my questions, concerns, and fears in an empathetic way” with constant instead of increasing returns and “The physician does not hurry during the medical treatment,” which has diminishing instead of constant returns. In addition, the service attribute “I have the impression that the physician will refer me to a specialist if this is medically necessary” has constant returns for the model with experience with results as a dependent variable.

    Willingness to Recommend and Willingness to Revisit

    For willingness to recommend, it can be observed in Table 4 that “The physician has a pleasant and friendly manner,” “The physician listens to me carefully,” and “The physician handles my questions, concerns, and fears in an empathetic way” have increasing returns but also “The physician indicates clearly how to take prescribed medication” and “The physician explains diagnoses, causes, and treatments so that I understand everything” are now added to this list. Here, once more, “The physician’s office is clean and neat” and “The physician does not hurry during the medical treatment” have constant returns, together with “I have the impression that the physician will refer me to a specialist if this is medically necessary” and “The physician explains exactly the benefits and associated risks of proposed medical treatments” when it comes to the willingness to recommend. For willingness to revisit (see Table 5), only “The physician has a pleasant and friendly manner” has constant returns, whereas all other service attributes have diminishing returns.


    Summary of Results and Comparison With Prior Work

    Collecting information reported by patients is necessary to make health care more customer oriented [86]. Consequently, analyzing online physician-rating data contributes to the body of knowledge in health care management. Our study makes an important contribution to this topic. We have access to a large number of online physician ratings, which allow a nuanced view on patient satisfaction. Our research goes beyond patient satisfaction (ie, overall impression and experience with results) by also looking at subsequent behavioral intentions that have important implications for physicians (ie, willingness to recommend and willingness to revisit). The empirical findings of our large-scale study are highly valuable for physicians because they identify service attributes that deserve an investment of resources. Analyzing perceived service quality helps to understand what patients think makes a good physician and what they value in addition to what medical training provides [87].

    The first important result of our study is that the more patients perceive the physician’s manner as being pleasant and friendly, the better is their overall impression as well as perceived experience with the results of the medical treatment. This relationship also applies to willingness to recommend as dependent variable. We demonstrated that improvements in these service attributes have increasing returns to the overall evaluation. Other service attributes with increasing returns with regard to overall impression, and willingness to recommend are being empathetic and listening carefully. Although previous studies about patient satisfaction [59,68,72,88-91] have also shown high importance of these factors, we can extend these findings by demonstrating that these service attributes have increasing returns. In addition, for willingness to recommend, it can be observed that “The physician indicates clearly how to take prescribed medication” and “The physician explains diagnoses, causes, and treatments so that I understand everything” also have increasing returns. Thus, communication behaviors of physicians that increase knowledge for patients have a large potential for increasing recommendation behavior if the service is fulfilled beyond average levels of satisfaction. In this context, it is important to mention that all starting points (ie, intercepts) of the latter service attributes are below the average level. Therefore, not fulfilling these services does, in fact, lead to dissatisfaction, whereas improving the perceived attribute quality above the average level leads to increasing returns to the overall evaluations. Another noteworthy finding is that the explanatory power of these models is considerably high (R2 between 0.27 and 0.49). This emphasizes the ability of these service attributes to influence the different overall evaluations.

    Van Oerle et al [92] argue that physicians are increasingly constrained by limited time and scarce budgets. This evokes a higher attractiveness of online health communities for patients to share their positive or negative experiences. Hence, physicians are well advised to make the most out of this limited time frame during the consultation. Physicians should consistently be friendly, pleasant, and empathetic and should listen carefully to their patients, despite time pressure and budget constraints. This corresponds with the findings of Berry and Bendapudi [93]. They asked patients by means of telephone interviews to recall the best and worst experiences that come into their minds with clinic doctors. Virtually, all the respondents referred to the physician’s behavior (the bedside manner, p 113) instead of the physician’s expertise or technical abilities. Berry and Bendapudi [93] argue that although technical skills are very important, they are more difficult to evaluate. Therefore, interpersonal skills appear to receive greater attention when it comes to evaluating the physician.

    Another finding of our study is that the service attribute “The physician’s office is clean and neat” has constant returns with respect to all overall evaluations (except willingness to revisit). Interestingly, in previous studies, this service attribute was found to have no significant impact on patient satisfaction [72,90] or a rather weak impact on overall quality evaluation [94]. The reasons for this change in patients’ preference between the studies from the 1990s and this study may be an increased knowledge and concern about the possibility of infections resulting from a visit to health care facilities where sick people congregate. Paddison et al [95] demonstrate this for hospital-based surroundings in which cleanliness plays a very important role for patients because of their concerns about infections. Thus, patients of physicians may transfer similar concerns to the primary care context and may have a higher awareness of infections resulting from a visit. A physician’s clean and neat office may signal to a patient that other patients’ germs and diseases are not transmitted easily. Hence, it should be emphasized that the results of this study indicate that cleanliness has constant returns with potential for improvements to the overall evaluations with the entire satisfaction range of perceived attribute quality.

    The results of our study also show that improving communication behaviors of physicians that increase knowledge for patients has constant returns. The service attribute “The physician indicates clearly how to take prescribed medication” and “The physician explains diagnoses, causes, and treatments so that I understand everything” shows constant returns for overall impression and experience with results. Such an influence on patient satisfaction is in line with previous research [96], as competence in communication is seen as a facet of medical competence [97]. When it comes to willingness to recommend, the service attributes “I have the impression that the physician will refer me to a specialist if this is medically necessary” and “The physician explains exactly the benefits and associated risks of proposed medical treatments” have constant returns. Again, the results corroborate previous research, and we are able to emphasize the importance of these service attributes to improve patient satisfaction and their willingness to recommend. Importantly, these models describing the service attributes with constant returns to scale also show considerably high explanatory power (R2 between 0.13 and 0.44). Lanjananda and Patterson [98] found significant predictors of nurses’ customer-oriented behavior: basic personality, customer orientation as surface trait, and nurses’ perceptions of the service climate and their commitment to the hospital. Thus, we can conclude that the physician’s personality, patient orientation, and their commitment are also important in explaining the degree of patient-oriented health care service.

    The results displayed at the lower end of Tables 2-5 reveal that extended waiting times for medical appointments, a lack of modernity of the medical equipment, and the facilities of the waiting area have diminishing returns. If fulfilled poorly, they are likely to have a strong negative impact on the overall evaluations. The most important of these service attributes with potential for decreased overall evaluation is related to indiscretion in the reception area or the waiting room. If a patient cannot state the reason for the visit without being overheard by others, this is likely to substantially reduce the overall evaluation. Privacy reflects perceptions that a patient’s intimacy may be compromised by the mere presence of others [99]. Respect of privacy was identified as the most important contributor to overall satisfaction by Carlucci et al [68]. This ties in with the results of our study. However, our approach allows for a more nuanced interpretation of the high importance of privacy: given that privacy shows diminishing returns in our results, it is highly likely that patients see privacy as a very basic factor and that lack of privacy leads to strong dissatisfaction. At the same time, the existence of privacy can only lead to average satisfaction but not to high levels of satisfaction with potential for excitement. Our results indicate that the abovementioned service attributes, in particular, privacy in the reception area or the waiting room, are all factors that patients appear to take for granted. Hence, absence or poor-quality levels are likely to substantially reduce overall evaluation, whereas high levels of fulfillment do not further contribute to patients’ overall evaluation.

    Implications for Health Care Management

    The number of PRWs is on the rise [8], and PRWs are becoming increasingly popular among patients [6]. Therefore, it is important to provide knowledge about what drives these publicly available overall evaluations. When PRWs collect and present information about patients’ experiences and satisfaction with individual physicians, our proposed approach can help physicians to classify the service attributes with regard to their returns, identify deficits, improve the quality of chosen service attributes, and stimulate improved ratings in the future. Monitoring these service attribute classifications (also over time) is, therefore, an important issue.

    Implications from our results for the service attributes with diminishing returns to the overall evaluation are as follows: a physician and his or her staff are well advised to work toward efficient patient scheduling, modern medical equipment, and a generously appointed waiting room to deliver personal space between the patients; and to ensure sufficient discretion at the reception desk to allow patients to state their reason for the visit without being overheard.

    If physicians want to improve their measures of overall evaluation on PRWs and aim to stand out from competitors, they are well advised to improve those service attributes that were shown to have constant and increasing returns. Many service attributes have diminishing returns with respect to patients’ overall evaluation of the physician. These factors still have great relevance for patients’ satisfaction because they lead to dissatisfaction if the perceived attribute quality is below the average level. All these service attributes should be seen as expected by patients to be at a satisfactory level, and therefore, delivering these standards is a prerequisite for patient satisfaction. However, further improvement of the perceived attribute quality beyond the average satisfaction level does not lead to substantial increases in overall evaluation because of the diminishing returns.

    In line with the claim to protect the voice of the patient needs [100], the advantage of using PRWs as a source of patients’ experiences is the condition of anonymity. Everyone posting a review on a PRW—at least in the case of the database we used for our study—can be secure in the knowledge that data privacy is taken seriously and that their evaluation will not influence future contact with the physician, at least on an individual level. On the other hand, the results of this study can be used by physicians to create patient delight—similar to customer delight [101,102], by focusing on the service attributes with increasing returns.

    Limitations

    This study has the following limitations that set the stage for future research opportunities. First, it should be recognized that the implications of our study are limited to a fixed set of attributes. This may have the potential to divert physicians’ attention away from other important aspects of health care [59]. Therefore, it is important for future research not only to avoid the exclusion of relevant service attributes but also to account for other aspects of service quality that may not be perceived by patients when it comes to improving health care management. Second, the average values of the perceived service quality are high and show a tendency toward high satisfaction in our sample (see Table 1). Such a response bias is well known in patient satisfaction surveys from Web-based PRWs [9,30,38] and thus is in accordance with these existing studies.

    With regard to the time frame of the data collection, the focus was set on the introductory phase of the PRW (May 2011 to September 2014), and the large-scale data were drawn from the PRW in September 2014. In the meantime, patients can only post their rating on the PRW and not offline as was possible during the earlier phase of the PRW. Thus, data available in the introductory phase should cover the evaluation and spectrum of opinions of a large range of patients throughout the whole population. This specific point of data collection, therefore, reflects the broad range of experiences of the patient-physician encounter from a representative sample of the total population quite well (both online and offline population segments). This provides the opportunity to use this initial phase of a PRW’s large-scale data as a reference point for further studies. Especially, a longitudinal setting would deliver fruitful insights into developments of categorization over time, bearing in mind that the introductory phase was also characterized by an additional opportunity for patients to rate physicians through mail. Thus, using the large-scale data, the results of this study deliver an important reference point to monitor patients’ evaluation of physicians over time.

    Regardless of the limitations discussed previously, the relevance of all derived implications is still high for health care management because of the fact that all the ratings on PRWs are publicly available and can influence patients in their choice of a physician.

    Acknowledgments

    The authors are grateful to Prof Martin Emmert for connecting their research team with Weisse Liste gemeinnützige GmbH, Gütersloh, Germany (Weisse Liste). The authors are especially thankful to Dr Marko Queitsch from Weisse Liste for providing the data and helpful comments for their study.

    Conflicts of Interest

    None declared.

    1. Zeithaml VA, Berry LL, Parasuraman A. The nature and determinants of customer expectations of service. J Acad Mark Sci 1993;21(1):1-12. [CrossRef]
    2. Roettl J, Bidmon S, Terlutter R. What predicts patients' willingness to undergo online treatment and pay for online treatment? Results from a web-based survey to investigate the changing patient-physician relationship. J Med Internet Res 2016 Feb 4;18(2):e32 [FREE Full text] [CrossRef] [Medline]
    3. Fox S. Pew Research Center. Washington, DC: Pew Research Center; 2014 Jan 15. The Social Life of Health Information   URL: http://www.pewinternet.org/~/media//Files/Reports/2011/PIP_Social_Life_of_Health_Info.pdf [accessed 2013-11-20]
    4. Laugesen J, Hassanein K, Yuan Y. The impact of internet health information on patient compliance: a research model and an empirical study. J Med Internet Res 2015 Jun 11;17(6):e143 [FREE Full text] [CrossRef] [Medline]
    5. Bidmon S, Terlutter R. Gender differences in searching for health information on the internet and the virtual patient-physician relationship in Germany: exploratory results on how men and women differ and why. J Med Internet Res 2015 Jun 22;17(6):e156 [FREE Full text] [CrossRef] [Medline]
    6. Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. Public awareness, perception, and use of online physician rating sites. J Am Med Assoc 2014 Feb 19;311(7):734-735. [CrossRef] [Medline]
    7. Emmert M, Meier F. An analysis of online evaluations on a physician rating website: evidence from a German public reporting instrument. J Med Internet Res 2013 Aug 6;15(8):e157 [FREE Full text] [CrossRef] [Medline]
    8. Emmert M, Meier F, Pisch F, Sander U. Physician choice making and characteristics associated with using physician-rating websites: cross-sectional study. J Med Internet Res 2013 Aug 28;15(8):e187 [FREE Full text] [CrossRef] [Medline]
    9. Terlutter R, Bidmon S, Röttl J. Who uses physician-rating websites? Differences in sociodemographic variables, psychographic variables, and health status of users and nonusers of physician-rating websites. J Med Internet Res 2014 Mar 31;16(3):e97 [FREE Full text] [CrossRef] [Medline]
    10. Bidmon S, Terlutter R, Röttl J. What explains usage of mobile physician-rating apps? Results from a web-based questionnaire. J Med Internet Res 2014 Jun 11;16(6):e148 [FREE Full text] [CrossRef] [Medline]
    11. Verhoef LM, van de Belt TH, Engelen LJ, Schoonhoven L, Kool RB. Social media and rating sites as tools to understanding quality of care: a scoping review. J Med Internet Res 2014 Feb 20;16(2):e56 [FREE Full text] [CrossRef] [Medline]
    12. Gill L, White L. A critical review of patient satisfaction. Leadersh Health Serv 2009;22(1):8-19. [CrossRef]
    13. Chang WJ, Chang YH. Patient satisfaction analysis: identifying key drivers and enhancing service quality of dental care. J Dent Sci 2013;8(3):239-247. [CrossRef]
    14. Davis J, Burrows JF, Khallouq BB, Rosen P. Predictors of patient satisfaction in pediatric oncology. J Pediatr Oncol Nurs 2017;34(6):435-438. [CrossRef] [Medline]
    15. Hattenberger G, Pechlaner H, Matzler K. Lifestyle-Typologies and Market Segmentation: The Case of Alpine Skiing Tourism. Bolzano, Italy: Eurac Research; 2004.
    16. Mikulic J, Prebežac D. A critical review of techniques for classifying quality attributes in the Kano model. Manag Serv Qual Int J 2011;21(1):46-66. [CrossRef]
    17. Emmert M, Sander U, Pisch F. Eight questions about physician-rating websites: a systematic review. J Med Internet Res 2013 Feb 1;15(2):e24 [FREE Full text] [CrossRef] [Medline]
    18. McLennan S, Strech D, Meyer A, Kahrass H. Public awareness and use of German physician ratings websites: cross-sectional survey of four north German cities. J Med Internet Res 2017 Nov 9;19(11):e387 [FREE Full text] [CrossRef] [Medline]
    19. McLennan S, Strech D, Kahrass H. Why are so few patients rating their physicians on German physician rating websites? A qualitative study. BMC Health Serv Res 2018 Aug 29;18(1):670 [FREE Full text] [CrossRef] [Medline]
    20. Emmert M, Sander U, Esslinger AS, Maryschok M, Schöffski O. Public reporting in Germany: the content of physician rating websites. Methods Inf Med 2012;51(2):112-120. [CrossRef] [Medline]
    21. Emmert M, Sauter L, Jablonski L, Sander U, Taheri-Zadeh F. Do physicians respond to web-based patient ratings? An analysis of physicians' responses to more than one million web-based ratings over a six-year period. J Med Internet Res 2017 Jul 26;19(7):e275 [FREE Full text] [CrossRef] [Medline]
    22. Emmert M, Meszmer N, Sander U. Do health care providers use online patient ratings to improve the quality of care? Results from an online-based cross-sectional study. J Med Internet Res 2016 Sep 19;18(9):e254 [FREE Full text] [CrossRef] [Medline]
    23. Greaves F, Pape UJ, King D, Darzi A, Majeed A, Wachter RM, et al. Associations between Web-based patient ratings and objective measures of hospital quality. Arch Intern Med 2012 Mar 12;172(5):435-436. [CrossRef] [Medline]
    24. Greaves F, Millett C. Consistently increasing numbers of online ratings of healthcare in England. J Med Internet Res 2012 Jun 29;14(3):e94 [FREE Full text] [CrossRef] [Medline]
    25. Greaves F, Ramirez-Cano D, Millett C, Darzi A, Donaldson L. Use of sentiment analysis for capturing patient experience from free-text comments posted online. J Med Internet Res 2013 Nov 1;15(11):e239 [FREE Full text] [CrossRef] [Medline]
    26. Powell J, Boylan AM, Greaves F. Harnessing patient feedback data: a challenge for policy and service improvement. Digit Health 2015;1:2055207615617910 [FREE Full text] [CrossRef] [Medline]
    27. Greaves F, Ramirez-Cano D, Millett C, Darzi A, Donaldson L. Harnessing the cloud of patient experience: using social media to detect poor quality healthcare. BMJ Qual Saf 2013 Mar;22(3):251-255. [CrossRef] [Medline]
    28. Patel S, Cain R, Neailey K, Hooberman L. Exploring patients' views toward giving web-based feedback and ratings to general practitioners in England: a qualitative descriptive study. J Med Internet Res 2016 Aug 5;18(8):e217 [FREE Full text] [CrossRef] [Medline]
    29. Patel S, Cain R, Neailey K, Hooberman L. General practitioners' concerns about online patient feedback: findings from a descriptive exploratory qualitative study in England. J Med Internet Res 2015 Dec 8;17(12):e276 [FREE Full text] [CrossRef] [Medline]
    30. Galizzi MM, Miraldo M, Stavropoulou C, Desai M, Jayatunga W, Joshi M, et al. Who is more likely to use doctor-rating websites, and why? A cross-sectional study in London. BMJ Open 2012;2(6):pii: e001493 [FREE Full text] [CrossRef] [Medline]
    31. Rothenfluh F, Schulz PJ. Content, quality, and assessment tools of physician-rating websites in 12 countries: quantitative analysis. J Med Internet Res 2018 Jun 14;20(6):e212 [FREE Full text] [CrossRef] [Medline]
    32. Rothenfluh F, Schulz PJ. Physician rating websites: what aspects are important to identify a good doctor, and are patients capable of assessing them? A mixed-methods approach including physicians' and health care consumers' perspectives. J Med Internet Res 2017 May 1;19(5):e127 [FREE Full text] [CrossRef] [Medline]
    33. Rothenfluh F, Germeni E, Schulz PJ. Consumer decision-making based on review websites: are there differences between choosing a hotel and choosing a physician? J Med Internet Res 2016 Jun 16;18(6):e129 [FREE Full text] [CrossRef] [Medline]
    34. Damman OC, van den Hengel YK, van Loon AJ, Rademakers J. An international comparison of web-based reporting about health care quality: content analysis. J Med Internet Res 2010 Apr 13;12(2):e8 [FREE Full text] [CrossRef] [Medline]
    35. Emmert M, Adelhardt T, Sander U, Wambach V, Lindenthal J. A cross-sectional study assessing the association between online ratings and structural and quality of care measures: results from two German physician rating websites. BMC Health Serv Res 2015 Sep 24;15:414 [FREE Full text] [CrossRef] [Medline]
    36. Burkle CM, Keegan MT. Popularity of internet physician rating sites and their apparent influence on patients' choices of physicians. BMC Health Serv Res 2015 Sep 26;15:416 [FREE Full text] [CrossRef] [Medline]
    37. Gao GG, McCullough JS, Agarwal R, Jha AK. A changing landscape of physician quality reporting: analysis of patients' online ratings of their physicians over a 5-year period. J Med Internet Res 2012 Feb 24;14(1):e38 [FREE Full text] [CrossRef] [Medline]
    38. Lagu T, Hannon NS, Rothberg MB, Lindenauer PK. Patients' evaluations of health care providers in the era of social networking: an analysis of physician-rating websites. J Gen Intern Med 2010 Sep;25(9):942-946 [FREE Full text] [CrossRef] [Medline]
    39. Trehan SK, Nguyen JT, Marx R, Cross MB, Pan TJ, Daluiski A, et al. Online patient ratings are not correlated with total knee replacement surgeon-specific outcomes. HSS J 2018 Jul;14(2):177-180 [FREE Full text] [CrossRef] [Medline]
    40. Chen J, Presson A, Zhang C, Ray D, Finlayson S, Glasgow R. Online physician review websites poorly correlate to a validated metric of patient satisfaction. J Surg Res 2018 Jul;227:1-6. [CrossRef] [Medline]
    41. Liu JJ, Matelski JJ, Bell CM. Scope, breadth, and differences in online physician ratings related to geography, specialty, and year: observational retrospective study. J Med Internet Res 2018 Mar 7;20(3):e76 [FREE Full text] [CrossRef] [Medline]
    42. Li J, Liu M, Li X, Liu X, Liu J. Developing embedded taxonomy and mining patients' interests from web-based physician reviews: mixed-methods approach. J Med Internet Res 2018 Aug 16;20(8):e254 [FREE Full text] [CrossRef] [Medline]
    43. Hao H, Zhang K. The voice of Chinese health consumers: a text mining approach to web-based physician reviews. J Med Internet Res 2016 May 10;18(5):e108 [FREE Full text] [CrossRef] [Medline]
    44. Hao H, Zhang K, Wang W, Gao G. A tale of two countries: international comparison of online doctor reviews between China and the United States. Int J Med Inform 2017 Mar;99:37-44. [CrossRef] [Medline]
    45. Hao H. The development of online doctor reviews in China: an analysis of the largest online doctor review website in China. J Med Internet Res 2015 Jun 1;17(6):e134 [FREE Full text] [CrossRef] [Medline]
    46. Zhang W, Deng Z, Hong Z, Evans R, Ma J, Zhang H. Unhappy patients are not alike: content analysis of the negative comments from China's good doctor website. J Med Internet Res 2018 Jan 25;20(1):e35 [FREE Full text] [CrossRef] [Medline]
    47. Fischer S, Emmert M. A review of scientific evidence for public perspectives on online rating websites of healthcare providers. In: Gurtner S, Soyez K, editors. Challenges And Opportunities in Health Care Management. Cham: Springer; 2015:279-290.
    48. Greaves F, Pape UJ, King D, Darzi A, Majeed A, Wachter RM, et al. Associations between internet-based patient ratings and conventional surveys of patient experience in the English NHS: an observational study. BMJ Qual Saf 2012 Jul;21(7):600-605. [CrossRef] [Medline]
    49. Greaves F, Pape UJ, Lee H, Smith DM, Darzi A, Majeed A, et al. Patients' ratings of family physician practices on the internet: usage and associations with conventional measures of quality in the English National Health Service. J Med Internet Res 2012 Oct 17;14(5):e146 [FREE Full text] [CrossRef] [Medline]
    50. Holliday AM, Kachalia A, Meyer GS, Sequist TD. Physician and patient views on public physician rating websites: a cross-sectional study. J Gen Intern Med 2017 Jun;32(6):626-631 [FREE Full text] [CrossRef] [Medline]
    51. McLennan S, Strech D, Reimann S. Developments in the frequency of ratings and evaluation tendencies: A review of German physician rating websites. J Med Internet Res 2017 Aug 25;19(8):e299 [FREE Full text] [CrossRef] [Medline]
    52. Patel S, Cain R, Neailey K, Hooberman L. Public awareness, usage, and predictors for the use of doctor rating websites: cross-sectional study in England. J Med Internet Res 2018 Jul 25;20(7):e243 [FREE Full text] [CrossRef] [Medline]
    53. Totten AM, Wagner J, Tiwari A, O'Haire C, Griffin J, Walker M. Closing the quality gap: revisiting the state of the science (vol. 5: public reporting as a quality improvement strategy). Evid Rep Technol Assess (Full Rep) 2012 Jul(208.5):1-645. [Medline]
    54. Meszmer N, Jaegers L, Schöffski O, Emmert M. Spiegeln Onlinebewertungen Strukturunterschiede in der Versorgung wider? Eine Analyse am Beispiel von Arztbewertungsportalen [Do online ratings reflect structural differences in healthcare? The example of German physician-rating websites]. Z Evid Fortbild Qual Gesundhwes 2018 Apr;131-132:73-80. [CrossRef] [Medline]
    55. Emmert M, Meier F, Heider AK, Dürr C, Sander U. What do patients say about their physicians? An analysis of 3000 narrative comments posted on a German physician rating website. Health Policy 2014 Oct;118(1):66-73. [CrossRef] [Medline]
    56. Guo Y, Barnes SJ, Jia Q. Mining meaning from online ratings and reviews: tourist satisfaction analysis using latent dirichlet allocation. Tour Manag 2017;59:467-483. [CrossRef]
    57. Healthgrades.   URL: https://www.healthgrades.com/ [accessed 2019-12-13]
    58. Otani K, Waterman B, Faulkner KM, Boslaugh S, Burroughs TE, Dunagan WC. Patient satisfaction: focusing on 'excellent'. J Healthc Manag 2009;54(2):93-102; discussion 102. [Medline]
    59. Schlesinger M, Grob R, Shaller D. Using patient-reported information to improve clinical practice. Health Serv Res 2015 Dec;50(Suppl 2):2116-2154 [FREE Full text] [CrossRef] [Medline]
    60. Oliver RL. Satisfaction: A Behavioral Perspective On The Consumer. New York: McGraw-Hill; 1997.
    61. Oliver RL. A cognitive model of the antecedents and consequences of satisfaction decisions. J Mark Res 1980;17(4):460-469. [CrossRef]
    62. Leisen B, Hyman MR. Antecedents and consequences of trust in a service provider: the case of primary care physicians. J Bus Res 2004;57(9):990-999. [CrossRef]
    63. Mummalaneni V, Gopalakrishna P. Access, resource, and cost impacts on consumer satisfaction with health care: a comparison across alternative health care modes and time. J Bus Res 1997;39(3):173-186. [CrossRef]
    64. Churchill GA, Surprenant C. An investigation into the determinants of customer satisfaction. J Mark Res 1982;19(4):491-504. [CrossRef]
    65. Anderson EW, Sullivan MW. The antecedents and consequences of customer satisfaction for firms. Mark Sci 1993;12(2):125-143. [CrossRef]
    66. Wilkie WL, Pessemier EA. Issues in marketing's use of multi-attribute attitude models. J Mark Res 1973;10(4):428-441. [CrossRef]
    67. van Ittersum K, Pennings JM, Wansink B, van Trijp HC. The validity of attribute-importance measurement: a review. J Bus Res 2007;60(11):1177-1190. [CrossRef]
    68. Carlucci D, Renna P, Schiuma G. Evaluating service quality dimensions as antecedents to outpatient satisfaction using back propagation neural network. Health Care Manag Sci 2013 Mar;16(1):37-44. [CrossRef] [Medline]
    69. Huang R, Sarigöllü E. Assessing satisfaction with core and secondary attributes. J Bus Res 2008;61(9):942-949. [CrossRef]
    70. Arbore A, Busacca B. Customer satisfaction and dissatisfaction in retail banking: exploring the asymmetric impact of attribute performances. J Retail Consum Serv 2009;16(4):271-280. [CrossRef]
    71. Matzler K, Sauerwein E. The factor structure of customer satisfaction: an empirical test of the importance grid and the penalty-reward-contrast analysis. Int J Serv Ind Manag 2002;13(4):314-332. [CrossRef]
    72. Mittal V, Ross WT, Baldasare PM. The asymmetric impact of negative and positive attribute-level performance on overall satisfaction and repurchase intentions. J Mark 1998;62(1):33-47. [CrossRef]
    73. Harrison-Walker LJ. The measurement of word-of-mouth communication and an investigation of service quality and customer commitment as potential antecedents. J Serv Res 2001;4(1):60-75. [CrossRef]
    74. Weisse Liste.   URL: https://www.weisse-liste.de/de/ [accessed 2019-12-13]
    75. Fischer S. Project 'Weisse Liste': a German best practice example for online provider ratings in health care. In: Gurtner S, Soyez K, editors. Challenges and Opportunities in Health Care Management. Cham: Springer; 2015:339-346.
    76. Brandt RD. How service marketers can identify value-enhancing service elements. J Serv Mark 1988;2(3):35-41. [CrossRef]
    77. Albayrak T, Caber M. Penalty–Reward-Contrast Analysis: a review of its application in customer satisfaction research. Total Qual Manag Bus Excel 2013;24(11-12):1288-1300. [CrossRef]
    78. Sampson SE, Showalter MJ. The performance-importance response function: observations and implications. Serv Ind J 1999;19(3):1-25. [CrossRef]
    79. Rucker DD, McShane BB, Preacher KJ. A researcher's guide to regression, discretization, and median splits of continuous variables. J Consum Psychol 2015;25(4):666-678. [CrossRef]
    80. Hanssens DM, Parsons LJ, Schultz RL. Market Response Models: Econometric and Time Series Analysis. Boston, MA: Kluwer Academic Publishers; 2001.
    81. Varian HR. Intermediate Microeconomics: A Modern Approach. Eighth Edition. New York: WW Norton & Company; 2010.
    82. Baltagi B. Econometrics. Fifth Edition. Chichester, West Sussex: John Wiley & Sons; 2013.
    83. Michell J. Measurement scales and statistics: a clash of paradigms. Psychol Bull 1986;100(3):398-407. [CrossRef]
    84. Zulueta J, Piscitello A, Rasic M, Easter R, Babu P, Langenecker SA, et al. Predicting mood disturbance severity with mobile phone keystroke metadata: a BiAffect digital phenotyping study. J Med Internet Res 2018 Jul 20;20(7):e241 [FREE Full text] [CrossRef] [Medline]
    85. Efron B, Tibshirani R. Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Stat Sci 1986;1(1):54-75. [CrossRef]
    86. Sofaer S, Firminger K. Patient perceptions of the quality of health services. Annu Rev Public Health 2005;26:513-559. [CrossRef] [Medline]
    87. Jain S. Googling ourselves--what physicians can learn from online rating sites. N Engl J Med 2010 Jan 7;362(1):6-7. [CrossRef] [Medline]
    88. Bacon N. Will doctor rating sites improve standards of care? Yes. BMJ 2009 Mar 17;338:b1030. [CrossRef] [Medline]
    89. Bartlett EE, Grayson M, Barker R, Levine DM, Golden A, Libber S. The effects of physician communications skills on patient satisfaction; recall, and adherence. J Chronic Dis 1984;37(9-10):755-764. [CrossRef] [Medline]
    90. Mittal V, Baldasare PM. Eliminate the negative. Managers should optimize rather than maximize performance to enhance patient satisfaction. J Health Care Mark 1996;16(3):24-31. [Medline]
    91. Robbins JA, Bertakis KD, Helms LJ, Azari R, Callahan EJ, Creten DA. The influence of physician practice behaviors on patient satisfaction. Fam Med 1993 Jan;25(1):17-20. [Medline]
    92. van Oerle S, Mahr D, Lievens A. Coordinating online health communities for cognitive and affective value creation. J Serv Manag 2016;27(4):481-506. [CrossRef]
    93. Berry LL, Bendapudi N. Health care: a fertile field for service research. J Serv Res 2007;10(2):111-122. [CrossRef]
    94. Seibert JH, Strohmeyer JM, Carey RG. Evaluating the physician office visit: in pursuit of a valid and reliable measure of quality improvement efforts. J Ambul Care Manage 1996 Jan;19(1):17-37. [CrossRef] [Medline]
    95. Paddison CA, Abel GA, Roland MO, Elliott MN, Lyratzopoulos G, Campbell JL. Drivers of overall satisfaction with primary care: evidence from the English General Practice Patient Survey. Health Expect 2015 Oct;18(5):1081-1092 [FREE Full text] [CrossRef] [Medline]
    96. Comstock LM, Hooper EM, Goodwin JM, Goodwin JS. Physician behaviors that correlate with patient satisfaction. J Med Educ 1982 Mar;57(2):105-112. [Medline]
    97. Buller MK, Buller DB. Physicians' communication style and patient satisfaction. J Health Soc Behav 1987 Dec;28(4):375-388. [CrossRef] [Medline]
    98. Lanjananda P, Patterson PG. Determinants of customer‐oriented behavior in a health care context. J Serv Manag 2009;20(1):5-32. [CrossRef]
    99. Ponsignon F, Smart A, Williams M, Hall J. Healthcare experience quality: an empirical exploration using content analysis techniques. J Serv Manag 2015;26(3):460-485. [CrossRef]
    100. Elg M, Engström J, Witell L, Poksinska B. Co‐creation and learning in health‐care service development. J Serv Manag 2012;23(3):328-343. [CrossRef]
    101. Rust RT, Oliver RL. Should we delight the customer? J Acad Mark Sci 2000;28(1):86-94. [CrossRef]
    102. Oliver RL, Rust RT, Varki S. Customer delight: foundations, findings and managerial insight. J Retail 1997;73(3):311-336. [CrossRef]

    ‎
    bs: bootstrapped
    PRW: physician-rating website

    Edited by T Kool; submitted 27.02.19; peer-reviewed by S McLennan, U Konerding, S Geletta; comments to author 23.04.19; revised version received 18.06.19; accepted 18.11.19; published 03.02.20

    Copyright

    ©Sonja Bidmon, Ossama Elshiewy, Ralf Terlutter, Yasemin Boztug. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 03.02.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.