There is a study by some economists and statisticans on the correlation between the price of a wine and ratings in blind tastings by tasters who are not informed of the price. The headline result in the paper is that higher priced wines don’t get higher ratings. If anything they get lower ratings. It is typically used in the first paragraph of blog posts to set up various theories about how people use price information to tell themselves what they should and shouldn’t like. (For example, here’s Jonah Lehrer.)
But why should we expect higher priced wine to get higher ratings in tastings? Suppose there are 100 different styles of wine and for every different style there is a group that likes that style and only that style. There will be a lot of variation in the price of different styles because the price will depend on the supply of that style and the size of the group that likes that style. Now ask a person to taste a randomly selected wine and rate it. There will be no correlation between price and ratings.
There are many styles of cheese with different prices. Would we expect the price of cheese to predict ratings in blind tastings?
Here’s another variation on the same idea. Suppose there are just two styles of wine, subtle and not-so-subtle. Some people appreciate the subtlety but most don’t. Suppose that the supply of subtle wine is lower so that its price is higher. Then again a study like this will produce an overall negative correlation between price and ratings.
And indeed if you read past page 3 of the paper you see that an effect like this is in the data.
Our data also indicates that experts, unlike non-experts, on average assign as high – or higher – ratings to more expensive wines. The coefficient on the expert*price interaction term is positive and highly statistically significant. The price coefficient for non-experts is negative, and about the same size as in the baseline model. The net coefficient on price for experts is the sum of these two coefficients. It is positive and marginally statistically significant.
The linear estimator offers an interpretation of these effects. In terms of a 100 point scale (such as that used by Wine Spectator), the extended model predicts that for a wine that costs ten times more than another wine, non-experts will on average assign an overall rating that is about four points lower, whereas experts will assign an overall rating that is about seven points higher.