In principle a prediction market should generate more accurate predictions than a simple poll. For example, in an election, the outcome of a poll should be known to the traders and incorporated into their trades. But in practice, the advantage of prediction markets is small, or so suggests a new study.

In a new study, Daniel Reeves, Duncan Watts, Dave Pennock and I compare the performance of prediction markets to conventional means of forecasting, namely polls and statistical models. Examining thousands of sporting and movie events, we find that the relative advantage of prediction markets is remarkably small.For example, the Las Vegas market for professional football is only 3% more accurate in predicting final game scores than a simple, three parameter statistical model, and the market is only 1% better than a poll of football enthusiasts.

More here. My view is that there is no theoretical reason for interpreting a market price in a prediction market as a probability. That is, if Coakley is trading at 75 cents there is no reason to interpret that as a 75 percent probability that Coakley will win. At best there is an ordinal relationship: a higher price means a higher probability. Likewise with a poll.

Studies like these should instead be measuring the conditional probability of an event given the price observed in the market. Because, even an “innacurate” prediction can be a very good one. To take an extreme case the market might always underprice the probability of a Coakley win by 25%. But then by dividing the market price by .75 we can infer the right probability with perfect accuracy.

## 7 comments

Comments feed for this article

January 21, 2010 at 6:13 am

Linkage for 1.21.2010 « bill | petti[…] that the relative advantage of prediction markets is remarkably small.” Jeff at Cheap Talk offers a methodological critique of the […]

January 21, 2010 at 8:56 am

PJ HealyFor similar results, see also:

Erikson, Robert S. and Wlezien, Christopher (2008) Are political markets really superior to polls as election predictors?, Public Opinion Quarterly, 72(2), 190–215.

January 21, 2010 at 9:41 am

Eransomething i don’t understand in your argument: if it was the case that the market always underprice the probability by 25% then wouldn’t some canny trader be making money of the rest of us ?

it seems to me that if you had a theoretical way to support or even give a meaning to a statement like `when the price is p, then correct probability is f(p)’ then standard economic reasoning would provide the theoretical reason to say that the price will be a fixed point of f.

January 21, 2010 at 4:40 pm

jeffThese studies equate probabilty with relative frequency. So let’s use that. The price could systematically diverge from relative frequency because traders aren’t maximizing expected payoffs evaluated according to relative frequency.

January 21, 2010 at 7:30 pm

Rajiv SethiJeff, if Coakley is trading at 75 cents to the dollar then arbitrage considerations imply that Brown is trading at 25 (net of transactions costs, which are at most 0.3% on Intrade). So it the former is overpriced (relative to the probability of occurrence) then the latter must be underpriced. There is no reason to expect this kind of asset-specific variation in the sign of the deviation between prices and probabilities. This is why most market participants and observers (myself included) do indeed interpret prices as reflecting the subjective probabilities of the marginal trader, who is indifferent between buying and selling at these prices.

January 21, 2010 at 8:23 pm

jeffThat is a good argument rajiv. It does assume that the traders are “betting” based on beliefs, which I am not convinced of. Very few people, even amond traders (maybe especially among traders) know what a probability is let alone understand how it is connected to betting. But I do like the argument.

January 27, 2010 at 3:57 pm

Daniel ReevesI agree with Rajiv about the theoretical reasons to expect that prices equal probabilities. There are reasons for the two to diverge for certain traders (eg, some traders are using the market to hedge) but that amounts to free money for other traders until prices and probabilities match. This is basically the Efficient Market Hypothesis.

Note that our paper doesn’t refute that. It only says that non-market methods like polls and simple statistical models seem to do just about as well.