Misunderstanding Prediction Market Failures

Wednesday, February 14, 2007 | 07:23 AM

Every now and again, I read something that simply makes my head spin. This morning’s NYT article on prediction markets was just such a head-spinning column. It reached such a bizarre conclusion that it begs for comment.

Longtime readers are familiar with my views on prediction markets. I believe they have some value, when applied correctly. Where prediction markets excel is in acting as a collective real time polling mechanism for their participants. The closer the collective group is to the broader population, the more accurate these markets tend to be. Where they do poorly is when the collective attempt to “pool their ignorance” and forecast the random or the unknowable.

There is strength in any mechanism that can factor a large pool of participants collective experience and knowledge, discount several possibilities into the most likely outcome. These market mechanisms are hardly the Oracles of Delphi their supporters make them out to be. It is only by din of Humans’ even worse forecasting abilities (with experts leading the way down) that these markets garner so much respectability in the first place. Blame fuzzy thinking for placing too much credibility on a mechanism with so mixed a track record. There is also a subtle but important distinction between forecasting the future versus discounting various outcomes

0214bizleonhardt_1Which leads me to today’s NYT column. In a bizarre twist of logic, the massive failure of the prediction markets in the US 2006 mid-term elections somehow gets credit for being right. They were not a day late and a dollar short, they were completely, totally and incontrovertibly wrong. 

Except in the NYTimes.

Consider the following: As of 11:50 p.m. on the evening of Election day, with the voting completed (except for Hawaii) and the majority of the ballots counted, Intrade gave the Republicans an 85 percent chance of retaining the Senate. Of course, we know that’s not at all what happened. If that’s not a forecasting failure, then what is?

If forecasting the results of an election held yesterday is considered prescience, then sign me up -- and find me a bookie who will take my Superbowl bets on the Monday after the game. I suspect I will at least cover the spread.

To reiterate my views, these markets -- thinly traded, easily manipulated, poorly administered -- do have  some real value. However, they have hardly been "remarkably clairvoyant," as the Times describes them. Consider these recent acts of "clairvoyance:"

GOP Retention of Senate
Michael Jackson Trial Results
Morgan Stanley CEO Purcell resignation
Howard Dean’s Iowa Primary
pre-Election 2004
Election day trading frenzy

There are some interesting explanations for some of these errors: Pooling the collective ignorance of millions does not produce Wisdom. The Michael Jackson Trial and Morgan Stanley's CEO are perfect examples of that.

As to the prediction market's failure of the Howard Dean primary and the 2006 mid-term elections, I've been toying with a theory. Since the political affiliation of Wall Street tends toward Republican, and since there is a big overlap between Wall Street and the participants of the prediction markets, the collective had an inherent bias built into it. Remember, where these markets excel is when they act as a realtime polling mechanism of their particpants. If the pool has a bias, the outcome may very well too. Hence, these widescale failures.


Bottom line: While having value, prediction markets are subject to error and bad outcomes. Some of this is due to their relative thinness of the markets; some is due to the inherent biases of the participants, and their failure to parallel the population at large.

They should be considered with the recognition of their fallibility.


Odds Are, They’ll Know ’08 Winner
NYT, February 14, 2007

Wednesday, February 14, 2007 | 07:23 AM | Permalink | Comments (12) | TrackBack (0)
de.li.cious add to de.li.cious | digg digg this! | technorati add to technorati | email email this post



TrackBack URL for this entry:

Listed below are links to weblogs that reference Misunderstanding Prediction Market Failures:


Under the spell of Surowiecki's book, without having read past the first few pages.

Posted by: Jim M | Feb 14, 2007 8:34:06 AM

The comments to this entry are closed.

Recent Posts

December 2008
Sun Mon Tue Wed Thu Fri Sat
  1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31      


Complete Archives List



Category Cloud

On the Nightstand

On the Nightstand

 Subscribe in a reader

Get The Big Picture!
Enter your email address:

Read our privacy policy

Essays & Effluvia

The Apprenticed Investor

Apprenticed Investor

About Me

About Me
email me

Favorite Posts

Tools and Feeds

AddThis Social Bookmark Button

Add to Google Reader or Homepage

Subscribe to The Big Picture

Powered by FeedBurner

Add to Technorati Favorites


My Wishlist

Worth Perusing

Worth Perusing

mp3s Spinning

MP3s Spinning

My Photo



Odds & Ends

Site by Moxie Design Studios™