Freakonomics did an hour on prediction. It’s well worth a listen. Some excerpts…

Philip Tetlock, a psychology professor at Penn and author of Expert Political Judgment (here’s some info on Tetlock’s latest forecasting project), on just how bad we are at predicting:

TETLOCK: That experts thought they knew more than they knew. That there was a systematic gap between subjective probabilities that experts were assigning to possible futures and the objective likelihoods of those futures materializing … With respect to how they did relative to, say, a baseline group of Berkeley undergraduates making predictions, they did somewhat better than that. How did they do relative to purely random guessing strategy? Well, they did a little bit better than that, but not as much as you might hope …

Christina Fang, a professor of management at NYU’s Stern business school, on people who make bold economic predictions. Turns out, those making bold predictions are less trustworthy than average:

FANG: In the Wall Street Journal survey, if you look at the extreme outcomes, either extremely bad outcomes or extremely good outcomes, you see that those people who correctly predicted either extremely good or extremely bad outcomes, they’re likely to have overall lower level of accuracy. In other words, they’re doing poorer in general. … Our research suggests that for someone who has successfully predicted those events, we are going to predict that they are not likely to repeat their success very often. In other words, their overall capability is likely to be not as impressive as their apparent success seems to be.

Fang and co-author Jerker Denrell used data from the Wall Street Journal’s Survey of Economic Forecasts to determine the accuracy of influential financial experts in “Predicting the Next Big Thing: Success as a Signal of Poor Judgement.”