Money and Finance
The Quixotic Quest to Make Pundits Suck Less
Thanks to Sean for passing this along.
“The average expert’s forecasts were revealed to be only slightly more accurate than random guessing—or, to put more harshly, only a bit better than the proverbial dart-throwing chimpanzee. And the average expert performed slightly worse than a still more mindless competition: simple extrapolation algorithms that automatically predicted more of the same.” ~ Dr. Philip Tetlock
During the presidential election, statistician Nate Silver consistently predicted on his FiveThirtyEight blog (hosted by the New York Times) that the odds of Mitt Romney winning the presidency were slim. Leading conservative political experts like Karl Rove and Newt Gingrich called his claims ridiculous. Silver kept the faith in his poll numbers and models; the pundits trusted their “years of experience.”
In the end, Silver (whose background was the moneyball world of applying statistics to baseball) embarrassed the pundits by accurately predicting which states would be won by each candidate. But you wouldn’t know it by looking at the careers of the experts. News channels never stopped welcoming Newt Gingrich and his ilk as experts, while Karl Rove continues to accept healthy paychecks as an expert consultant for political campaigns around the world. The consequences of their nationally televised embarrassment were exactly nil.
It is this lack of accountability when it comes to making predictions that fascinates Philip Tetlock, a professor at the University of Pennsylvania. His interest in expert judgment began after the end of the Cold War. Despite both conservatives and liberals failing to predict how the Soviet Union would end, he saw both sides fit the emerging developments to their original ideas - the same ideas that had just failed.
Related book: Expert Political Judgment: How Good Is It? How Can We Know?
Related link: HOW TO WIN AT FORECASTING: A Conversation with Philip Tetlock
-
Henry Ford Quote
“None of our men are ‘experts.’ We have most unfortunately found it necessary to get rid of a man as soon as he thinks himself an expert because no one ever considers himself expert if he really knows his job . . . Thinking always ahead, thinking...
-
Nate Silver On The Launch Of Espn’s New Fivethirtyeight, Burritos, And Being A Fox
Link to article: Nate Silver on the Launch of ESPN’s New FiveThirtyEight, Burritos, and Being a FoxStats superstar Nate Silver hates the term “data-driven.” He also hates the work of just about every popular columnist at all of the major newspapers,...
-
Garrett Hardin Quote
“The greatest folly is to accept expert statements uncritically. At the very least, we should always seek another opinion. Moreover, to the extent that time allows, we may become a little bit expert ourselves; but we don't have time enough to go...
-
How To Win At Forecasting: A Conversation With Philip Tetlock
There's a question that I've been asking myself for nearly three decades now and trying to get a research handle on, and that is why is the quality of public debate so low and why is it that the quality often seems to deteriorate the more important...
-
Don’t Blink! The Hazards Of Confidence - By Daniel Kahneman
We often interact with professionals who exercise their judgment with evident confidence, sometimes priding themselves on the power of their intuition. In a world rife with illusions of validity and skill, can we trust them? How do we distinguish the...
Money and Finance