Black Swans Or Black Albinos? Why Bell Curves Still Rule

With financial markets suddenly crashing, the timeliest book of 2008 has turned out to be published in 2007: The Black Swan: The Impact of the Highly Improbable, an exuberant attack on the very concept of forecasting the future by a Wall Street trader-turned-philosopher named Nassim Nicholas Taleb.

Taleb's book is proving highly influential. His popular but confusing buzz phrase, "black swan", is seemingly already ensconced permanently in our jargon.

For example, Joe Nocera's article in Saturday's New York Times Magazine, "Risk Mismanagement," [January 2, 2009] does a fine job of applying Taleb's insights to how Wall Street catastrophically underestimated the riskiness of securitized junk mortgages during the Housing Bubble. The financial industry assumed that the performance of financial instruments is distributed according to a bell curve, with few outliers. They assumed that mortgage defaults happen randomly due to miscellaneous human tragedies (illness, death, divorce, and job loss), so bundling a lot of mortgages together would virtually eliminate the risk of the entire package.

But instead, of course, the new populations of California, Nevada, Arizona, and Florida proved unable to earn enough money to afford their mortgages.

Taleb argues that bell curve-based forecasts fail frequently because outliers—"black swans"—have dramatic impact. If the bell curve actually applied to wealth, for instance, Bill Gates couldn't exist.

Or consider history. Taleb, who is from the affluent Christian upper class of Lebanon (his grandfather was deputy prime minister), saw his homeland devastated when a 15-year-long civil war broke out while he was a teenager. For anyone who remembers the post World War II era, when Lebanon was universally viewed as the Switzerland of the Middle East, that was a black swan indeed.

Similarly, who, say, could have predicted in 1924 that a failed artist then moldering in jail would go on to such a cataclysmic career that even 85 years later in 2009 the world remained structured around the geopolitical and intellectual arrangements forged in his defeat? A Hitler couldn't exist in a world wholly framed by the bell curve.

Taleb's conclusion: human beings think too much in terms of normal probability distributions—of means and standard deviations—and not enough about the Hitlers and Bill Gates of the world.

Oh yeah? Obviously, Taleb has spent much of his life hanging around with a rather unrepresentative slice of humanity—Wall Street quant jocks.

Do we really not think about Hitler enough? For instance, there were at least five (5) Hollywood movies released in the last couple of months to qualify for the Academy Awards that were set in Nazi-controlled Europe (Valkyrie, The Reader, Defiance, Good, and The Boy in the Striped Pajamas.)

Moreover, in recent years we've heard various Middle Eastern politicians compared to Hitler, such as Hassan Nasrallah of Hezbollah in Taleb's Lebanon, Mahmoud Ahmadinejad of Iran, and, most disastrously, Saddam Hussein.

Of course, as it turned out, Saddam didn't particularly want to conquer the world. And even if he did, he didn't have the human capital do it: he was ruling Iraqis (average IQ of 87), not Germans.

Unfortunately, as its correct but limited lessons get overgeneralized in the elite mind, The Black Swan is likely to wind up doing similar damage to intellectual life over the next few years. After all, there's always a big market for attacks on bell curves. And Taleb's ideas will inevitably get misapplied outside of Wall Street to public policy, where they will be used to buttress the conventional wisdom.

For example, in the new bestseller Outliers: The Story of Success, which I reviewed in last month, the always-trendy Malcolm Gladwell attempts to apply Talebian-style anti-bell curve thinking to the perennial question of why ethnic groups perform so differently on average. (Here's Gladwell's 2002 article on Taleb's money management business, "Blowing Up.")

Taleb isn't to blame for Gladwell getting befuddled. But his inability to fully understand why he's right about the things he's right about doesn't help matters.

Still, if you can put up with Taleb's egomania and free-floating hostility, which I find amusing but you might not, The Black Swan is almost as easy to read as Gladwell's book, and far more valuable.

Taleb's term for rare but important events, "black swans," (which, in a display of his sometimes-excessive erudition, he borrowed from John Stuart Mill paraphrasing David Hume) is disastrously mischosen. Black swans aren't rare at all, much less particularly important. In Australia, all wild swans are black, and there are as many as a half million of them. Even in the rest of the world, they have become a bit of a cliché of ornamental water gardens—there are a pair of black swans paddling about in a fake stream outside a dowdy restaurant a mile from my house.

Instead of calling his book "The Black Swan," Taleb, had he wanted to stay with his the white-black theme, could have used as a title a more self-explanatory term such as "The Black Albino". White-skinned blacks are indeed rare, yet rather more common than you might assume.

Of course, nobody (except maybe the VDARE Foundation) would have published it with that title. Still, an all-around better title for his book would have been Outliers—which would have had the auxiliary benefit of preventing Gladwell from misusing it.

Needless to say, Taleb is right than making an accurate forecast is difficult, especially about the future.

But here are many phenomena in life where the glass is both half-full and half-empty. The secret is knowing when it's one or the other.

Taleb is smart enough to admit on about 5 percent of his pages that making predictions using bell curves is a half-full glass. But he's cocksure enough in the other 95 percent in denouncing its half-emptiness that careless readers likely will take away the lesson that it's completely empty.

In fact, there's a general pattern here that Taleb misses. The things we are most likely to argue the longest over, such as whether bell curves are useful or not or over nature vs. nurture, are those where the evidence is most abundant for both sides.

For example, Taleb draws a helpful distinction between probability distributions in "Mediocristan", where datapoints tend to be distributed roughly according to bell curves (he lists 11 examples including height, weight, car accidents, and IQ) and in "Extremistan" where a small number of rare items greatly influence the average (he lists 21 examples, such as wealth, income, and book sales per author).

Taleb then asserts: "The Extremistan list is much longer than the [Mediocristan list]." Of course, he made up the lists

Is he right about which list is actually longer? Personally, I have no idea i.e. I have no more idea than Taleb does.

All I know for sure is that I have less self-confidence than Taleb. His experience as a Wall Streeter skews his view toward Extremistan. My experience as a marketing researcher showed me the pervasiveness of Mediocristan.

Consider the recent election. You will all recall how the countless election polls, with their bell curve-based "margins of error", were humiliated on Election Day when the ultimate Black Swan was elected President—John McCain!

Oh, wait! That only happened in Talebstan. In the world in which you and I live, the pre-election polls proved to be extremely accurate.

Another only thing I know for sure: events in Extremistan are much more interesting than events in Mediocristan. Enron is much more interesting than Procter & Gamble. WWI and WWII are far more exciting than America and Canada not having a war.

We've all seen the "Dewey Defeats Truman" picture a million times. But the reason we've seen it is because it's unusual. Contra Taleb, human beings love rarities.

 But that doesn't mean rarities are, as Taleb claims, more important.

We are most interested in things that are hard to predict, not necessarily those things that are most important. Science is in the business of making predictions, but the better it gets at predicting anything, the more boring those predictions become for us.

Indeed, much of what interests us has been carefully contrived by experts to seize our attention. It's very hard to predict the winner of the Super Bowl because the NFL carefully rigs the system to make each team's chances as equal as possible.

In contrast, if something becomes a sure thing, we get bored.

Thus, from 1934-1976, there was a prestigious summer exhibition football game at Soldier Field in Chicago pitting the previous year's College All-Stars versus the reigning NFL champion team. Initially, as the famous rookies held their own against the champs, winning nine of the first 30 games, it was wildly popular. It drew over 100,000 people three times in the 1940s. After 1963, however, as professional teams got more professional, the college boys never won again. Attendance drooped as the outcome became more predictable. In 1976, a thunderstorm stopped the game in the third quarter with the Pittsburgh Steelers up 24-0. Nobody saw any point in resuming the series.

By the same token, rare events and people are interesting because they are rare.

For example, every few years Hollywood makes a movie Based on a True Story about some Black Swan teacher who goes into a gritty slum school and unleashes the brilliant scholars within the minority students. But they don't make movies about the countless White Swan teachers who try equally hard and don't succeed. Guess why not.

In fact, it's actually easy to make accurate predictions about important phenomena. For example, I predict that public schools in gritty Compton CA will have lower average test scores than public schools in lovely San Marino, CA in 2009. And in 2010, and in 2011 and…

Well? Anybody care to bet against me?

I can make a 100 such predictions about significant topics with a 99 percent accuracy rate. But nobody would even bother reading all the way through the list. It would just get so boring and depressing, as I kept reiterating things that we all know but don't want to talk about.

Let me, therefore, make some fun predictions.

  • I foresee that Oklahoma will win the college football national title game 35-31 when Florida quarterback Tim Tebow, having heroically driven his team 91 yards, throws a heartbreaking interception in the end zone on the last play.

  • In the week before the inauguration, the Obamas will begin secretly consulting about resolving problems in their marriage with Oprah Winfrey.

How do I know these things?

Well, obviously, I don't. I just made them up. Indeed, to be frank, I chose Florida to lose because I couldn't remember the name of the Oklahoma quarterback.

But aren't these predictions more interesting than me yammering on about how school test scores won't change?

How can we tell whether we are in Mediocristan or Extremistan?

Bell curves work best when individuals are of fairly equal importance. That's definitely not true on Wall Street. But it is somewhat true in voting, public life, and, for many items, shopping.

A quarter of a century ago, I was running a test market in Eau Claire, Wisconsin for a Procter & Gamble deodorant brand.  I was comparing two samples of 1250 households who were being shown different TV commercials over their cable television systems. I noticed a screwy pattern in our results. I eventually figured out that one household was buying 30 to 40 sticks of deodorant per month, affecting the market share for its entire sample. That's Taleb's definition of a Black Swan.

But did P&G pay us a bonus for discovering the Black Swan of the Deodorant Market—the Bill Gates of Perspiration Prevention?

Did P&G suddenly wake up to the fallacy of the bell curve and stop doing marketing research?

Nah. We all just agreed to drop that &%#$* outlier from the sample.

Surely, though, the world is dominated by outliers as Taleb says—not by boring stuff like deodorant?

Well, what about cars, which are in the news a lot these days, what with GM and Chrysler demanding a bailout? Is the world market for cars driven by a small number of Jay Lenos who buy hundreds of cars?


How about mortgages? Now we're playing on Taleb's own financial turf. Do houses tend more toward Bill Gates's Extremistan or Joe Average's Mediocristan?

Well, let's look at Bill Gates's house. He famously built one of the largest houses in America, 66,000 square feet. It's almost 40 times larger than my house. And yet his net worth is, I would imagine, considerably more than 40 times my own.

(I can't make a precise calculation because I haven't been masochistic enough to open any letters from my 401k funds since last summer.)

And that turns out to be hugely important in understanding the mortgage meltdown, which has come straight out of Mediocristan. It wasn't caused by a few giant defaults by Bill Gates, or even by Ed McMahon. It was caused by a huge number of defaults among people of average or below average wealth, typically working class folks in the second quartile up from the bottom.

And that followed, unsurprisingly, a decade and a half of government policy pushing easier credit for home buyers, especially for minorities. Both the Clinton and Bush Administrations wanted to boost the home ownership rate, which had been stuck at about 64 percent since the 1970s. Bush explicitly demanded the raising of black and Hispanic homeownership rates from about 50 percent to the white rate of 75 percent by such obviously risky ploys as zero down payments.

In other words, the marginal homeowners would primarily come from the financially marginal portions of the population.

The subprime fiasco wasn't an impossible-to-predict Black Swan. It was the result of average humans following straightforward incentives to the best of their capabilities.

And they turned out to be distributed on bell curves.

Despite the Black Swan flap, bell curves still rule. We ignore them at our peril.

In fact, we just did.

[Steve Sailer (email him) is movie critic for The American Conservative. His website features his daily blog. His new book, AMERICA'S HALF-BLOOD PRINCE: BARACK OBAMA'S "STORY OF RACE AND INHERITANCE", is available here.]