The Black Swan: The Impact of the Highly Improbable

This is my attempt to review the first edition of Nassim Nicholas Taleb’s The Black Swan: The Impact of the Highly Improbable.

I want to start with Taleb’s tone in the book: This is not trivial, because whether or not you can finish the book depends quite a bit on whether you can stomach his… idiosyncratic writing style. I’m accustomed to reading his style of writing having read his earlier book, Fooled by Randomness: The Hidden Role of Chance in Markets and in Life. But a reader new to Taleb should be warned: Taleb is rude. He loathes most economists, mathematicians, and statisticians – and he does not mince his words. Whether or not his dislike for them and their ideas is founded is beyond my ability to evaluate. He may have a point, and a very good one at that, but his no-holds-barred attacks on many people (again, I don’t know if they deserve it. Maybe, but his rants are often overboard and unprofessional*) and his very poorly veiled attempts to portray himself as a humble (definitely not) and deep thinker (definitely so) can come across as annoying.

On to the real stuff, I’d like to make a quick summary of his ideas; a very risky business because I don’t want to misrepresent any ideas, but I shall attempt to do so because the ideas are valuable.

The Black Swan Problem

The title is The Black Swan because of what he calls the Black Swan Problem, otherwise known as Hume’s problem of induction (after David Hume, the 18th century philosopher): Can we be certain that all swans are white simply based on the fact that all the swans we have seen are white? We may be tempted to make that inference if all our lives we’ve only seen white swans, but all it takes is one black swan to prove us wrong.

This asymmetry underlies his argument that much of the forecasting we see in financial markets is useless. If we accept that knowledge by induction is flawed, then statistical techniques like regression analysis, which involves extrapolating data points to forecast the future, has limited use. Having seen so many white swans, we naturally assume that the swans we see in the future will also be white. But when we bank our entire fortune on it – as many banks did in the recent subprime mortgage crisis – we may, one day, be in for a surprise.

Capitalizing on the asymmetry

When I first heard the idea of the black swan problem, my first thought was, yes Taleb is right, but if we constantly fear the occurrence of the black swan, aren’t we letting go of what can reasonably work with, i.e. swans are most likely to be white? In other words, should we just do nothing?

No. Taleb has a way around it: limit our downside risk as far as possible, and maximize our exposure to unforeseen benefits, or serendipity. A good example of exposing ourselves to positive consequences is Pascal’s wager. Blaise Pascal said that in deciding whether or not to believe in God, one should choose the latter because he has everything to gain if God exists and nothing to lose if God does not.

In the financial world, derivatives serve the purpose of exposing the buyer to positive consequences (or negative consequences if you happen to be the seller). The seller of a call option gives the buyer the right to buy a share at the exercise price, and the seller of a put option gives the buyer the right to sell a share back to him at the exercise price.

So suppose I have a stock worth $20 now. Thinking that stock prices are likely to fall since the economy doesn’t look too good, I want to make a quick buck by selling an American call option with an exercise price of $25. I do this for $2. This gives the buyer the right but not the obligation to buy my stock for $25 at any time before expiration, say in 6 months’ time. 6 months later, if my stock price indeed falls, then I have nothing to fear: I’ve made an easy profit on the sale of the call option. But if the price of the stock rises beyond $25, then the buyer can exercise the option and buy the stock from me for only $25.  What is the most he can lose? $2. But what he can earn is virtually unlimited – limited only by the final price of the stock. By buying out-of-money call and put options, he can only “bleed” slowly to death, but cannot “blow up” because of a sudden unexpected event such as a financial crisis. During unexpected events favorable to him, his profits are immense.

The Gaussian Distribution – Great Intellectual Fraud (GIF)?

Taleb makes one final claim in this book: that the ubiquitous bell curve is an intellectual fraud when employed in areas such as finance. Here, he makes the distinction between Mediocristan and Extremistan. Mediocristan is the land in which variables are “mediocre”, i.e. they fluctuate mildly around a certain average, and outliers are rare – if they exist, they do not significantly change the aggregate. Such variables include physical quantities such as height, weight, and IQ, which follow a Gaussian distribution. Extremistan on the other hand, is the land where variables can be extreme, and outliers can exert a massive influence. Examples include wealth (Bill Gates can greatly distort the distribution of wealth), use of words in the vocabulary (see Zipf’s law), and they do not follow the Gaussian.

So why Great Intellectual Fraud? Harsh words, and very unfair, but Taleb makes it clear within the book that the Gaussian distribution is rightly used in places where we are looking for a Yes/No answer. For example, statistical testing in psychology uses the bell curve appropriately. But when it comes to financial markets, creating sophisticated models based on the Gaussian is akin to living in your own world because the models simply don’t fit the facts.

Instead, empirical findings by the late Benoit Mandelbrot (who developed the Mandelbrot Set used in fractal geometry and Chaos Theory) showed that stock market returns exhibit memory effects, an observation which goes against one fundamental assumption of the Gaussian distribution – that of independence between trials. With that, Mandelbrot came up with the idea that stock returns exhibit fractal or wild randomness, instead of mild and controllable randomness.

Whether or not stock returns follow a Gaussian distribution makes a world of a difference, because sophisticated models taught in finance are all largely based on the Gaussian, which does not adequately measure the probability of extreme events. In fact, the Gaussian tells us that the probability that we observe a deviation from the mean decreases exponentially as this deviation increases. For example, the probability of observing a four-sigma event, or one that is four standard deviations from the mean, is 1 in 32,000. The corresponding probabilities for five- and six-sigma events are 1 in 3.5 million and 1 in a billion respectively: an exponential increase.

Indeed, if we believe that stock returns are normally distributed and consequently apply models such as the capital asset pricing model (CAPM), we would greatly underestimate the true risk of extreme events. On the other hand, fractal randomness attributes more accurate probabilities to extreme events, an issue discussed in greater depth in Mandelbrot’s book, The (Mis)Behavior of Markets: A Fractal View of Risk, Ruin, and Reward.

Conclusion

I’ve only presented the main ideas that jumped out at me when I was reading the book. There are some other valuable ones pertaining to how we process information and make decisions, but again, these are better discussed in books on cognitive psychology and behavioral economics. This is a book that will change the way you view the world, and in particular, the role of randomness in our lives. If you can stomach his arrogant tone (and that is a big If), this is a great read.

*Taleb has on a few occasions traded public attacks with Myron Scholes (of the Black-Scholes formula, which Taleb condemns). After Long-Term Capital Management blew up, he commented that Scholes would be better off doing sudoku in a retirement home instead of giving advice on risk management.

Advertisements

Resolving the unfair subway

From my earlier post:

Marvin gets off work at random times between 3 and 5 P.M. His mother lives uptown, his girlfriend downtown. He takes the first subway that comes in either direction and eats dinner with the one he is first delivered to. His mother complains that he never comes to see her, but he says she has a 50-50 chance. He has had dinner with her twice in the last 20 working days. Explain.

Downtown trains run past Marvin’s stop at, say, 3:00, 3:10, 3:20,…, etc., and uptown trains at 3:01, 3:11, 3:21,…. To go uptown, Marvin must arrive in the 1 minute interval between a downtown and an uptown train.

The Unfair Subway

More probability from Frederick Mosteller:

Marvin gets off work at random times between 3 and 5 P.M. His mother lives uptown, his girlfriend downtown. He takes the first subway that comes in either direction and eats dinner with the one he is first delivered to. His mother complains that he never comes to see her, but he says she has a 50-50 chance. He has had dinner with her twice in the last 20 working days. Explain.

The Monty Hall Paradox

For some reason, I’ve suddenly become crazy over randomness and probability. I’m only halfway through The Drunkard’s Walk but this will definitely be one of my favorite books.

Here’s the Monty Hall problem:

Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, “Do you want to pick door No. 2?” Is it to your advantage to switch your choice?

Marilyn vos Savant, famous for being listed for years in the Guinness World Records Hall of Fame as the person with the world’s highest recorded IQ (228), published this question in her column in Parade Magazine in September 1990. The question does appear quite silly. After all, when one door is shown to be a loser (assuming that we prefer a car to a goat), the probability of either remaining choice – neither which is more likely than the other – becomes 1/2. So why would switching make any difference? But Marilyn said in her column that it is better to switch.

This resulted in plenty of controversy, attracting some 10,000 mails including almost a thousand from PhDs. Many readers seemed to feel let down. How could a person they trusted so much on such a broad range of issues be confused by such a simple question?

When I was first asked the question, I too thought there would be no way that switching could make a difference. But the truth is, switching does increase your chances of winning. Here’s the solution from Wikipedia:

Singapore’s casino has officially opened – and never has the study of randomness been more timely.

Randomly yours,

Tim