Read on Twitter
Survivorship Bias 101
History is written by the victors. But if we exclusively focus on these successes, we allow survivorship bias to distort our understanding of the world.
But what is "survivorship bias" and how does it work?
Here's Survivorship Bias 101!
History is written by the victors. But if we exclusively focus on these successes, we allow survivorship bias to distort our understanding of the world.
But what is "survivorship bias" and how does it work?
Here's Survivorship Bias 101!
1/ First, a few definitions.
Survivorship bias is the logical error of concentrating on survivors (successes) and ignoring casualties (failures).
When we do so, we miss the true "base rates" of survival (the actual probability of success) and arrive at flawed conclusions.
Survivorship bias is the logical error of concentrating on survivors (successes) and ignoring casualties (failures).
When we do so, we miss the true "base rates" of survival (the actual probability of success) and arrive at flawed conclusions.
2/ When we completely ignore failures, we lose our ability to correctly identify the differences between successes and failures.
Put simply, exclusively focusing on successes may actually inhibit our ability to identify (and replicate) the actions that led to such success.
Put simply, exclusively focusing on successes may actually inhibit our ability to identify (and replicate) the actions that led to such success.
3/ Cicero wrote on the topic over 2,000 years ago.
An atheist named Diagoras is shown portraits of people who prayed and were saved from death at sea as proof of God's existence.
Diagoras replies, "I see those who were saved, but where are those painted who were shipwrecked?"
An atheist named Diagoras is shown portraits of people who prayed and were saved from death at sea as proof of God's existence.
Diagoras replies, "I see those who were saved, but where are those painted who were shipwrecked?"
4/ Cicero cuts right to the point - we cannot conclude that prayer led to being saved from the sea if we ignore those who prayed and then drowned.
Survivorship bias can lead to a deeply-flawed understanding of cause-and-effect relationships.
Let's look at some examples.
Survivorship bias can lead to a deeply-flawed understanding of cause-and-effect relationships.
Let's look at some examples.
5/ One of the famous examples of survivorship bias comes from World War II.
The U.S. wanted to add reinforcement armor to specific areas of its planes.
Analysts plotted the bullet holes and damage on returning bombers, deciding the tail, body, and wings needed reinforcement.
The U.S. wanted to add reinforcement armor to specific areas of its planes.
Analysts plotted the bullet holes and damage on returning bombers, deciding the tail, body, and wings needed reinforcement.
6/ But a young statistician named Abraham Wald noted that this would be a tragic mistake.
By only plotting data on the planes that returned, they were systematically missing data on a critical, informative subset - the planes that were damaged and unable to return.
By only plotting data on the planes that returned, they were systematically missing data on a critical, informative subset - the planes that were damaged and unable to return.
7/ The "seen" planes had sustained damage that was survivable.
The "unseen" planes had sustained damage that was not.
Wald concluded that armor should be added to the unharmed regions of the survivors.
Where the survivors were unharmed is where the planes were most vulnerable.
The "unseen" planes had sustained damage that was not.
Wald concluded that armor should be added to the unharmed regions of the survivors.
Where the survivors were unharmed is where the planes were most vulnerable.
8/ Based on his observation, the military reinforced the engine and other vulnerable parts, significantly improving the safety of the crews during combat.
Wald had identified the survivorship bias and avoided its wrath.
Where else do we see survivorship bias arise?
Wald had identified the survivorship bias and avoided its wrath.
Where else do we see survivorship bias arise?
9/ In business?
We all love business success stories. We love to read about companies and people who have achieved incredible success and fame.
We scan their backgrounds in desperate search of the actions or features that led to that success.
But these conclusions are flawed.
We all love business success stories. We love to read about companies and people who have achieved incredible success and fame.
We scan their backgrounds in desperate search of the actions or features that led to that success.
But these conclusions are flawed.
10/ For every entrepreneur who took out a second mortgage in a last-ditch attempt and succeeded, there may be 10 who did the same and went bankrupt.
For every company that pursued a moonshot idea and is now worth billions, there may be 10 who did the same and collapsed.
For every company that pursued a moonshot idea and is now worth billions, there may be 10 who did the same and collapsed.
11/ In markets?
Analysts may try to calculate the performance of funds or groups of stocks using only the survivors at the end of the period.
By ignoring closed funds or stocks removed from an index, we bias the data and overstate returns.
Flawed data
Flawed decisions
Analysts may try to calculate the performance of funds or groups of stocks using only the survivors at the end of the period.
By ignoring closed funds or stocks removed from an index, we bias the data and overstate returns.
Flawed data
Flawed decisions
12/ The point? By ignoring the companies or people who failed, we miss out on valuable data points from those cases.
To actively fight this, we must study the features of both successes and failures.
The "seen" and the "unseen" are equally important in guiding our decisions.
To actively fight this, we must study the features of both successes and failures.
The "seen" and the "unseen" are equally important in guiding our decisions.
13/ As @nntaleb wrote in his classic, Fooled by Randomness, "Heroes are heroes because they are heroic in behavior, not because they won or lost."
So the next time you read a story of the hero who won, be sure to reflect on all of the heroes who lost, whose stories are not told.
So the next time you read a story of the hero who won, be sure to reflect on all of the heroes who lost, whose stories are not told.
14/ So that was Survivorship Bias 101. As someone who shares many success stories on Twitter, I thought it was an important topic to address.
Special thanks to @austin_rief of @MorningBrew, whose interesting tweet on new founders sparked this thread. https://twitter.com/austin_rief/status/1335294451674193920?s=20
Special thanks to @austin_rief of @MorningBrew, whose interesting tweet on new founders sparked this thread. https://twitter.com/austin_rief/status/1335294451674193920?s=20
15/ For more on the topic of survivorship bias, I highly recommend the below resources.
Fooled by Randomness by @nntaleb: https://www.amazon.com/gp/product/0812975219/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0812975219&linkCode=as2&tag=sbloomadvisor-20&linkId=40e02f721736b9da17fd3eb4b9b16647
Great blog from @ShaneAParrish and @FarnamStreet: https://fs.blog/2019/12/survivorship-bias/
Fooled by Randomness by @nntaleb: https://www.amazon.com/gp/product/0812975219/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0812975219&linkCode=as2&tag=sbloomadvisor-20&linkId=40e02f721736b9da17fd3eb4b9b16647
Great blog from @ShaneAParrish and @FarnamStreet: https://fs.blog/2019/12/survivorship-bias/
16/ And for more educational threads on business, money, finance, and economics, check out my meta-thread below. Turn on post notifications so you never miss one! https://twitter.com/SahilBloom/status/1284583099775324161?s=20
