Trap/Statistical Bias/No. 1008
Survivorship Bias
Survivorship bias is a form of selection bias that arises when conclusions rely on cases that remain visible while excluding those that failed or disappeared. Abraham Wald’s aircraft studies illustrate how missing cases can distort findings about the original population.
Also called Survivor Bias
- Evidence
- Well established
- Read
- 6 min
- Links
- 11 connections
01You've seen this when…
- in life
Your watch list fills with people who make a living from day trading. People who lose their savings and return to regular jobs rarely make videos explaining their routines.
- at work
Your team asks current customers whether the software is easy to use. Ratings look excellent. Everyone who canceled during setup is missing from the survey.
- out in the world
A neighborhood tour passes houses that have stood for a century. The guide praises the quality of old construction; the houses that collapsed or were demolished aren’t on the route.
02The idea
The customers still paying, the buildings still standing, the businesses still trading: all have passed a filter. Something determined which cases remained available for you to inspect. If you draw conclusions about the original population from those cases alone, you risk survivorship bias.
Survival can mean staying alive. It can also mean completing a course, keeping a job, remaining in a database or continuing to post online. The important question is whether disappearing is connected to what you’re trying to learn.
Suppose every thriving restaurant you interview has an elaborate menu. Establishing whether elaborate menus help restaurants thrive requires evidence from failed restaurants too. Failed restaurants may have had them too, and may even have had them more often. Without those cases, you cannot distinguish a winning strategy from a common practice among winners and losers alike.
This is a form of selection bias. A survivor-only sample poses a problem beyond having too few observations: it can remain misleading even when it contains thousands. Adding more survivors leaves the missing failures unrepresented.
03Why it happens
- The available records start too late. A directory lists businesses still operating, leaving closed businesses out of view. A customer database may keep active accounts while archiving canceled ones. The filter has already acted before your analysis begins.
- Success leaves a bigger trail. Winners attract interviews, biographies and speaking invitations. People who abandon an effort often stop documenting it. The visible stories become easier to retrieve than the missing ones.
- Disappearance carries information. If struggling funds close or frustrated users quit, their absence is connected to performance or satisfaction. They are missing for a reason, with their chances of disappearing tied to those outcomes.
- A vivid recipe hides the denominator. A founder’s habits give you something concrete to copy. Counting everyone who tried those habits, including failures, takes more work. Without that count, you can miss the base rate of success.
This can overlap with publication bias, but the filters differ. Publication bias concerns which results get published. Survivorship bias concerns which cases remain observable after surviving or continuing.
04A worked example
During World War II, the Statistical Research Group at Columbia University works on problems for the U.S. military. One concerns aircraft vulnerability: what can damage on planes returning from combat reveal about the hits that bring planes down? Abraham Wald develops a mathematical treatment of the problem in 1943.
What it looks like Use a survey of returning planes to find where damage appears most often, then put additional protection there. The visible damage seems to identify the most urgent weaknesses.
What’s actually going on Every inspected aircraft has survived its damage. A region with many recorded hits may be a place where an aircraft can sustain damage and still return. A region with few recorded hits may be especially vulnerable: planes hit there might be missing from the survey altogether. The damage map represents only the aircraft that survived exposure to fire.
What would have helped Treat returning and nonreturning aircraft as parts of the same problem. Wald’s approach uses survivor damage to estimate vulnerability under explicit assumptions about how hits occur. Armoring every undamaged area would require justification under those assumptions. The lesson is to account for the selection process before interpreting the holes.
05How to spot it
06What to do instead
- Define the group before the outcome. Start with everyone who enrolled, opened a business or invested at a particular time. Follow that group forward rather than assembling a list of today’s survivors.
- Search specifically for departures. Ask for canceled accounts, closed funds, former employees and dropouts. Record why they left and what happened afterward where you can.
- Compare the proposed recipe across outcomes. Before copying a successful person’s practice, check whether unsuccessful people used it too. A trait shared by both groups cannot explain the difference by itself.
- Use the original denominator. Report completions out of all starters, not just respondents. Reference-class forecasting helps by examining outcomes across a relevant group of comparable attempts.
- Make the gap visible when you can’t fill it. State who is excluded. Test whether plausible outcomes for missing cases would change the conclusion. Don’t silently assume their results resemble those of survivors.
If you want to understand current customers, say so. Understanding everyone who tries the product requires evidence about both current customers and those who leave.
07When it isn’t a misleading sample
Studying survivors is legitimate when survivors are the population you actually care about. A staffing survey may ask what today’s employees need, without claiming to describe everyone ever hired. Trouble begins when you extend its conclusions to former employees, applicants or all workers.
A reversal is only one possible effect of missing cases. They may strengthen the original conclusion or barely change it. The bias comes from treating a selected sample as representative without checking the filter. You can identify it while the accuracy of the answer is still unknown.
Recovering the missing cases leaves questions about causation open. Successful businesses may start at different times, have different amounts of capital or operate in different locations. A complete dataset still needs attention to confounding. Survivorship bias is one of several obstacles to a sound conclusion.
08Roots
Abraham Wald left Vienna for the United States in 1938, after Nazi rule made life there dangerous for a Jewish mathematician. A few years later, he was working at Columbia’s wartime Statistical Research Group. The group turned military problems into questions mathematics could answer. For Wald, one such question came with a peculiar constraint: the aircraft most relevant to understanding fatal damage were unavailable for inspection.
His 1943 reports approached that absence as part of the evidence. The returning planes formed a filtered sample. Their return depended on the damage they had sustained. Wald worked out ways to estimate vulnerability from the visible hits while accounting for that filter. Later retellings compressed this technical work into a memorable aircraft-and-bullet-holes story, often dropping the assumptions that made the inference possible.
Wald’s aircraft work became a classic illustration of selection bias, a broader statistical problem already known at the time. The name ‘survivorship bias’ has a separate origin. Finance researchers later gave another concrete version: historical performance databases can omit funds that closed. Studies including Brown and colleagues’ 1992 work showed why analyzing the survivors alone could distort conclusions about investment performance. The missing aircraft and missing funds pose the same question: what happened to the cases we can no longer see?
09How solid is this?
The statistical problem is well established, with documented applications in investment performance and other selected datasets. The direction and size of the distortion depend on which cases disappear; survivor-only data don’t automatically exaggerate every result.
10Connections
- Often confused with Lindy Effect, Matthew Effect
- Countered byReference-Class Forecasting
- Part of Sampling Bias, Selection Bias, Missing Not at Random, Hasty Generalization
- Includes Publication Bias
- See also Conditional Probability, Base Rate Fallacy, Confounding
+ 1 more in the list
11Origin and sources
Part of the broader selection-bias literature, with no single agreed originator. Abraham Wald’s 1943 aircraft-vulnerability reports provide a classic early illustration.
- [1]Mangel, M., & Samaniego, F. J. (1984). Abraham Wald's Work on Aircraft Survivability. Journal of the American Statistical Association, 79(386), 259–267.
- [2]Brown, S. J., Goetzmann, W., Ibbotson, R. G., & Ross, S. A. (1992). Survivorship Bias in Performance Studies. The Review of Financial Studies, 5(4), 553–580.
- [3]Elton, E. J., Gruber, M. J., & Blake, C. R. (1996). Survivor Bias and Mutual Fund Performance. The Review of Financial Studies, 9(4), 1097–1120.
Suggest an edit· Updated 2026-10-02