Trap/Cognitive Bias/No. 0553
Law of Small Numbers Bias
The law of small numbers bias is the belief that a small sample will closely reflect the population it comes from. Amos Tversky and Daniel Kahneman described it in psychology in 1971 as a form of representativeness bias that leads people to overtrust unstable results.
Also called Belief in the Law of Small Numbers
- Evidence
- Well established
- Read
- 6 min
- Links
- 11 connections
01You've seen this when…
- in life
You try a new cafe twice. Both visits involve a long wait, so you expect every visit to be slow.
- at work
A new salesperson closes four of their first five deals. The manager starts teaching the rest of the team their approach.
- out in the world
A school with 18 graduates reports that all of them found jobs. Officials rank it above a school with 900 graduates and a 94% employment rate.
02The idea
A few observations can feel like a miniature version of the whole. Five customers seem to reveal the market. Two difficult conversations seem to reveal someone’s character. A short winning streak seems to reveal a successful strategy.
The law of small numbers bias gives these little samples more stability than they deserve. Every observation carries a large share of the result, so a few unusual cases can move the average or percentage sharply.
The name plays on the law of large numbers: under suitable conditions, averages settle toward the population average as samples grow. The bias extends that expectation to samples too small to provide much steadiness.
This overlaps with hasty generalization. Hasty generalization describes the reasoning error of drawing a broad conclusion from insufficient evidence. The law of small numbers bias describes an expectation that helps produce it: even a handful of cases should look like the population.
The practical question is how much weight the sample can carry. Small samples contain evidence, with uncertainty that often remains substantial.
03Why it happens
- A sample is expected to resemble its source. The representativeness heuristic encourages people to judge a sample by how much it looks like the population. If half of births are boys, a small maternity ward feels as though it should also produce roughly half boys each day. Its daily proportions actually swing more than those of a large hospital.
- Percentages hide the number of observations. An 80% success rate sounds substantial. Four successes out of five expose how much rests on each case. A polished chart can make both figures look equally settled.
- A short sequence invites an explanation. A run of wins suggests skill; a run of complaints suggests a broken process. Once an explanation fits, the ease of telling it can crowd out uncertainty about how often the pattern would appear by chance.
- Randomness is expected to balance quickly. If small batches should resemble the whole, a run of heads can make tails seem due. This expectation can feed the gambler’s fallacy. Independent coin tosses retain the same chances throughout the run.
04A worked example
Consider a hypothetical checkout test. Visitors are randomly assigned to two pages, with an uneven traffic split. Page A produces six purchases from ten visitors. Page B produces 45 purchases from 100 visitors. The team plans to send all traffic to A.
What it looks like A clear improvement. A converts 60% of visitors, compared with B’s 45%. The dashboard gives A a green badge and B a red one.
What’s actually going on A’s estimate rests on ten people. One additional purchase among those ten would move its rate by ten percentage points; the same change among B’s 100 visitors would move B’s rate by one point. A may be better, but these counts leave considerable uncertainty about the size and direction of the difference. The badge makes a fragile estimate look settled.
What would have helped Showing purchase counts alongside rates, displaying uncertainty around the difference, and agreeing in advance on a testing plan. The team needs enough observations to detect an improvement that matters to the business. A statistical power calculation can help plan that sample. The cost of waiting and the risk of choosing poorly also belong in the decision.
05How to spot it
06What to do instead
- Put counts beside percentages. Write six purchases out of ten visitors alongside 60%. Keep both visible in reports and conversations.
- Check how much one case can move the result. Recalculate after adding or removing one observation. A large swing signals that the estimate needs cautious handling. This is a quick sensitivity check, not a formal measure of uncertainty.
- Show the range of plausible values. A suitable confidence interval helps communicate sampling uncertainty. For a comparison, examine uncertainty around the difference itself; overlapping intervals alone don’t settle whether the groups differ.
- Gather evidence where the decision needs it. Plan sample size around the effect that would change the choice and the cost of an error. There is no universal minimum that makes a sample trustworthy.
- Look for another sample. Replication tests whether an apparent pattern survives fresh observations. An extreme first result often becomes less extreme, a pattern called regression to the mean.
- Match the claim to the evidence. A promising pilot can justify a larger test. An urgent decision may justify acting provisionally, with a review date and a reversible commitment.
07When it isn’t overconfidence
A small sample can support a decision when the signal is strong or the stakes favor early action. One confirmed failure in a safety-critical component may justify stopping operations while investigators learn more. That decision depends on the consequences of another failure.
Sample size also addresses only part of the problem. Ten thousand responses from a self-selected audience can still misrepresent the population through selection bias. More observations reduce random sampling variation under appropriate conditions; systematic distortions can remain.
People’s sensitivity to sample size varies with the task and its presentation. Research finds better intuitions in some formats, especially when the question makes variation across samples easier to understand.
08Roots
In their 1971 paper, Amos Tversky and Daniel Kahneman asked research psychologists to advise a colleague about study size and judge the dependability of research results. The unsettling detail was the audience: these were people whose work already involved statistics. Their answers showed excessive confidence in what small studies could establish and how reliably findings would recur.
The paper’s title, Belief in the law of small numbers, captured the gap between statistical training and intuitive judgment. The familiar law of large numbers had acquired an imaginary smaller cousin. Researchers appeared to expect modest samples to reproduce population patterns with surprising fidelity.
Their later work brought the problem into an everyday setting: two hospitals with different numbers of births. Which would have more days with an unusually high proportion of boys? The smaller hospital has more variable daily proportions, but many respondents treated the hospitals alike. The example helped carry the idea beyond research design into the broader study of judgment under uncertainty.
09How solid is this?
Experiments and surveys document overconfidence in small samples and neglect of sample-size differences. The strength of the effect depends on the task and presentation; some formats reveal substantially better intuitions about sampling variability.
10Connections
- Often confused withHasty Generalization, Selection Bias, Law of Large Numbers, Regression to the Mean
- Countered byReplication, Confidence Interval, Statistical Power
- Can lead to Gambler’s Fallacy, Overconfidence Effect
- Can follow from Representativeness Heuristic
- See also Hot Hand Fallacy
+ 1 more in the list
11Origin and sources
Amos Tversky and Daniel Kahneman described the bias in their 1971 paper, Belief in the law of small numbers, using judgments by research psychologists.
- [1]Tversky, A., & Kahneman, D. (1971). Belief in the law of small numbers. Psychological Bulletin, 76(2), 105–110.
- [2]Kahneman, D., & Tversky, A. (1972). Subjective probability: A judgment of representativeness. Cognitive Psychology, 3(3), 430–454.
- [3]Sedlmeier, P., & Gigerenzer, G. (1997). Intuitions about sample size: The empirical law of large numbers. Journal of Behavioral Decision Making, 10(1), 33–51.
Suggest an edit· Updated 2026-10-02