Situations / Reading data and statistics
Reading data and statistics
Interpreting numbers, charts, studies or A/B test results.
Traps you might be in
- No. 0068 Availability HeuristicAfter watching a clip of a shark attack, you cancel a beach swim you were looking forward to.
- No. 0077 Base Rate FallacyYour screening result comes back positive, and you assume you’re almost certain to have the disease.
- No. 0140 Cherry PickingThe sales deck celebrates three happy customers, while the cancellation spreadsheet stays closed on the presenter’s laptop.
- No. 0191 Confirmation BiasYou decide a job offer is right, then spend the evening reading only the reviews that make it sound good.
- No. 0196 Conjunction FallacyIn the forecast meeting, a detailed story about a recession and a market crash feels likelier than a recession alone.
- No. 0217 Correlation-Causation FallacyYour best salespeople use the new dashboard most, so you make everyone use it.
- No. 0295 Dunning-Kruger EffectYou finish your first practice exam feeling ready, then the answer key reveals mistakes you didn’t even know to check.
- No. 0403 Framing EffectYou feel safer choosing surgery with a 90% survival rate than surgery with a 10% death rate.
- No. 0413 Gambler’s FallacyAfter five heads in a row, you put more money on tails.
- No. 0476 Hot Hand FallacyAfter three winning roulette bets, you double your next stake because tonight you feel unstoppable.
- No. 0635 Motivated ReasoningYou want the apartment, so you read the glowing reviews once and reread the complaints until they sound unreasonable.
- No. 0663 Negativity BiasYou reread the one harsh comment on your presentation while the ten thank-you messages sit unopened.
- No. 0732 Peak-End RuleYou call the whole weekend wonderful because of one spectacular dinner and the easy ride home.
- No. 0846 Representativeness HeuristicA stranger talks enthusiastically about chess, and you decide he’s a mathematician before learning what he does for a living.
- No. 0895 Selection BiasYou read glowing reviews of a difficult course, then notice that every reviewer made it to the final lesson.
- No. 1008 Survivorship BiasYou scroll through traders’ success stories and start planning how to quit your job.
- No. 1027 Texas Sharpshooter FallacyAfter trying twenty cuts of a sales dashboard, you find one winning customer group and call it the campaign’s intended audience.
Tools to try
- No. 0001 A/B TestingYour team prefers the new signup page, but you keep the old one live for half the visitors to see what happens.
- No. 0087 Bayesian UpdatingThe fraud alert looks damning, until you learn how often it flags perfectly ordinary purchases.
- No. 0125 CalibrationYou open your prediction log and find that only half the things you called 80% certain actually happened.
- No. 0199 Consider-the-Opposite StrategyBefore sending a furious reply, you jot down three reasons your coworker’s message might not be an insult.
- No. 0686 Occam’s RazorThree apps stop working at once, and you start reinstalling each one before checking the shared connection.
Patterns at play
- No. 0018 Adverse SelectionYou turn down another low offer for your well-maintained car because buyers price it like one hiding an expensive fault.
- No. 0437 Goodhart’s LawWhen a number becomes the goal, people find ways to move the number that don’t move the goal.
- No. 0458 Hawthorne EffectWith a researcher watching you try the new app, you read every instruction you would normally skip.
- No. 0721 Pareto PrincipleYour bank statement has forty small purchases, but rent and two bills swallow most of the money.
- No. 0936 Simpson’s ParadoxYour team’s conversion rate rises on both phones and computers, but the overall rate falls.
Concepts to know
- No. 0084 Bayes’ TheoremYour screening result comes back positive, and you start making plans as if the diagnosis is settled.
- No. 0193 ConfoundingYou buy the supplement used by the healthiest people at your gym, without asking what else they do.
- No. 0292 Dual-Process TheoryYou reread an email you nearly sent and realize your first reply would have made the argument worse.
- No. 0307 Effect SizeThe tutoring program boasts a statistically significant gain; you look closer and find an average improvement of one test point.
- No. 0343 Expected ValueThe repair shop offers a $180 protection plan, and you weigh that price against repairs you might never need.
- No. 0376 Fat-Tailed DistributionsYour insurance budget survives hundreds of ordinary claims, then one warehouse fire costs more than all of them put together.
- No. 0551 Law of Large NumbersYour first ten coin flips produce eight heads, and you start wondering whether the coin is fair.
- No. 0556 Leading vs. Lagging IndicatorYour support dashboard stays green all week, but unanswered tickets are piling up before customers start canceling.
- No. 0589 Map vs. TerritoryYour phone says the path continues, but you’re standing in front of a locked gate.
- No. 0599 Mean vs. MedianYour boss announces that average pay is $100,000, and nobody at your lunch table earns anything close to it.
- No. 0830 Regression to the MeanAfter your lowest sales month all year, you take a course and celebrate when the next month’s numbers bounce back.