Trap/Logical Fallacy/No. 0140

Cherry Picking

Cherry picking is the selection of favorable evidence while omitting relevant contrary evidence, creating a misleading view of the whole record. Also called the fallacy of incomplete evidence in informal logic, it weakens an argument without proving that its conclusion is false.

Also called Fallacy of Incomplete Evidence

a trap: easy to walk into

Evidence: Well established
Read: 6 min
Links: 9 connections
Useful when: Arguing and debating · Evaluating a claim · Persuading and explaining · Reading data and statistics

01You've seen this when…

in life
You decide a new sleep supplement works. You remember the two mornings you wake refreshed and forget the five when you feel no different.
at work
A launch report highlights the one region where sales beat expectations. Results from the other six regions sit in an appendix nobody presents.
out in the world
A campaign flyer lists three months when burglaries fell. The full year’s figures show an increase.

02The idea

Every customer testimonial in the deck can accurately describe that customer’s experience, and the deck can still mislead. Selecting favorable facts can distort the picture of the whole record.

Cherry picking means choosing favorable observations while leaving out relevant contrary evidence. The selection can involve people, studies, dates, locations, outcomes, or comparison points. A company might report its strongest product line as if it represents the business. A commentator might start a graph at an unusually high or low point.

The key question is whether the omitted evidence would materially change the impression. A summary can leave things out, provided its selection gives a fair account of the whole.

This differs from selection bias, which can make the data unrepresentative before anyone builds an argument. Cherry picking concerns which evidence gets used or presented, though it can create selection bias in what an audience sees.

Finding cherry picking weakens the argument while leaving the truth of its conclusion open. The claim might survive a fair review.

03Why it happens

A conclusion becomes something to defend. Once you’ve backed a policy, product, or person, supporting evidence feels useful and contrary evidence feels like an obstacle. Motivated reasoning can turn an investigation into a search for ammunition.
Supporting cases come to mind more easily. Confirmation bias affects what you notice, seek, and remember. You can sincerely believe you’ve considered the record while overlooking the parts that resist your story.
Short formats reward clean stories. A pitch, headline, or social post has little room for exceptions. Compression becomes distortion when the missing material would change the audience’s judgment.
The selector has more information than the audience. Someone who sees every customer review can choose three glowing ones. Readers see the three, not the search that produced them.

Cherry picking can be deliberate, careless, or unconscious. You don’t need to establish someone’s motive to identify a misleading selection.

04A worked example

Erick Turner and colleagues reviewed 74 studies of 12 antidepressant drugs in 2008. They compared published trials with the trial record held by the U.S. Food and Drug Administration.

What it looks like In the published literature, 94% of the trials appeared positive. A reader relying on journal articles would encounter an overwhelmingly favorable record.

What’s actually going on The FDA classified only 51% of the full set as positive. Trials with negative or questionable results were often unpublished, or published in a way that conveyed a positive outcome. The favorable slice gave an overly positive picture of the complete evidence. This was cherry picking at the level of the available research record, through publication bias and selective reporting. A reader could encounter that skew even while considering every published study.

What would have helped A review that searched beyond published journal articles would have benefited from access to the full trial record and advance registration of studies and outcomes. Turner’s team could expose the gap because it had a second source against which to check the visible record.

The finding showed that selective reporting made antidepressant benefits look more consistent and larger than the fuller evidence supported. That result leaves open the possibility that antidepressants work.

05How to spot it

The boundaries look convenient. A chart begins at a peak, ends before a reversal, or covers only the strongest locations.
Examples replace the overall result. You get successful customers, graduates, or investments without learning how many cases failed.
Only favorable outcomes appear. A program reports attendance but not completion, or revenue but not costs.
The selection rule stays hidden. Nobody explains why these studies, dates, or people were included.
The evidence shifts with the objection. Each challenge produces another favorable anecdote rather than a fuller accounting.

These are warning signs, not proof. A narrow window or unusual case can be justified; the justification should be available.

06What to do instead

Define the evidence before inspecting the results. Specify the population, time window, outcomes, and exclusion rules. For research, preregistration makes it harder to quietly change those choices after seeing what looks good.
Ask for the denominator. Three successful businesses mean something different out of four attempts than out of four hundred. Request a count of all attempts that includes failures and missing cases.
Look for the strongest contrary evidence. Use the consider-the-opposite strategy: identify what would most seriously challenge your conclusion, then actively search for it.
Test alternative boundaries. Extend the date range, include other regions, or use another reasonable comparison point. A conclusion that disappears under these checks deserves less confidence.
Make the selection process visible. State what you searched, what you included, and why you excluded anything. A systematic review formalizes this approach for research evidence.
Match the audit to the stakes. A quick evidence check can be enough for choosing lunch. A medical claim, investment pitch, or major policy decision warrants a broader review that places handpicked examples within the larger body of relevant evidence.

07When it isn’t cherry picking

Selection is unavoidable. A useful summary leaves things out. A study can legitimately exclude broken measurements, irrelevant populations, or evidence too weak to answer its question.

The distinction is whether the selection has a defensible rule that would apply regardless of which conclusion it favors. Such a rule can justify excluding unreliable data even when the exclusion changes the result. Treating data as unreliable only after they contradict you is a warning sign.

A fair account weights claims by the relevance and quality of their evidence; counting examples alone gives an incomplete picture. Ten strong studies can outweigh one poor study.

The Texas sharpshooter fallacy is a close relative: it treats a pattern noticed after searching as if it had been specified beforehand. Cherry picking is broader; pattern discovery is only one possible part of it.

08Roots

Francis Bacon wanted inquiry to examine both supporting and contrary evidence for beliefs people already held. In Novum Organum, published in 1620, he retold a story about a skeptic being shown tablets in a temple commemorating people who had survived shipwrecks. The missing evidence was the people who drowned. Their deaths kept them from adding their tablets.

The story answered a question that still matters: what can favorable observations establish when the contrary cases are absent? Bacon described how people notice evidence that supports an accepted opinion while neglecting or explaining away evidence against it. This early account of the underlying error predates the first known use of the modern name.

The cherry-picking metaphor gives that problem an everyday image: taking the appealing fruit while leaving the rest behind. There is no single widely accepted coiner of the fallacy’s name.

Research methods turned the warning into procedures. Trial registration, complete outcome reporting, and systematic searches make the missing evidence easier to detect. Modern reporting guidelines such as PRISMA ask reviewers to explain how studies were found and selected. Researchers still select evidence; the aim is to make those choices inspectable.

09How solid is this?

Selective reporting and publication are well documented, including comparisons between published trials and regulatory records. The logical problem is clear. A suspicious selection alone leaves the selector’s intent and the truth of the underlying conclusion unresolved.

10Connections

Often confused with Texas Sharpshooter Fallacy
Countered bySystematic Review, Consider-the-Opposite Strategy, Preregistration
Can lead toHasty Generalization
Part of Motivated Reasoning, Selection Bias, Confirmation Bias
See also Publication Bias

11Origin and sources

An informal-logic label with no single established coiner. Francis Bacon described the underlying selective-evidence problem in Novum Organum (1620); research methodology later developed procedures for detecting and limiting it.

Suggest an edit· Updated 2026-10-02