The Art of Skepticism in a Data-Driven World
0525509208
Carl T. Bergstrom and Jevin D. West
Notes
Full-on bullshit is intended to distract, confuse, or mislead – which means that the bullshitter needs to have a mental model of the effect that his actions have on an observer’s mind.
…it’s often better to mislead without lying outright. This is called paltering. If I deliberately lead you to draw the wrong conclusions by saying things that are technically not untrue, I am paltering.
“The amount of energy needed to refute bullshit is an order of magnitude bigger than (that needed) to produce it.”
Alberto Brandolini
- Bullshit takes less work to create than clean up
- Takes less intelligence to create than clean up
- Spreads faster than efforts to clean it up.
Harry Frankfurt described bullshit as what people create when they try to impress you or persuade you, without any concern for whether what they are saying is true or false, correct or incorrect…G.A, Cohen notes that a lot of bullshit – particularly of the academic variety – is meaningless and so cloaked in rhetoric and convoluted language that no one can even critique it. Thus for Cohen, bullshit is “unclarifiable unclarity.”…If you can negate a sentence and it’s meaning doesn’t change, it’s bullshit.
Most often, bullshit arises either because there are biases in the data that get fed into the black box, or because there are obvious problems with the results that come out. Occasionally the technical details of the black box matter, but in our experience such cases are uncommon.
Goodhart’s Law – When a measure becomes a target, it ceases to be a good measure.
Never assume malice or mendacity when incompetence is a sufficient explanation, and never assume incompetence when a reasonable mistake can explain things.
While aesthetics are important, data graphics should be about the data, not about eye-catching decoration. Graphs that violate this principle are called “ducks.”…What bothers us about ducks is that the attempt to be cute makes it harder for the reader to understand the underlying data.
Glass slippers take one type of data to shoehorn it into a visual form designed to display another. In doing so, they trade on the authority of good visualizations to appear authoritative themselves. They are to data visualization what mathiness is to mathematical equations.
Bullshit Tricks
- Question the source information.
- Who is telling me this?
- How does he or she know it?
- What is this person trying to sell me?
- Beware of unfair comparisons
- If it seems to good or too bad to be true…
- Think in orders of magnitude
- Well-crafted lies will be plausible, whereas a lot of bullshit will be ridiculous even on the surface.
- Often one needs to do a few simple mental calculations to check a numerical claim.
- Avoid confirmation bias
- Consider multiple hypotheses
—
Ducks
Glass Slippers
Goodhart’s Law
Prosecutor’s Fallacy
Fermi estimation