Skepticism at work - Logical Fallacies. ASQ Buffalo

SkepticismAt WorkRecognizing

Logical FallaciesSteve Cena

2013

Basic Definitions

Cynicism: To doubt without evidence. To dismiss a claim out of hand.

Skepticism: To doubt based on the evidence or the lack thereof.

Faith: To believe without evidence or in spite of contrary evidence. “Faith is believing what you know ain’t so.”*

Belief: To accept based on the available evidence.

* Mark Twain

“I’m a skeptic not because I do not want

to believe but I want to know.”

Michael Shermer, The Believing Brain Prolog

Discussions of belief & skepticism are often centered on paranormal or

pseudoscientific topics

Business Claims

• We need a fresh set of eyes to work on this problem.

• We tried that before & it didn’t work.• The test is 99% accurate.• It’s just an outlier.• I can’t explain why but whenever we do it this

way it works.

Which Claims Should We Believe?

WE SHOULD ONLY BELIEVE THINGS FOR

WHICH THERE IS ADEQUATE EVIDENCE.

What is adequate evidence?

How do we evaluate the data?

DATA

JudgmentsReasoning

Judgments = OpinionsReasoning = Logic

Errors in Reasoning = Logical Fallacies

Claims are made on what seem to be good arguments but these arguments are based on incorrect logical principles. Thus, the claims are not valid.

Ad Hominem Attack

Ad hominem is Latin for "to the person”

• Not invented here

• Outside expert error

• Guilty by group association

False Dichotomy(The Excluded Middle)

• “You are either for us or against us.”

• "You are either part of the solution or part of the problem.”

Arbitrarily reducing the set of probabilities to only 2.

• "It's either the design or the process."

Response or Problem

Measurement Method Material

Machine Mother Nature Manpower

Response or Problem

This

That

Anecdotes Do Not Equal Evidence• Memory is fallible & malleable

– Memories are reconstructed not replayed

• Anecdotes rely on intuition & subjective interpretation

• “… we are apt to want our version of the truth, rather than the truth itself, to prevail.”*

• Anecdotex ≠ Data– The plural of anecdote is not data.

• Show me the data * Steven Pinker, How the Mind Works, pg. 289

Conjunction Fallacy

• 80% of the time delivery is late• 80% of the time the quantity is wrong• What is more likely?

– The next delivery will be late– The next delivery will be late and the wrong

quantity

The tendency to think 2 conclusions are more likely than either conclusion

alone.

Conjunction Fallacy

• 80% of the time delivery is late– P(late) = .80

• 80% of the time the quantity is wrong– P(quantity) = .80

• P(late) x P(quantity) = .80 x .80 = .64

Base Rate Fallacy

When considering the probability of something happening failing to consider

the probability that it can happen.

Base Rate Fallacy

Test 100 acceptable parts99 are correctly identified

1 incorrectly identified

Test 100 unacceptable parts99 are correctly identified

1 incorrectly identified

Base Rate Fallacy

99% accurate!99% of shipped product is acceptable!

99% of rejected product is unacceptable!

Only if half of the product made is unacceptable.

Base Rate Fallacy

Produce 10,000 units

95% are acceptable

10,000 x .95 = 9,500 acceptable units10,000 – 9,500 = 500 unacceptable

Base Rate Fallacy

9,500 acceptable units tested at 99% accuracy

9,500 x .99 = 9,405 accepted 95 incorrectly rejected

Base Rate Fallacy

500 unacceptable units tested at 99% accuracy

500 x .99 = 495 rejected 5 incorrectly accepted

Base Rate Fallacy

9405 correctly accepted + 5 incorrectly accepted 9410 total accepted units

What Shipped

9405/9410 = .9995Considering the BASE RATE

99.95% of shipped product is acceptableNOT 99% as claimed.

Base Rate Fallacy

495 correctly rejected + 95 incorrectly rejected 590 total rejected units

What is Rejected

495/590 = .839Considering the BASE RATE

83.9% of rejected product is unacceptableNOT 99% as claimed.

Post Hoc Ergo Propter Hoc

• Assumption: if A precedes B then A must cause B

• Coincidences happen• Correlation ≠ Causation

– “New York City Has Most Millionaires In The Country” *

• By What Mechanism?

Latin for "after this, therefore because of this“

* http://www.huffingtonpost.com/2010/08/04/new-york-city-has-most-mi_n_670474.html#s122573&title=New_York_667200

Post Hoc Ergo Propter Hoc

XKCD

Regression to the mean

An extreme measurement will tend to be followed by a measurement that is closer to the mean.

Regression to the mean

Regression to the mean

Regression to the mean

Regression to the mean

Regression to the mean

Regression to the mean

Why do you believe?

• Is it because the belief is comfortable or convenient?

• Does the data reaffirm what you already "know" to be true?

• Are contrary data ignored or rationalized away?

• Did you objectively analyze the data and come to this conclusion?

What would change your mind?

• NOTHING!• Falsifiable claim

– “The Scientific method consists of the use of procedures designed to show not that our predictions and hypotheses are right, but that they might be wrong”*

• Rabbit in the Precambrian**

What is your rabbit?* Mistakes Were Made, Tarvis & Aronson**J.B.S. Haldane

What would you do if you’re wrong?

• Minimize – It’s not that bad• Mobilize – Moving target• Optimize – (POB) Pervasive Optimistic Bias• Rationalize – Justify• Invoke any logical fallacy

What would you do if you’re wrong?

Why do you believe?

What would change your mind?

A Short Story

MinimizationMobilization

Argument From Authority

Rationalization

RationalizationStraw Man Argument

Argument From IgnoranceOptimize (POB)