Experimental Bias Class 12. Toot Suite Survey Do you drive? ___ YES ___ NO Do you listen to music while driving?___ YES ___ NO Please provide brief singing.

Experimental Bias

Class 12

Toot Suite Survey

Do you drive? ___ YES ___ NO

Do you listen to music while driving? ___ YES ___ NO

Please provide brief singing sample: _______________

Do you ever sing while driving? ___YES ___ NO

How do questions make you feel? About self? experimenter?

How much did these feelings affect your answer?

TO OBSERVE IS TO CHANGE

Heisenberg Uncertainty Principle:

The more precisely one property is measured, the less precisely the other can be controlled, determined, or known.

In other words, the very act of trying to measure a thing introduces biases that distort the thing being measured.

In other words, we risk changing things by studying them!!!

The Totalitarian Experimenter

We want subjects to:

Be blank slatesLook, feel, think only as directedAccept instructions without questionRespond only to given info, no other info

Modes of Subject Defections

Willful or negligent undermining

Try to sabotage experiment

Try to figure out the experiment

Get bored

Get distracted

Over-cooperativeness

Self-presentation motives

Desire to help experimenter

Positive response to neutral conditions

Self Image Motives Lead to:

Positivity bias: I am better than others Normative bias: What I do is normal, sane, correct. Distinctiveness bias: But I am also unique and exceptional "Will this experiment make me appear stupid, weird, yet boringly common?"

Evidence of Subject Self-Presentation Motives

* Will do meaningless tasks for long time, willingly. * Will do noxious tasks: poisonous snakes, hand in acid * Will do ethically unsound acts: shock others to death

Placebo Effects * 30%-40% of drug effect is due to placebo * Placebos reduce tumors, diabetic blood sugar, pain

* Neurological effect among depressants

Relevance to research? Subjects can be very suggestible. Can unconsciously comply with deduced purpose of research.

Cause: Cues from the experimenter, the experimental situation, confederates, other subjects, or other sources that inadvertently direct subjects' responses.

Demand Characteristics

Examples? * Clever Hans* Facilitated Communication

Deception Experiments

Good cover storyPiloting and debriefingIV and DV in separate settingsKeep subjects unaware of being in an experiment

Behavioral Measures

Advantages over verbal, self report measuresNon-obvious measures

Bias-reducing Designs

Post-test onlyFactorial designs

Bogus Pipeline

Techniques for Counteracting Demand

BOGUS PIPELINE

Sources of Experimenter Bias

* Men friendlier than women, all experimenters nicer to women * Experimenter reaction to Subject attributes * Subject reaction to Experimenter attributes * Experimenter's research goals, knowledge of hyp.

-- "smart" vs. "dumb" rats-- "smart" vs. "dumb" worms-- Neutral faces

Source of Bias

Experimenter is unintended stimulus to subject

Subject is unintended stimulus to experimenter

Experimenter’s goals, knowledge of hypothesis

Is Bias Random or Systematic?

Random

Random

Systematic

Experimenter Bias

Techniques for Minimizing Experimenter Bias

1. Naïve experimenters (NOT!)

2. Experimenters blind to subject condition, not to hypothesis

3. Two experimenters, each half-blind

4. Automated instructions

5. Run all conditions at once

6. Experimenter blind to one variable in factorial design

7. Separate experimenter for each subject

8. Experimenter blind to incoming data

9. Combine techniques, above.

Confederate Race

Confederate Temperament 

  Friendly Unfriendly

Black Black/Friendly Black/Unfriendly

White White/Friendly White/Unfriendly

Experimenter Blind to One Factor in Factorial Design

Expt’r had to know confederate race

Could be blind to confederate temperament

Time Perspective Types

Future Oriented Decisions about how to act are influenced by considerations of future consequences. Long-term concerns are “real” and compelling to future oriented people. They “see” further out into the future.

Present Oriented Focus more on concrete reality of the immediate present. More likely to get sucked into the moment, and think less about long-term consequences. The see less far into the future, but may see the present in greater detail.

0

10

20

30

40

50

60

70

First Study Middle Studies Final Study

Nu

mb

er

of

Da

ys

Futures, Female

Futures, Male

Presents, Female

Presents, Male

Time Perspective and Subject Sign-upsHarber, Zimbardo, & Boyd, 2003

First Third of Term

2.57

0.79

1.29

0.33

Middle Third of Term 1.03 0.78 1.08 1.10 

Female

Future

(n = 102)

Male

Future

(n = 75)

Female

Present

(n = 86)

Male

Present

(n = 112)

Final Third of Term 0.39 1.45 0.75 2.05 

Odds ratios: Likelihood of sampling time orientation/ gender groupings during the 1st, 2nd, or 3rd of term.

1. Studies done early in term will have disproportionate numbers of future oriented women, studies done toward the end of the term will be disproportionately represented by present-oriented men.

2. Longitudinal studies may systematically lose more data (due to tardiness, etc.) from present oriented subjects than from future oriented subjects.

Time Perspective Differences—So What?

OK, so what's the big deal?

Time orientation might interact with research question, and bias results in unexpected ways.

Study originally done at one time in the term may not replicate if second study done later in term.

How Do Cows Get Milk? A Critique of Eve’s H2O,M Study

What is the research question? Is it clearly stated?

What is the hypothesis?

What is conceptual IV Fluid deprivation

What is empirical realization of IV? Milk cow dry, prevent drinking

What is dependent variable? Amount of milk produced at retest

Results Milk IS produced despite absence of of drinking!!!

Hypothesis confirmed?

Evidence of bias (demand?)

Alternative explanations?

Cows extract milk from air

YesNo

??????