Non-Experimental Methods I Class 22
Non-Experimental Methods I
Jan 21, 2016
Non-Experimental Methods I. Class 22. Why Alternatives to Controlled Experiment? Advantage of controlled experiment? Advantages to not using controlled experiment?. Controlled Experiment: Great, but Not Always the Best Ethical Concerns Practical Concerns Methodological Concerns - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Non-Experimental Methods I
Class 22
Why Alternatives to Controlled Experiment?
Advantage of controlled experiment?
Advantages to not using controlled experiment?
Controlled Experiment: Great, but Not Always the Best
Ethical Concerns Practical Concerns Methodological Concerns
Generalizability
Internal analyses Not all interesting questions are causal
Types of non-experimental studies
a. Correlational
b. Pseudo-experimental
c. Quasi-experimental
Correlational Studies
FIASCO STUDY: Shared meals will reduce major fights Correlation of meals to major fights = -. 37 Interpretations:
1. ???????????
2. ???????????
3. ???????????
Correlational Studies
FIASCO STUDY: Shared meals will reduce major fights Correlation of meals to major fights = -. 37 Interpretations:
Dinners -----------------> Fewer fights
Fewer fights ----------> More dinners
more dinners
fewer fights
Less overtime at work
??? Panel Design
Do shared meals reduce the frequency of major family conflicts?
Time 1
Time 2(3 years)
Dinners
Fights
Dinners
Fights
r = +.24
r = +.40
r = - .31
r = - .37
A
B
Cross-Lagged Panel Design
Do shared meals reduce the frequency of major family conflicts?
Time 1
Time 2(3 years)
Dinners
Fights
Dinners
Fights
r = +.24
r = +.40
r = - .31
r = - .37
A
B
Cross-Lagged Panel Design
Do shared meals reduce the frequency of major family conflicts?
Time 1
Time 2(3 years)
Dinners
Fights
Dinners
Fights
r = +.24
r = +.40
r = - .31
r = - .37
r = -.15
r = -.39
(Pssst: Ask Kent about Walter Michel study)
????? Designs
1. One-shot (post-test only) case study
2. One-group pretest-posttest design
3. Static-group comparison design
Pseudo-experimental Designs
1. One-shot (post-test only) case study
2. One-group pretest-posttest design
3. Static-group comparison design
One Shot Case Study
Design: Observe population of interest Examples:
Client case studies (client case studies, famous people)Group case studies (focus groups)Population case studies (anthropological studies)
FIASCO: Study families at dinner time,
observe dinner together --> less conflicts. Problems?
One Shot Case Study
Design: Observe population of interest Examples:
Client case studies (client case studies, famous people)Group case studies (focus groups)Population case studies (anthropological studies)
FIASCO: Study families at dinner time,
observe dinner together --> less conflicts. Problems:
No comparison: with other families displaying same behaviorwith same family under different conditions.
Excellent Case Studies
De Tocqueville in America
Freud’s studies of Anna O, and the Wolfman
Cantril’s study of NJ after Mars Invasion
Wertheimer on train Gestalt psychology
One Group Pretest-Posttest Design
Observe before and after treatment Pretest and posttest provide basis for comparison Example: * Monitor families before and after dinner
* Record number/intensity of conflicts pre vs. post dinner.
* Finding: Less tension after dinner Problems with this design???
Threats to Internal Validity
1. History effects: ????? 2. ????: People change over time 3. Testing: ???? 4. ?????: Experimenter, apparatus change. 5. Mortality: ????
6. Selection: ????
Threats to Internal Validity
1. History effects: Things co-occur with TX 2. Maturation: People change over time 3. Testing: People react to being observed 4. Instrumentation: Experimenter, apparatus change. 5. Mortality: People drop out before study ends.
6. Selection: People volunteering for study may differ from those who did not volunteer.
Static Group Comparison Design
General idea: Take advantage of naturally occurring comparison group.
Examples:
School intervention: Look at students not in new program
FIASCO: Look at families nearby, that don't have dinner together.
Advantage: Provides some basis for comparison. Problem? Selection.
Maybe non-dinner family found that not sharing dinner helps prevent conflicts.
???-ExperimentsFeatures
* Control administration of treatment (TX)* Control the collection of DV measures* BUT, no random assignment to TX or control
WITHIN GROUP QUASI-DESIGNS
Simple time series designs
Equivalent time-samples design
BETWEEN GROUP QUASI-DESIGN
Non-equivalent Control-group design
Multiple-group time-series designs
Quasi-ExperimentsFeatures
* Control administration of treatment (TX)* Control the collection of DV measures* BUT, no random assignment to TX or control
WITHIN GROUP QUASI-DESIGNS
Simple time series designs
Equivalent time-samples design
BETWEEN GROUP QUASI-DESIGN
Non-equivalent Control-group design
Multiple-group time-series designs
The Arith-Matisse Project
Problem: Many students have math anxiety. Also,
they don't know how to engage in visual skills that might improve their math achievement.
Solution: Provide students 1 hour of drawing / fine
arts instruction to help develop visual skills, andalso to relax them before math class. Art class must immediately precede math class.
Prediction: Art class preceding math class will lead
to better math performance.
No Monday 60%
No Tuesday 57%
No Wednedsay 63%
Yes Thursday 84%
No Friday 74%
No Monday 64%
No Tuesday 59%
Art Hour Offered Day Score on Daily Math Quiz
Time Series Experiment: Simple Time Series Design
Effect of art hour on math class performance
Why It’s Necessary to Monitor Outcome Beyond Treatment Administration
No Monday 60%
No Tuesday 57%
No Wednedsay 63%
Yes Thursday 84%
Art Hour Offered Day Score on Daily Math Quiz
No Friday 81%
No Monday 87%
No Tuesday 90%
Question: Why not stop monitoring after treatment (e.g., Thurs)?
Reason: If stopped on Thurs., would miss continuation of change.
Problems with Simple Time Series
Hawthorne Effect: Being observed can, in itself, be motivating or otherwise affect change.
History Effect Events co-occurring with treatment could
be influencing outcomes.
Problems with Simple Time Series
Hawthorne Effect: ????
History Effect: ????
No Monday 60%
No Tuesday 57%
No Wednesday 63%
Yes Thursday 84%
No Friday 74%
No Monday 64%
No Tuesday 59%
Yes Wednesday 85%
No Thursday 62%
No Friday 65%
Art Hour Offered Day Score on Daily Math Quiz
Time Series Experiment: Equivalent Time Samples Design
Effect of art hour on math class performance
No Monday 60%
No Tuesday 57%
No Wednesday 63%
Yes Thursday 84%
No Friday 74%
No Monday 64%
No Tuesday 59%
Yes Wednesday 85%
No Thursday 62%
No Friday 65%
Art Hour Offered Day Score on Daily Math Quiz
Problems With Equivalent Time Samples Design?
No Monday 60%
No Tuesday 57%
No Wednesday 63%
Yes Thursday 84%
No Friday 74%
No Monday 64%
No Tuesday 59%
Yes Wednesday 85%
No Thursday 62%
No Friday 65%
Art Hour Offered Day Score on Daily Math Quiz
Problems With Equivalent Time Samples Design?
No comparison condition. Example: Students do well on math in order to retain art class, but not b/c art helps with math
Psychologists Sans Data
William James Solomon Asch
Father of Amer. psychology PerceptionThe Self Conformity, group dynamicsEmotions Group Dynamics Gordon AllportPerception Founds Personality Theory
Prejudice research
Sigmund Freud
Unconscious Processes HumanistsPsychosomatics Victor FranklSocial, Personality, Developmental Rolo May
Abraham MaslowKurt Lewin Carl Rogers
Group Processes Social Systems The Self
“Invasion From ???”: The ‘Big Bad Boo’ of 1938
Orson Well broadcast of invasion from Mars—as if a breaking news story, Oct. 30, 1938
Approximately 30 million people affected, to varying degrees
Historical context
a. ??? relatively new—and trustedb. World on brink of ???
“Invasion From Mars”: The ‘Big Bad Boo’ of 1938
Orson Wells broadcast of invasion from Mars—as if a breaking news story, Oct. 30, 1938
Approximately 30 million people affected, to varying degrees
Historical context
a. Radio relatively new—and trustedb. World on brink of world war
Rationale: Rare event; oppty to study mass behavior
Event structures research:
* How people respond during panic/stress* Power of real-life event, outside the lab
Method:
* Multi-method approach—provides converging evidence:
-- Interviews (n = 135)-- Unobtrusive measures (e.g., phone company)-- Commercial survey research (Gallop, AIPO)-- Newspapers
Cantril Study (1940)
Observations from Invasion from Mars Study Recovery from panic
* Prolonged anxiety – sometimes“I was sick in bed for 3 days after the broadcast”
* “Nevermind” – sometimes“We listened to the rest of the play and went out dancing”
Role of religion and faith
* “I held a crucifix in my hand and prayed while looking out ofmy window for falling meteors”
Role of purposeful action in coping
* “...having to concentrate on the driving held me together somewhat”
Fear of unknown
* I didn’t (know) what I was fleeing from, and that made me all the more afraid.”
Background fears based on reality influence immediate, ambiguous threat.
* “Don’t you know New Jersey is destroyed by the Germans...” Final note: Open-ended interviews provide invaluable historical record.
Observations from Invasion from Mars Study (continued)
Related Documents
