Chap 11

Power Point Slides by Ronald J. Shope in collaboration with John W. Creswell

Chapter 11

Experimental Designs

Educational Research 2e: Creswell

Key Ideas

• Definition and uses for experimental designs• Procedures used in experimental research• Internal and external validity threats in

experimental research• Types of experimental designs• Steps in conducting experimental research• Evaluating experimental research

Educational Research 2e: Creswell

What is experimental research?

• In an experiment, you test an idea (or practice or procedure) to determine whether it influences an outcome or dependent variable.

• Determine activities with which to “experiment” • Assign individuals to experience it (and have

some individuals experience something different)

• Determine whether those who experienced the activities (or practice or procedure) performed better on some outcome than those who did not experience it.

Educational Research 2e: Creswell

When do you use experimental procedures?

• To establish cause and effect– Between independent and dependent

variables– Control for all variables that might influence

the outcome

• Used when testing theories

• Used when comparing two or more groups

Educational Research 2e: Creswell

How did experiments develop?

• Psychological experiments (late 19th early 20th century)

• Procedures for comparing groups (McCall 1925)• Statistics for comparing groups (e.g. Chi-Square)

Identification of types of experimental designs (Campbell and Stanley 1963)

• Types of basic designs and threats (Cook and Campbell 1979)

• Complicated experiments with many variables (Since 1980)

Educational Research 2e: Creswell

Key Characteristics of Experimental Designs

• Random assignment

• Control over extraneous variables

• Manipulation of the treatment conditions

• Outcome Measures

• Group Comparisons

• Threats to validity

Educational Research 2e: Creswell

Random assignment

Randomly assign individuals at random conditions or to groups

• Random assignment equates groups and distributes variability between or among groups and conditions

• Different from Random Selection which is selection of participants at random

Educational Research 2e: Creswell

Control over extraneous variables

• Extraneous Variables: influences in participant selection, procedures, statistics, or the design likely to affect the outcome and provide an alternative explanation results than what was expected.

• Random assignment helps to control for extraneous variables

• Done before the experiment begins

Educational Research 2e: Creswell

Control over extraneous variables

Other control procedures– pretest/posttest– covariates– matching participants– selecting homogenous samples– using blocking variables

Educational Research 2e: Creswell

Pre-Test and Post-Tests

Time 1 Time 2

Pre-Test Post-Test

Intervention

Educational Research 2e: Creswell

Controlling for Covariates

DependentVariable

IndependentVariable

No Covariates

Covariate Introduced

Covariate:Parents Who

Smoke

VarianceRemovedVariance

DependentVariable:Rates of Smoking

IndependentVariable: Typeof Instruction

Educational Research 2e: Creswell

Matching Process Based on Gender

ExperimentalGroup

ControlGroup

JohnJimJamesJoshJacksonJaneJohannaJulieJeanJeb

Educational Research 2e: Creswell

Manipulation of the treatment conditions

• Identify a treatment variable

• Identify the conditions or levels of the treatment variable

• Manipulate the treatment conditions

Educational Research 2e: Creswell

The Experimental Manipulation of a Treatment Group

Independent Variables1. Age (can’t manipulate)

2. Gender (can’t manipulate)

3. Types of Instruction (can manipulate)

a. Lecture (control)

b. Lecture + Hazard Instruction (Comparison)

c. Lecture + Hazard Instruction + slides of damaged lungs (experiment)

Dependent Variable

Frequency of

Smoking

Educational Research 2e: Creswell

Outcomes

• Criterion or effect variable

• Outcome variable

• Measured on a continuous scale

Educational Research 2e: Creswell

Group comparisons in an experiment

+

* p<.05

Phase 1: Relationship PictureError Correction Treatment Spelling Accuracy

Phase 2: Timeline Picture Test 1 Test 2 Test 36 Weeks 6 Weeks 6 Weeks6 Weeks 6 Weeks 6 Weeks

Class A: Regular Spelling Practice (Control)Class B: Reduced word list (Comparison)

Phase 3: Statistical ComparisonsClass A Class B Class C F-value

Test 1

Test 2

Test 3

10.3(3.6)

10.7 (3.3)

11.1(3.3)

10.8(4.3)

10.6(3.8)

10.3(3.6)

9.9(3.9)

13.9(4.2)

13.1(3.8)

0.27

4.90*

3.31*

Error Correction Treatment 6 Weeks 6 Weeks 6 Weeks

Class C: Error Correction (Experimental)

Educational Research 2e: Creswell

Threats to internal validity

• History

• Maturation

• Regression

• Selection

• Mortality

• Interactions with selection

• Diffusion of treatments• Compensatory

equalization• Compensation rivalry• Resentful

demoralization• Testing• Instrumentation

Educational Research 2e: Creswell

Threats to external validity

• Interaction of selection and treatment

• Interaction of setting and treatment

• Interaction of history and treatment

Educational Research 2e: Creswell

Types of experiments: Between groups

• True Experiments– Pre- and posttest– Posttest Only

• Quasi Experiments– Pre– Posttest – Factorial Designs

Educational Research 2e: Creswell

Means and Main Effects of Eight groups in Factorial Design

Depression

Mean rateof

smoking

LowMean rate

ofsmoking

Mean rateof

smoking

Mean rateof

smoking

Mean rateof

smoking

Mean rateof

smoking

Health lecture

Type of Instruction

Standard lecture

Main Effectsof Type ofInstruction

Main Effects ofDepression

Medium High

Educational Research 2e: Creswell

Graphs Showing Main and Interaction Effects

High

LowLow Medium HighHigh

LowLow Medium High

High

LowLow Medium High

(a) No Interaction Effects(Parallel)

Extent of Smoking

Extent of Smoking

Extent of Smoking

(b) Interaction Effects(Crossed)

(c) Interaction Effects(Not Parallel)

Standard lecture

Health lecture

Standard lecture

Health lecture

Standard lecture

Health lecture

Educational Research 2e: Creswell

Types of Experiments: Within-group or individual

• Time series experiments– interrupted– uninterrupted

• Repeated measures experiments

• Single-subject experiments– A/B design– Multiple baseline design– Alternating treatments

Educational Research 2e: Creswell

Steps in Conducting Experimental Research

• Decide if an experimental design fits research problem

• Form cause/effect hypotheses• Select experimental unit and identify study

participants• Select an experimental treatment and introduce it• Choose a type of experimental design• Conduct the experiment• Organize and analyze the data• Develop an experimental research report

Educational Research 2e: Creswell

Criteria for Evaluating Experimental Research

• Does the experiment have a powerful intervention?• Does it employ few treatment groups (e.g. only two)?• Will participant gain from the intervention(s)?• Is there a systematic way the researcher derived the

number of participants (cell size)?• Was there an adequate number of participants used in

the study?• Were valid, reliable, and sensitive measures or

observations used?• Did the study control for extraneous factors?• Did the researcher control for threats to internal

validity?