Introduction to ANOVA July 27, 2006 Bryan T. Karazsia, M.A.

Introduction to ANOVA

July 27, 2006Bryan T. Karazsia, M.A.

ANOVA: Overview Hand-in HW #5 Limitations of the t - test What is ANOVA and why must we

learn it? Sums of Squares Mean Squares The F Statistic

The limited t While t –test is very common and

useful, it’s not without its limitations Can only test differences between 2 groups

Class year? Ethnicity? Can only examine effects of 1 IV on 1 DV

Gender X Social Support (High, medium, low)? Depression?

With t – test, would have to either “collapse” categories…or just not run the analyses

Analysis of Variance (ANOVA)

Can examine data that the t –test cannot

Many varieties of ANOVA One-way Factorial Repeated Measures Mixed Model

Analysis of Variance (ANOVA)

Intro. to different types of ANOVAs One-Way

1 continuous DV 1 IV with 2 or more Levels (categorical

groups) E.g., if we have 4 class years, then we would have

4 Levels FYI, an ANOVA with 1 DV and 1 IV with 2 groups is

equivalent to the independent samples t

Analysis of Variance (ANOVA)

Intro. to different types of ANOVAs Factorial

1 Continuous DV 2+ IVs, each with 2+ groups

2 IVs = Two-Way Factorial ANOVA 3 IVs = Three-Way Factorial ANOVA

In a Factorial ANOVA, each level of each IV is paired with EVERY level of all other IVs

2 X 2 Contingency Table

Analysis of Variance (ANOVA)

Intro. to different types of ANOVAs Repeated Measures ANOVA

1 Continuous DV 1 Independent Variable consisting of 2+

“categorical” time points The DV is assessed at EACH time point

Analysis of Variance (ANOVA)

Intro. to different types of ANOVAs Mixed-Model ANOVA

1 Continuous DV 1+ IV with 2+ levels 1+ IV with 2+ time points

DV is assessed at EACH time point

Analysis of Variance (ANOVA)

We will cover One-Way ANOVAs, and time permitting (I hope it will), the Factorial ANOVA

I want to point you to the chart (decision tree) in the back cover of your text book

Simplified Underlying Model Assume:

Average 18 year-old human being weights approximately 138 pounds

Men, on average, weigh 12 more than the average human

Women, on average, weigh 10 pounds less than the average human

Simplified Underlying Model For any given human being, I can break

weight down into 3 components… 1- Average weight for all individuals

138 lbs 2- Average weight for each group

Men: 138 + 12 lbs Women: 138 – 12 lbs

3- The individual’s unique difference

Simplified Underlying Model Male Weight

138 + 12 + uniqueness Female Weight

138 – 10 + uniqueness

If you understand this process, then you understand the basic theory behind the ANOVA

ANOVA: Partitioning Variance

The main idea being an ANOVA is to divide or separate (partition) variance observed in the data into categories of what we CAN and CANNOT explain

ANOVA: Structural Model Mathematically, we partition the total

variance of our data using the structural form of the ANOVA model

In English: the score for any individual equals the sum of population mean (μ) plus mean of group (Tj) plus unique contribution

ijjij TX

ANOVA: Structural Model For our weight example…

μ = population weight = 138 T = group difference in weight = 12 or 10 lbs ε = “unique” contribution of individual’s

score

μ and T can be explained in our model ε cannot

ANOVA: uniqueness/error We often value our uniqueness…what

makes us who we are… In statistics, unique variance is BAD Since we cannot explain it, we call it “error”

Based on this model, ANOVA seeks to examine the relative proportion of explainable variance in our data to the unexplainable variance

ANOVA: assumptions Homogeneity of variance Normally distributed variances (across

the groups as well) Independence of observations

ANOVA is actually fairly robust when assumptions are violated, especially when sample size is large!

ANOVA: The Null

That all population groups have equal means Ho: μ1 = μ2 = μ3 = μ4 = μ5

ANOVA: Calculations Several intermediate steps before we

arrive at our final F-statistic Easiest to keep track of all steps

using a Summary Table

ANOVA: Partitioning Variance

The main idea being an ANOVA is to divide or separate (partition) variance observed in the data into categories of what we CAN and CANNOT explain

ANOVA: df

ANOVA: SS Sums of Squares…

Sums of squared deviations 3 Types

SSTotal: “Total Sum of Squares” ---sum of squared deviations of all observations from the grand mean, regardless of group membership

SSgroup: differences due to groups (diffs between group means)

SSerror: sum of squared deviations (residuals) within each group

ANOVA: SS

ANOVA: MS Mean Squares…(2 types):

MSwithin (MSerror): Variability among subjects in the same treatment group (their uniqueness, within their groups)

MSgroup: Variability among group means

ANOVA: MS

ANOVA: F Most important column in our table…

but based on every other column! Obtained by dividing MSgroup by MSerror

MS-error estimate of population variance MS-group estimate of population variance IF Ho IS TRUE!!!

When Ho is true, then both are estimates of the same thing, so the resulting F-statistic will be very close to 1 (give or take sampling error)

So, our questions becomes, is our resulting F close enough to 1 to support Ho

ANOVA: F

ANOVA: FANOVA: F So, our questions becomes, if our resulting So, our questions becomes, if our resulting

F F close enoughclose enough to 1 to support Ho to 1 to support Ho How close is close How close is close enough???enough??? Table E. 3Table E. 3

ANOVA: decision making

Fobt(dfgroup , dferror) = XXXXFcrit(dfgroup , dferror) = XXXX

Reject Ho if Fobt > Fcrit

What does it mean to reject Null of an F-test?

ANOVA: Summary of formulas

ANOVA: Multiple Comparisons

Allow us to investigate hypotheses about means of individual groups Why not just run t – tests repeatedly?

Ex- G1, G2, G3, G4, etc… G1 vs. G2; G1 vs. G3; G1 vs. G4; G2 vs. G3; etc…

ANOVA: Multiple Comparisons

Fisher’s LSD test: (Least Significant Difference) A.k.a. “protected t” Very LIBERAL (I tend to lean on the conservative

side) Only use if overall F is significant!!!

If F is NOT significant, do NOT use (conclude there are no group differences and be done with it)

Modified t – test Replace s2-pooled (pooled variance) with Mserror

Why? b/c by definition, Mserror is average of variances within each group

When have only 2 groups, then Mserror = s2-pooled

ANOVA: Multiple Comparisons

Fisher’s LSD test: (Least Significant Difference)

Why also called the “protected t”?

21

21

11nn

MS

xxt

error

ANOVA: Multiple Comparisons

Bonferroni Correction: More Conservative Uses the same modified t Familywise error rate is divided by # of comparisons

Ex: if I run 5 tests & I use alpha = .05, then the familywise error rate cannot exceed 5 X .05 = .25 (but that’s way too high to be acceptable)

But, if I divided alpha by the number of tests I run (5), then alpha = .05/5 = .05…and the familywise error rate cannot exceed 5 X .01 = .05

Ex: if I run 3 tests, and I don’t want my familywise error rate to exceed .05, then I can set alpha at .0167

ANOVA: Multiple Comparisons

Tukey We won’t go into it here…but I want you

to at least be exposed to it Most Popular…but much more complex

(easy to do with computers) Holds familywise error rate constant

ANOVA: magnitude of effect If we have a significant difference,

doesn’t mean it is meaningful Magnitudes of effect (NOT effect size)

tell us how much variability can be accounted for (explained or attributed to) our knowledge of the IV (group membership)

2 magnitudes of effect…

ANOVA: magnitude of effecteta squared (η2)

Biased because overestimates the effect, but very easy to calculate

Based on our SS

Total

group

SSSS

2

ANOVA: magnitude of effectOmega squared (ω2)

Much less biased (but more difficult to calculate -- but not that much harder)

errorTotal

errorgroup

MSSSMSkSS

)1(2

ANOVA: write-up A One-Way ANOVA was conducted to

determine…. The Independent Variable was XX with XX levels, and the Dependent Variable was XX. Results indicated that significant differences in the means of groups existed, F (df, df) = XXX, p < .05. The IV accounted for XX% of the dependant variable. Participants in the XX groups scored significantly higher than XX (etc.).

Class ExampleSee Handout…Does learning performance (# of

problems solved correctly) differ by Room Temperature?

What are research and null hypotheses for this study? What is alpha level and how many tails (one or two tailed)? Are there any differences between groups?

If so, where are they? What is the size of any observed differences?

Don’t Forget…HW #6

Due Tuesday

READ YOUR BOOK!!! (2-way ANOVA)

Evals on Tuesday