ANOVA Determining Which Means Differ in Single Factor Models Determining Which Means Differ in Single Factor Models.

ANOVAANOVA

Determining Which Means Determining Which Means Differ in Single Factor ModelsDiffer in Single Factor Models

Single Factor ModelsSingle Factor ModelsReview of AssumptionsReview of Assumptions

• Recall that the problem solved by ANOVA is to determine if at least one of the true mean values of several different treatments differs from the others.

• For ANOVA we assumed:1. The distribution of the populationpopulation for each

treatment is normalnormal.

2. The standard deviationsstandard deviations of each population, although unknown, are equalequal.

3. Sampling is randomrandom and independentindependent.

Determining Which Means DifferDetermining Which Means DifferBasic ConceptBasic Concept

• Suppose the result of performing a single factor ANOVA test is a low p-value, which indicates that at least one population mean does, in fact, differ from the others.

• The natural question is, “Which differ?”

• The answer is that we conclude that two population means differ if their two sample means differ by “a lot”.– The statistical question is, “What is ‘a lot’?”

ExampleExample

• The length of battery life for notebook computers is of concern to computer manufacturers.

• Toshiba is considering 5 different battery models (A, B, C, D, E) that have different costs.

• The question is, “Is there enough evidence to show that average battery life differs among battery types?”

DataData

A B C D E

130 90 100 140 160

115 80 95 150 150

130 95 110 150 155

125 98 100 125 145

120 92 105 145 165

110 85 90 130 125

x =121.67 90 100 140 150

Grand MeanGrand Mean

x =120

OUTPUTOUTPUT

p-value = .000000000108p-value = .000000000108Can conclude differencesCan conclude differences

Motivation for The Fisher Motivation for The Fisher ProcedureProcedure

• Fisher’s Procedure is a natural extension of the comparison of two population means when the unknown variances are assumed to be equal – Recall this is an assumption in single factor ANOVA

• Testing for the difference of two population means (with equal but unknown σ’s) has the form:

H0: μ1 – μ2 = 0

HA: μ1 – μ2 ≠ 0Reject H0 (Accept HA) if:

DFα/2,DFα/2,

21

2

21 t- or t

n

1

n

1)σfor estimate(best

0)xx( t

What do we use forthese two values?

Best Estimate for Best Estimate for σσ22 and the and the Appropriate Degrees of FredomAppropriate Degrees of Fredom

• Recall that when there were only 2 populations, the best estimate for σ2 is sp

2 and the degrees of freedom is (n1-1) + (n2-1) or n1 + n2 - 2.

• For ANOVA, using all the information from the k populations the best estimate for σ2 is MSE and the degrees of freedom is DFE.

Two populationsWith Equal Variances ANOVA

Best estimate for σ2 sp2 MSE

Degrees of Freedom n1 + n2 – 2 DFE

Two Types of TestsTwo Types of Tests• There are two types of tests that can be

applied:1. A test or a confidence interval for the

difference in two particular means • e.g. µE and µB

2. A set of tests which determine differences among all means.• This is called a set of experiment wise (EW) tests.

• The approach is the same. – We will illustrate an approach called the Fisher Fisher

LSD approachLSD approach.– Only the value used for αα will be different will be different.

Determining if Determining if μμii Differs From Differs From μμjj

Fisher’s LSD ApproachFisher’s LSD ApproachH0: μi – μj = 0

HA: μi – μj ≠ 0

Reject HReject H00 (Accept H (Accept HAA) if:) if:

DFEα/2,DFEα/2,

ji

ji t- or t

n

1

n

1 MSE

0)xx( t

That is, we conclude there is a differences between That is, we conclude there is a differences between μμii and and μμjj if if

jiDFEα/2,ji n

1

n

1 MSE txx LSDLSD

LSD stands for “Least Significant Difference”

When Do We Conclude Two When Do We Conclude Two Treatment Means (Treatment Means (µµii and µ and µjj) Differ?) Differ?

• We conclude that two means differ, if their sample means,xi andxj, differ by “a lot”.

• “A lot” is LSD given by:

jiDFEα/2, n

1

n

1 MSE t LSD

Confidence Intervals for the Difference in Two Population Means

• A confidence interval for μi – μj is found by:

Confidence Interval for Confidence Interval for μμii – – μμjj

jiDFEα/2,ji n

1

n

1MSEt)xx(

Confidence Interval for Confidence Interval for μμii – – μμjj

LSD)xx( ji

Equal vs. Unequal Sample SizesEqual vs. Unequal Sample Sizes• If the sample sizes drawn from the various

populations differ, then the denominator of the t-statistic will be different for each pairwise comparison.

• But if the sample sizes are equal (n1 = n2 = n3 = ….) , we can designate the equal sample size by N

• Then the t-test becomes:Reject H0 (Accept HA) if:

N

2 MSEtxx ifor t- or t

N

2 MSE

0)xx( t DFEα/2,jiDFEα/2,DFEα/2,

ji

LSDLSD

LSD For Equal Sample SizesLSD For Equal Sample Sizes

N

2 MSE t LSD DFEα/2,

What Do We Use For What Do We Use For αα??

• Recall that α is • In Hypothesis Tests: the probability of concluding that there is

a difference when there is not.• In Confidence Intervals: the probability the interval will not

contain the true difference in mean values

• If doing a single comparison test or constructing a confidence interval,

• For an experimentwise comparison of all means,• We will actually be conducting 10 t-tests:

(1) μE - μD, (2) μE - μC, (3) μE - μB, (4) μE - μA, (5) μD - μC,

(6) μD - μB, (7) μD - μA, (8) μC – μB, (9) μC - μA, (10) μB - μA

select α as usual

Use αEW

ααEWEW = The probability of = The probability of

Making at least one Type I ErrorMaking at least one Type I Error• Suppose each test has a probability of

concluding that there is a difference when there is not (making a Type I error) =(making a Type I error) = αα..– Thus for each test, the probability of not probability of not

making a Type I error is 1-making a Type I error is 1-αα..

• So the probability of not making any Type I errors on any of the 10 tests is: (1- (1- αα))1010

• For α = .05, this is (.95)10 = .5987• The probability of making at least one Type I probability of making at least one Type I error in error in

this experimentthis experiment, is denoted by ααEW.EW.

• Here, ααEW EW = 1 - .5987 = .4013 -- = 1 - .5987 = .4013 -- That is, the probability the probability we make at least one mistake is now over 40%! over 40%!

• To have a lower lower ααEWEW, α for each test must be significantly reduced.

The Bonferroni Adjustment for The Bonferroni Adjustment for αα • To make αEW reasonable, say .05, α for each test must

be reduced.• The Bonferroni Adjustment is as follows:

NOTE: decreasing α, increases β, the probability of not concluding that there is a difference between two means when there really is. Thus, some researchers are reluctant to make α too small because this can result in very high β values.

αα for each Test for each TestFor an experimentwise value, αEW, for each test use

αα = = ααEWEW/c/c

c = number of testsc = number of testsFor k treatments,

c = k(k-1)/2c = k(k-1)/2

What Should What Should αα for Each Test Be? for Each Test Be?

For αEW = .10

Number of Treatments, k

α

for each test

3 0.03333

4 0.01667

5 0.01000

6 0.00667

7 0.00476

8 0.00357

9 0.00278

10 0.00222

For αEW = .05

Number of Treatments, k

α

for each test

3 0.01667

4 0.00833

5 0.00500

6 0.00333

7 0.00238

8 0.00179

9 0.00139

10 0.00111

The required α values for the individual t-tests for αEW = .05 and αEW = .10 are:

LSDLSDEWEW

For Multiple Comparison TestsFor Multiple Comparison TestsWhen doing the series of multiple comparison tests

to determine which means differ, the test would be to conclude that µi differs from µj if :

Where LSDEW is given by:

EWji LSD |xx|

jiDFE/c)/2,(αEW n

1

n

1 MSE tLSD

EW

When ni ≠ nj

N

2 MSE tLSD DFE/c)/2,(αEW EW

When ni = nj = N

α for each test

Procedure for Testing Differences Procedure for Testing Differences Among All MeansAmong All Means

• We begin by calculating LSDEW which we have shown will not change from test to test if the sample sizes are the same from each sample. That is the situation in the battery example that we illustrate here.

– A different LSD would have to be calculated for each comparison if the sample sizes are different.

Procedure (continued)Procedure (continued)

• Then construct a matrix as follows

A B C D E

ABCDE

Procedure (continued)Procedure (continued)• Fill in mean of each treatment across the top row and down the left-most column; (in our example,

XA = 121.67, XB = 90, XC= 100, XD = 140, XE = 150)

A B C D EMean 121.67 90 100 140 150

A 121.67 B 90 C 100 D 140 E 150

Procedure (continued)Procedure (continued)• For each cell below the main diagonal,

compute the absolute value of difference of the means in the corresponding column and row

A B C D EMean 121.67 90 100 140 150

A 121.67 B 90 31.67 C 100 21.67 10.00 D 140 18.33 50.00 40.00 E 150 28.33 60.00 50.00 10.00

ji XX

Procedure (continued)Procedure (continued)

• Compare each difference with LSDEw(17.235 in our case). If the

• difference between and > LSDEw. we can conclude that there is difference in µi and µj.

A B C D EMean 121.67 90 100 140 150

A 121.67 B 90 31.67 C 100 21.67 10.00 D 140 18.33 50.00 40.00 E 150 28.33 60.00 50.00 10.00

X i X j

Tests For the Battery ExampleTests For the Battery Example

• For the battery example,1. Which average battery lives can we

conclude differ?

2. Give a 95% confidence interval for the difference in average battery lives between:

• C batteries and B batteries• E batteries and B batteries

Use LSDEW

Multiple Comparisons

Use LSD

Individual Comparisons

Battery Example Calculations

• Experimental error of EW = .05

• For k = 5 populations, α = αEW /10 = .05/10 = .005

• From the Excel output: xE = 150, xD = 140, xA = 121.67, xc = 100, xB = 90

MSE = 94.05333, DFE = 25, N = 6 from each population

• Use TINV(.005,25) to generate t.0025,25 = 3.078203

2355.176

205333.94078203.3

225,0025.

NMSEtLSDEW

Analysis of Which Means DifferAnalysis of Which Means Differ

• We conclude that two population means differ if their sample means differ by more than LSDEW = 17.2355.

• Construct a matrix of differences,

• Compare with LSDEW

i A B C D Ej 121.67 90 100 140 150 A 121.67 B 90 31.67 C 100 21.67 10 D 140 18.33 50 40 E 150 28.33 60 50 10

ji XX

Conclusion of ComparisonsConclusion of Comparisons

ABS(XA-BAR - XB-BAR) = 31.67 Greater than LSD(EW); Can conclude a difference

ABS(XA-BAR - XC-BAR) = 21.67 Greater than LSD(EW); Can conclude a difference

ABS(XA-BAR - XD-BAR) = 18.33 Greater than LSD(EW); Can conclude a difference

ABS(XA-BAR - XE-BAR) = 28.33 Greater than LSD(EW); Can conclude a difference

ABS(XB-BAR - XC-BAR) = 10 Less than LSD(EW); Cannot conclude a difference

ABS(XB-BAR - XD-BAR) = 50 Greater than LSD(EW); Can conclude a differenceABS(XB-BAR - XE-BAR) = 60 Greater than LSD(EW); Can conclude a differenceABS(XC-BAR - XD-BAR) = 40 Greater than LSD(EW); Can conclude a differenceABS(XA-BAR - XB-BAR) = 50 Greater than LSD(EW); Can conclude a differenceABS(XD-BAR - XE-BAR) = 10 Less than LSD(EW); Cannot conclude a difference

LSD For Confidence IntervalsLSD For Confidence Intervals

Confidence intervals for the difference between two mean values, i and j, are of the form:

(Point Estimate) ± t/2,DFE(Standard Error)

ji xx = .05

ji n

1

n

1MSE

N

2MSE

N nn If ji

LSDLSD(not LSD(not LSDEWEW))

Confidence Interval forConfidence Interval for

LSDxx ji ji xx

LSD for Battery ExampleLSD for Battery Example

For the battery example:

5318.116

205333.9405954.2

225,025.

NMSEtLSD

The Confidence Intervals• 95% confidence interval for the difference in

mean battery lives between batteries of type C and batteries of type B.

• 95% confidence interval for the difference in mean battery lives between batteries of type E and batteries of type B.

Confidence Interval for Confidence Interval for μμCC – – μμBB

21.5398 1.5398-

11.5398 90)- (100 LSD)xx( BC

Confidence Interval for Confidence Interval for μμEE – – μμBB

71.5398 48.4602

11.5398 90)- (150 LSD)xx( BE

REVIEWREVIEW

• The Fisher LSD Test– What to use for:

• Best Estimate of σ2 = MSE• Degrees of Freedom = DFE

– Calculation of LSD

• Bonferroni Modification– Modify α so that αEW is reasonable

– α = αEW/c, where the # of tests, c = k(k-1)/2

– Calculation of LSDEW

• Excel Calculations