1 1 Slide Simple Linear Regression Coefficient of Determination Chapter 14 BA 303 – Spring 2011.

1 1 Slide Slide

Simple Linear RegressionCoefficient of Determination

Chapter 14BA 303 – Spring 2011

2 2 Slide Slide

COEFFICIENT OF DETERMINATION

3 3 Slide Slide

Assessing the Regression Model

The Coefficient of Determination provides a measure of the goodness of fit for the estimated regression equation.

Sum of Squares

Coefficient of Determination

Correlation Coefficient

4 4 Slide Slide

Coefficient of Determination

Relationship Among SST, SSR, SSE

where:

SST = total sum of squares

SSR = sum of squares due to regression

SSE = sum of squares due to error

SST = SSR + SSE

2( )iy y 2ˆ( )iy y 2ˆ( )i iy y

5 5 Slide Slide

Sum of SquaresReed Auto Sales

1 14 15

3 24 25

2 18 20

1 17 15

3 27 25

ix iy iy

ˆ 10 5y x

What is the relationship between TV ads and auto sales?

6 6 Slide Slide

Sum of Squares Due to Error

1 14 15 -1 13 24 25 -1 12 18 20 -2 41 17 15 2 43 27 25 2 4

14

iy )ˆ( ii yy 2)ˆ( ii yy ix iya

b

c

d

e

2ˆ( )i iy y

7 7 Slide Slide

Sum of Squares Due to Regression

1 15 -5 253 25 5 252 20 0 01 15 -5 253 25 5 25

100

)ˆ( yyi 2)ˆ( yyi ix iy

20y

2ˆ( )iy y

a

b

c

d

e

8 8 Slide Slide

Sum of Squares Total

1 14 -6 363 24 4 162 18 -2 41 17 -3 93 27 7 49

114

ix )( yyi 2)( yyi iy

20y

2( )iy y

a

b

c

d

e

9 9 Slide Slide

SST = SSR + SSE?

SST = SSR + SSE

114 = 100 + 14

10 10 Slide Slide

The coefficient of determination is:

Coefficient of Determination

where:SSR = sum of squares due to regressionSST = total sum of squares

r2 = SSR/SST

11 11 Slide Slide

Coefficient of Determination

r2 = SSR/SST = 100/114 = 0.8772

The regression relationship is very strong; 87.72% of the variability in the number of cars sold can be explained by the linear relationship between the number of TV ads and the number of cars sold.

12 12 Slide Slide

The sign of b1 in the equation is “+”.

Sample Correlation Coefficient

rxy = +0.9366

21 ) of(sign rbrxy

=+ .8772xyr

ˆ 10 5y x

The correlation coefficient of +0.9366 indicates a strong positive relationship between the independent variable and the dependent variable.

13 13 Slide Slide

SUM OF SQUARESPRACTICE

14 14 Slide Slide

Practice

1 3

2 7

3 5

4 11

5 14

ix iy

ii xbby 10ˆ =0.2+2.6*x

21 21 Slide Slide

TESTS FOR SIGNIFICANCE

22 22 Slide Slide

Testing for Significance

To test for a significant regression relationship, we must conduct a hypothesis test to determine whether the value of b1 is zero.

Two tests are commonly used:

t Test and F Test

Both the t test and F test require an estimate of s 2, the variance of e in the regression model.

23 23 Slide Slide

An Estimate of s 2

Testing for Significance

210

2 )()ˆ(SSE iiii xbbyyy

where:

s 2 = MSE = SSE/(n - 2)

The mean square error (MSE) provides the estimateof s 2, and the notation s2 is also used.

24 24 Slide Slide

Testing for Significance

An Estimate of s

2

SSEMSE

n

s

To estimate s we take the square root of s 2.

The resulting s is called the standard error of the estimate.

25 25 Slide Slide

t TEST

26 26 Slide Slide

Hypotheses

Test Statistic

Testing for Significance: t Test

0 1: 0H

1: 0aH

1

1

b

bt

s where

1 2( )b

i

ss

x x

27 27 Slide Slide

Rejection Rule

Testing for Significance: t Test

where: is the desired level of significancet is based on a t distribution with n - 2 degrees of freedom

Reject H0 if p-vzalue < a or t < -tor t > t

28 28 Slide Slide

t Distribution Table

29 29 Slide Slide

1. Determine the hypotheses.

2. Specify the level of significance.

3. Select the test statistic.

a = 0.05

4. State the rejection rule.Reject H0 if p-value < 0.05 or |t| > 3.182 (with 3 degrees of freedom)

Testing for Significance: t Test

0 1: 0H

1: 0aH

1

1

b

bt

s

30 30 Slide Slide

Testing for Significance: t Test

5. Compute the value of the test statistic.

6. Determine whether to reject H0.

t = 4.63 > 3.182. We can reject H0.

1

1 54.63

1.08b

bt

s

31 31 Slide Slide

A Little Bit Slower! t Test

Already known:

b1=5

4)( 2 xxi

n=5

Today:2

SSEMSE

n

s

SSE = 14

So s = 2.1602 = 2.16

32 32 Slide Slide

A Little Bit Slower! t Test

1

1

b

bt

s

1 2( )b

i

ss

x x

= 2.16/2 = 1.08

= 5/1.08 = 4.6296 = 4.63

Since our t=4.63 is greater than the test t/2=3.182 , we reject the null hypothesis and conclude b1 is not equal to zero.

33 33 Slide Slide

PRACTICEt TEST

34 34 Slide Slide

t Test

b1 = 2.6

10)( 2 xxi

n = 5What We Know

SSE = 12.4

Find t for a = 0.10

1 2( )b

i

ss

x x

2

SSEMSE

n

s

1

1

b

bt

s

38 38 Slide Slide

t CONFIDENCE INTERVAL

39 39 Slide Slide

Confidence Interval for 1

H0 is rejected if the hypothesized value of 1 is not included in the confidence interval for 1.

We can use a 95% confidence interval for 1 to test the hypotheses just used in the t test.

40 40 Slide Slide

The form of a confidence interval for 1 is:

Confidence Interval for 1

11 / 2 bb t s

where is the t value providing an areaof a/2 in the upper tail of a t distributionwith n - 2 degrees of freedom

2/tb1 is the

pointestimat

or

is themarginof error

1/ 2 bt s

41 41 Slide Slide

Confidence Interval for 1

Reject H0 if 0 is not included in the confidence interval for 1.

0 is not included in the confidence interval. Reject H0

= 5 +/- 3.182(1.08) = 5 +/- 3.4412/1 bstb

or 1.56 to 8.44

Rejection Rule

95% Confidence Interval for 1

Conclusion

42 42 Slide Slide

F TEST

43 43 Slide Slide

Hypotheses

Test Statistic

Testing for Significance: F Test

F = MSR/MSE

0 1: 0H

1: 0aH

Table 4 – F DistributionMSR d.f. = 1 (numerator)MSE d.f. = n – 2 (denominator)

44 44 Slide Slide

Rejection Rule

Testing for Significance: F Test

where:F is based on an F distribution with

1 degree of freedom in the numerator andn - 2 degrees of freedom in the denominator

Reject H0 if p-value < a or F > F

45 45 Slide Slide

1. Determine the hypotheses.

2. Specify the level of significance.

3. Select the test statistic.

a= 0.05Fa=10.13

4. State the rejection rule. Reject H0 if p-value < 0.05 or F > 10.13 (with 1 d.f.in numerator and 3 d.f. in denominator)

Testing for Significance: F Test

0 1: 0H

1: 0aH

F = MSR/MSE

46 46 Slide Slide

Testing for Significance: F Test

5. Compute the value of the test statistic.

F = 10.13 provides an area of 0.05 in the tail. Thus, the p-value corresponding to F = 21.43 is less than 0.05. Hence, we reject H0.

F = MSR/MSE = 100/4.667 = 21.43

The statistical evidence is sufficient to concludethat we have a significant relationship between thenumber of TV ads aired and the number of cars sold.

6. Determine whether to reject H0.

47 47 Slide Slide

PRACTICEF TEST

48 48 Slide Slide

F Test

Fa

n = 5

SSE = 12.4

SSR = 67.6

Find a = 0.05

MSR = SSR/d.f. Regression

F

MSE = SSE/(n – 2)

51 51 Slide Slide

CAUTIONS

52 52 Slide Slide

Some Cautions about theInterpretation of Significance Tests

Just because we are able to reject H0: b1 = 0 and demonstrate statistical significance does not enable

us to conclude that there is a linear relationshipbetween x and y.

Rejecting H0: b1 = 0 and concluding that the

relationship between x and y is significant does not enable us to conclude that a cause-and-effect

relationship is present between x and y.

53 53 Slide Slide

Assumptions About the Error Term e

1. The error is a random variable with mean of zero.

2. The variance of , denoted by 2, is the same for all values of the independent variable.

3. The values of are independent.

4. The error is a normally distributed random variable.

54 54 Slide Slide