11 - 11 - 1 1 © 2003 Pearson Prentice © 2003 Pearson Prentice Hall Hall Chapter 11 Chapter 11 Simple Linear Regression Simple Linear Regression
Apr 01, 2015
11 - 11 - 11
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Chapter 11Chapter 11
Simple Linear Regression Simple Linear Regression
11 - 11 - 22
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Learning ObjectivesLearning Objectives
1.1. Describe the Linear Regression ModelDescribe the Linear Regression Model
2.2. State the Regression Modeling StepsState the Regression Modeling Steps
3.3. Explain Ordinary Least SquaresExplain Ordinary Least Squares
1.1. Understand and check model assumptionsUnderstand and check model assumptions
4.4. Compute Regression CoefficientsCompute Regression Coefficients
5.5. Predict Response VariablePredict Response Variable
6.6. Interpret Computer OutputInterpret Computer Output
11 - 11 - 33
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
ModelsModels
11 - 11 - 44
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
ModelsModels
1.1. Representation of Some PhenomenonRepresentation of Some Phenomenon
2.2. Mathematical Model Is a Mathematical Mathematical Model Is a Mathematical Expression of Some PhenomenonExpression of Some Phenomenon
3.3. Often Describe Relationships between Often Describe Relationships between VariablesVariables
4.4. TypesTypes Deterministic ModelsDeterministic Models Probabilistic ModelsProbabilistic Models
11 - 11 - 55
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Deterministic Deterministic ModelsModels
1.1. Hypothesize Exact RelationshipsHypothesize Exact Relationships
2.2. Suitable When Prediction Error is Suitable When Prediction Error is NegligibleNegligible
3.3. Example: Force Is Exactly Example: Force Is Exactly Mass Times AccelerationMass Times Acceleration FF = = mm··aa
© 1984-1994 T/Maker Co.
11 - 11 - 66
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Probabilistic ModelsProbabilistic Models
1.1. Hypothesize 2 ComponentsHypothesize 2 Components DeterministicDeterministic Random ErrorRandom Error
2.2. Example: Sales Volume Is 10 Times Example: Sales Volume Is 10 Times Advertising Spending + Random ErrorAdvertising Spending + Random Error YY = 10 = 10X X + + Random Error May Be Due to Factors Random Error May Be Due to Factors
Other Than AdvertisingOther Than Advertising
11 - 11 - 77
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Types of Types of Probabilistic ModelsProbabilistic Models
ProbabilisticModels
RegressionModels
CorrelationModels
OtherModels
ProbabilisticModels
RegressionModels
CorrelationModels
OtherModels
11 - 11 - 88
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Regression ModelsRegression Models
11 - 11 - 99
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Types of Types of Probabilistic ModelsProbabilistic Models
ProbabilisticModels
RegressionModels
CorrelationModels
OtherModels
ProbabilisticModels
RegressionModels
CorrelationModels
OtherModels
11 - 11 - 1010
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Regression ModelsRegression Models
1.1. Answer ‘What Is the Relationship Answer ‘What Is the Relationship Between the Variables?’Between the Variables?’
2.2. Equation UsedEquation Used 1 Numerical Dependent (Response) Variable1 Numerical Dependent (Response) Variable
What Is to Be PredictedWhat Is to Be Predicted 1 or More Numerical or Categorical 1 or More Numerical or Categorical
Independent (Explanatory) VariablesIndependent (Explanatory) Variables
3.3. Used Mainly for Prediction & EstimationUsed Mainly for Prediction & Estimation
11 - 11 - 1111
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Regression Modeling Regression Modeling Steps Steps
1.1. Hypothesize Deterministic ComponentHypothesize Deterministic Component
2.2. Estimate Unknown Model ParametersEstimate Unknown Model Parameters
3.3. Specify Probability Distribution of Specify Probability Distribution of Random Error TermRandom Error Term Estimate Standard Deviation of ErrorEstimate Standard Deviation of Error
4.4. Evaluate ModelEvaluate Model
5.5. Use Model for Prediction & Estimation Use Model for Prediction & Estimation
11 - 11 - 1212
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Model SpecificationModel Specification
11 - 11 - 1313
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Regression Modeling Regression Modeling Steps Steps
1.1. Hypothesize Deterministic ComponentHypothesize Deterministic Component
2.2. Estimate Unknown Model ParametersEstimate Unknown Model Parameters
3.3. Specify Probability Distribution of Random Specify Probability Distribution of Random Error TermError Term Estimate Standard Deviation of ErrorEstimate Standard Deviation of Error
4.4. Evaluate ModelEvaluate Model
5.5. Use Model for Prediction & Estimation Use Model for Prediction & Estimation
11 - 11 - 1414
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Specifying the Specifying the ModelModel
1.1. Define VariablesDefine Variables
2.2. Hypothesize Nature of RelationshipHypothesize Nature of Relationship Expected Effects (i.e., Coefficients’ Signs)Expected Effects (i.e., Coefficients’ Signs) Functional Form (Linear or Non-Linear)Functional Form (Linear or Non-Linear) InteractionsInteractions
11 - 11 - 1515
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Model Specification Model Specification Is Based on TheoryIs Based on Theory
1.1. Theory of Field (e.g., Sociology)Theory of Field (e.g., Sociology)
2.2. Mathematical TheoryMathematical Theory
3.3. Previous ResearchPrevious Research
4.4. ‘Common Sense’‘Common Sense’
11 - 11 - 1616
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Advertising
Sales
Advertising
Sales
Advertising
Sales
Advertising
Sales
Advertising
Sales
Advertising
Sales
Advertising
Sales
Advertising
Sales
Thinking Challenge: Thinking Challenge: Which Is More Which Is More
Logical?Logical?
11 - 11 - 1717
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Types of Types of Regression ModelsRegression Models
11 - 11 - 1818
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Types of Types of Regression ModelsRegression Models
RegressionModels
11 - 11 - 1919
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Types of Types of Regression ModelsRegression Models
RegressionModels
Simple
1 Explanatory1 ExplanatoryVariableVariable
11 - 11 - 2020
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Types of Types of Regression ModelsRegression Models
RegressionModels
2+ Explanatory2+ ExplanatoryVariablesVariables
Simple Multiple
1 Explanatory1 ExplanatoryVariableVariable
11 - 11 - 2121
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Types of Types of Regression ModelsRegression Models
RegressionModels
Linear
2+ Explanatory2+ ExplanatoryVariablesVariables
Simple Multiple
1 Explanatory1 ExplanatoryVariableVariable
11 - 11 - 2222
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Types of Types of Regression ModelsRegression Models
RegressionModels
LinearNon-
Linear
2+ Explanatory2+ ExplanatoryVariablesVariables
Simple Multiple
1 Explanatory1 ExplanatoryVariableVariable
11 - 11 - 2323
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Types of Types of Regression ModelsRegression Models
RegressionModels
LinearNon-
Linear
2+ Explanatory2+ ExplanatoryVariablesVariables
Simple Multiple
Linear
1 Explanatory1 ExplanatoryVariableVariable
11 - 11 - 2424
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Types of Types of Regression ModelsRegression Models
RegressionModels
LinearNon-
Linear
2+ Explanatory2+ ExplanatoryVariablesVariables
Simple Multiple
Linear
1 Explanatory1 ExplanatoryVariableVariable
Non-Linear
11 - 11 - 2525
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Linear Regression Linear Regression ModelModel
11 - 11 - 2626
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Types of Types of Regression ModelsRegression Models
RegressionModels
LinearNon-
Linear
2+ ExplanatoryVariables
Simple
Non-Linear
Multiple
Linear
1 ExplanatoryVariable
RegressionModels
LinearNon-
Linear
2+ ExplanatoryVariables
Simple
Non-Linear
Multiple
Linear
1 ExplanatoryVariable
11 - 11 - 2727
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Y
Y = mX + b
b = Y-intercept
X
Changein Y
Change in X
m = Slope
Linear EquationsLinear Equations
High School TeacherHigh School Teacher© 1984-1994 T/Maker Co.
YY XXii ii ii 00 11
Linear Regression Linear Regression ModelModel
1.1. Relationship Between Variables Is a Relationship Between Variables Is a Linear FunctionLinear Function
Dependent Dependent (Response) (Response) VariableVariable(e.g., income)(e.g., income)
Independent Independent (Explanatory) (Explanatory) Variable Variable (e.g., education)(e.g., education)
Population Population SlopeSlope
Population Population Y-InterceptY-Intercept
Random Random ErrorError
11 - 11 - 2929
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Population & Population & Sample Regression Sample Regression
ModelsModels
11 - 11 - 3030
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Population & Population & Sample Regression Sample Regression
ModelsModels
PopulationPopulation
$ $
$
$
$
11 - 11 - 3131
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Population & Population & Sample Regression Sample Regression
ModelsModels
Unknown Relationship
PopulationPopulation
Y Xi i i 0 1
$
$
$
$ $
11 - 11 - 3232
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Population & Population & Sample Regression Sample Regression
ModelsModels
Unknown Relationship
PopulationPopulation Random SampleRandom Sample
Y Xi i i 0 1
$ $$
$
$ $$
$$ $$
11 - 11 - 3333
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Population & Population & Sample Regression Sample Regression
ModelsModels
Unknown Relationship
PopulationPopulation Random SampleRandom Sample
Y Xi i i 0 1
Y Xi i i 0 1Y Xi i i 0 1
$ $$
$
$ $$
$$ $$
11 - 11 - 3434
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Y
X
Y
X
Population Linear Population Linear Regression ModelRegression Model
Y Xi i i 0 1Y Xi i i 0 1
iXYE 10 iXYE 10
ObservedObservedvaluevalue
Observed valueObserved value
ii = Random error= Random error
11 - 11 - 3535
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Y
X
Y
X
Y Xi i i 0 1Y Xi i i 0 1
Sample Linear Sample Linear Regression ModelRegression Model
Y Xi i 0 1 Y Xi i 0 1
Unsampled Unsampled observationobservation
ii = Random = Random
errorerror
Observed valueObserved value
^
11 - 11 - 3636
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Estimating Parameters:Estimating Parameters:Least Squares MethodLeast Squares Method
11 - 11 - 3737
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Regression Modeling Regression Modeling Steps Steps
1.1. Hypothesize Deterministic ComponentHypothesize Deterministic Component
2.2. Estimate Unknown Model ParametersEstimate Unknown Model Parameters
3.3. Specify Probability Distribution of Specify Probability Distribution of Random Error TermRandom Error Term Estimate Standard Deviation of ErrorEstimate Standard Deviation of Error
4.4. Evaluate ModelEvaluate Model
5.5. Use Model for Prediction & EstimationUse Model for Prediction & Estimation
11 - 11 - 3838
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
0204060
0 20 40 60
X
Y
ScattergramScattergram
1.1. Plot of All (Plot of All (XXii, , YYii) Pairs) Pairs
2.2. Suggests How Well Model Will FitSuggests How Well Model Will Fit
11 - 11 - 3939
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
0204060
0 20 40 60
X
Y
Thinking ChallengeThinking Challenge
How would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?
11 - 11 - 4040
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
0204060
0 20 40 60
X
Y
Thinking ChallengeThinking Challenge
How would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?
11 - 11 - 4141
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
0204060
0 20 40 60
X
Y
Thinking ChallengeThinking Challenge
How would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?
11 - 11 - 4242
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
0204060
0 20 40 60
X
Y
Thinking ChallengeThinking Challenge
How would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?
11 - 11 - 4343
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
0204060
0 20 40 60
X
Y
Thinking ChallengeThinking Challenge
How would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?
11 - 11 - 4444
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
0204060
0 20 40 60
X
Y
Thinking ChallengeThinking Challenge
How would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?
11 - 11 - 4545
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
0204060
0 20 40 60
X
Y
Thinking ChallengeThinking Challenge
How would you draw a line through the How would you draw a line through the points? How do you determine which line points? How do you determine which line ‘fits best’?‘fits best’?
11 - 11 - 4646
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Least SquaresLeast Squares
1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values Actual Y Values & Predicted Y Values Are a MinimumAre a Minimum ButBut Positive Differences Off-Set Negative Positive Differences Off-Set Negative
11 - 11 - 4747
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Least SquaresLeast Squares
1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values Actual Y Values & Predicted Y Values Are a MinimumAre a Minimum ButBut Positive Differences Off-Set Negative Positive Differences Off-Set Negative
n
ii
n
iii YY
1
2
1
2ˆˆ
n
ii
n
iii YY
1
2
1
2ˆˆ
11 - 11 - 4848
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Least SquaresLeast Squares
1.1. ‘Best Fit’ Means Difference Between ‘Best Fit’ Means Difference Between Actual Y Values & Predicted Y Values Are Actual Y Values & Predicted Y Values Are a Minimuma Minimum ButBut Positive Differences Off-Set Negative Positive Differences Off-Set Negative
2.2. LS Minimizes the Sum of the Squared LS Minimizes the Sum of the Squared Differences (SSE)Differences (SSE)
n
ii
n
iii YY
1
2
1
2ˆˆ
n
ii
n
iii YY
1
2
1
2ˆˆ
11 - 11 - 4949
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Least Squares Least Squares GraphicallyGraphically
2
Y
X
1 3
4
^^
^2
Y
X
1 3
4
^^
^^
Y X2 0 1 2 2 Y X2 0 1 2 2
Y Xi i 0 1 Y Xi i 0 1
LS minimizes ii
n2
112
22
32
42
LS minimizes ii
n2
112
22
32
42
11 - 11 - 5050
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient Coefficient EquationsEquations
Sample SlopeSample Slope
Sample Y-interceptSample Y-intercept
Prediction EquationPrediction Equation
xy 10 ˆˆ
21
xx
yyxxSS
SS
i
ii
xx
xy
ii xy 10 ˆˆˆ
11 - 11 - 5151
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Computation TableComputation Table
Xi Yi Xi2 Yi
2 XiYi
X1 Y1 X12 Y1
2 X1Y1
X2 Y2 X22 Y2
2 X2Y2
: : : : :
Xn Yn Xn2 Yn
2 XnYn
XiYi
Xi2 Yi
2 XiYi
Xi Yi Xi2 Yi
2 XiYi
X1 Y1 X12 Y1
2 X1Y1
X2 Y2 X22 Y2
2 X2Y2
: : : : :
Xn Yn Xn2 Yn
2 XnYn
XiYi
Xi2 Yi
2 XiYi
11 - 11 - 5252
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Interpretation of Interpretation of CoefficientsCoefficients
11 - 11 - 5353
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Interpretation of Interpretation of CoefficientsCoefficients
1.1. Slope (Slope (11)) Estimated Estimated YY Changes by Changes by 11 for Each 1 for Each 1
Unit Increase in Unit Increase in XX If If 11 = 2, then Sales ( = 2, then Sales (YY) Is Expected to ) Is Expected to
Increase by 2 for Each 1 Unit Increase in Increase by 2 for Each 1 Unit Increase in Advertising (Advertising (XX))
^
^
^
11 - 11 - 5454
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Interpretation of Interpretation of CoefficientsCoefficients
1.1. Slope (Slope (11)) Estimated Estimated YY Changes by Changes by 11 for Each 1 for Each 1
Unit Increase in Unit Increase in XX If If 11 = 2, then Sales ( = 2, then Sales (YY) Is Expected to Increase ) Is Expected to Increase
by 2 for Each 1 Unit Increase in Advertising (by 2 for Each 1 Unit Increase in Advertising (XX))
2.2. Y-Intercept (Y-Intercept (00)) Average Value of Average Value of YY When When XX = 0 = 0
If If 00 = 4, then Average Sales ( = 4, then Average Sales (YY) Is Expected to ) Is Expected to Be 4 When Advertising (Be 4 When Advertising (XX) Is 0) Is 0
^
^
^^
^
11 - 11 - 5555
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Parameter Parameter Estimation ExampleEstimation Example
You’re a marketing analyst for Hasbro Toys. You’re a marketing analyst for Hasbro Toys. You gather the following data:You gather the following data:
Ad $Ad $ Sales (Units)Sales (Units)11 1122 1133 2244 2255 44
What is the What is the relationshiprelationship between sales & advertising?between sales & advertising?
11 - 11 - 5656
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
0
1
2
3
4
0 1 2 3 4 5
Scattergram Scattergram Sales vs. AdvertisingSales vs. Advertising
Sales
Advertising
11 - 11 - 5757
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Guess The Parameters!Guess The Parameters!
11 - 11 - 5858
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
0
1
2
3
4
0 1 2 3 4 5
Scattergram Scattergram Sales vs. AdvertisingSales vs. Advertising
Sales
Advertising
11 - 11 - 5959
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Parameter Parameter Estimation Solution Estimation Solution
TableTableXi Yi Xi
2 Yi2 XiYi
1 1 1 1 1
2 1 4 1 2
3 2 9 4 6
4 2 16 4 8
5 4 25 16 20
15 10 55 26 37
Xi Yi Xi2 Yi
2 XiYi
1 1 1 1 1
2 1 4 1 2
3 2 9 4 6
4 2 16 4 8
5 4 25 16 20
15 10 55 26 37
11 - 11 - 6060
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Parameter Parameter Estimation SolutionEstimation Solution
10.0370.02ˆˆ
70.0
515
55
51015
37ˆ
10
2
1
2
12
11
11
XY
n
X
X
n
YX
YX
n
i
n
ii
i
n
ii
n
iin
iii
10.0370.02ˆˆ
70.0
515
55
51015
37ˆ
10
2
1
2
12
11
11
XY
n
X
X
n
YX
YX
n
i
n
ii
i
n
ii
n
iin
iii
11 - 11 - 6161
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient Coefficient Interpretation Interpretation
SolutionSolution
11 - 11 - 6262
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient Coefficient Interpretation Interpretation
SolutionSolution1.1. Slope (Slope (11))
Sales Volume (Sales Volume (YY) Is Expected to Increase ) Is Expected to Increase by .7 Units for Each $1 Increase in by .7 Units for Each $1 Increase in Advertising (Advertising (XX))
^
11 - 11 - 6363
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient Coefficient Interpretation Interpretation
SolutionSolution1.1. Slope (Slope (11))
Sales Volume (Sales Volume (YY) Is Expected to Increase ) Is Expected to Increase by .7 Units for Each $1 Increase in Advertising by .7 Units for Each $1 Increase in Advertising ((XX))
2.2. Y-Intercept (Y-Intercept (00)) Average Value of Sales Volume (Average Value of Sales Volume (YY) Is ) Is
-.10 Units When Advertising (-.10 Units When Advertising (XX) Is 0) Is 0 Difficult to Explain to Marketing ManagerDifficult to Explain to Marketing Manager Expect Some Sales Without AdvertisingExpect Some Sales Without Advertising
^
^
11 - 11 - 6464
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Parameter EstimatesParameter Estimates
ParameterParameter Standard T for H0: Standard T for H0:
VariableVariable DF DF EstimateEstimate Error Param=0 Prob>|T| Error Param=0 Prob>|T|
INTERCEPINTERCEP 1 1 -0.1000-0.1000 0.6350 -0.157 0.8849 0.6350 -0.157 0.8849
ADVERTADVERT 1 1 0.70000.7000 0.1914 3.656 0.0354 0.1914 3.656 0.0354
Parameter Parameter Estimation Computer Estimation Computer
OutputOutput
0^ 1
^
k^
11 - 11 - 6565
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Derivation of Derivation of Parameter Parameter EquationsEquations
Goal: Minimize squared errorGoal: Minimize squared error
xnnyn
xy
xy
ii
iii
10
10
0
210
0
2
ˆˆ2
ˆˆ2
ˆˆˆ
ˆˆ
0
xy 10 ˆˆ
Derivation of Derivation of Parameter Parameter EquationsEquations
iii
iii
iii
xxyyx
xyx
xy
11
10
1
210
1
2
ˆˆ2
ˆˆ2
ˆˆˆ
ˆˆ
0
xx
xy
iiii
iiii
SS
SS
yyxxxxxx
yyxxxx
1
1
1
ˆ
ˆ
ˆ
11 - 11 - 6767
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Parameter Parameter Estimation Thinking Estimation Thinking
ChallengeChallengeYou’re an economist for the county You’re an economist for the county cooperative. You gather the following data:cooperative. You gather the following data:
Fertilizer (lb.)Fertilizer (lb.) Yield (lb.)Yield (lb.) 4 4 3.03.0 6 6 5.55.51010 6.56.51212 9.09.0
What is the What is the relationshiprelationship between fertilizer & crop yield?between fertilizer & crop yield?
© 1984-1994 T/Maker Co.
11 - 11 - 6868
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
02468
10
0 5 10 15
02468
10
0 5 10 15
Scattergram Scattergram Crop Yield vs. Crop Yield vs.
Fertilizer*Fertilizer*
Yield (lb.)Yield (lb.)Yield (lb.)Yield (lb.)
Fertilizer (lb.)Fertilizer (lb.)Fertilizer (lb.)Fertilizer (lb.)
11 - 11 - 6969
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Parameter Parameter Estimation Solution Estimation Solution
Table*Table*
Xi Yi Xi2 Yi
2 XiYi
4 3.0 16 9.00 12
6 5.5 36 30.25 33
10 6.5 100 42.25 65
12 9.0 144 81.00 108
32 24.0 296 162.50 218
Xi Yi Xi2 Yi
2 XiYi
4 3.0 16 9.00 12
6 5.5 36 30.25 33
10 6.5 100 42.25 65
12 9.0 144 81.00 108
32 24.0 296 162.50 218
11 - 11 - 7070
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Parameter Parameter Estimation Solution*Estimation Solution*
80.0865.06ˆˆ
65.0
432
296
42432
218ˆ
10
2
1
2
12
11
11
XY
n
X
X
n
YX
YX
n
i
n
ii
i
n
ii
n
iin
iii
80.0865.06ˆˆ
65.0
432
296
42432
218ˆ
10
2
1
2
12
11
11
XY
n
X
X
n
YX
YX
n
i
n
ii
i
n
ii
n
iin
iii
11 - 11 - 7171
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient Coefficient Interpretation Interpretation
Solution*Solution*
11 - 11 - 7272
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient Coefficient Interpretation Interpretation
Solution*Solution*
1.1. Slope (Slope (11)) Crop Yield (Crop Yield (YY) Is Expected to Increase ) Is Expected to Increase
by .65 lb. for Each 1 lb. Increase in Fertilizer by .65 lb. for Each 1 lb. Increase in Fertilizer ((XX))
^
11 - 11 - 7373
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient Coefficient Interpretation Interpretation
Solution*Solution*
1.1. Slope (Slope (11)) Crop Yield (Crop Yield (YY) Is Expected to Increase ) Is Expected to Increase
by .65 lb. for Each 1 lb. Increase in Fertilizer by .65 lb. for Each 1 lb. Increase in Fertilizer ((XX))
2.2. Y-Intercept (Y-Intercept (00)) Average Crop Yield (Average Crop Yield (YY) Is Expected to Be ) Is Expected to Be
0.8 lb. When No Fertilizer (0.8 lb. When No Fertilizer (XX) Is Used) Is Used
^
^
11 - 11 - 7474
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Probability Distribution Probability Distribution
of Random Errorof Random Error
11 - 11 - 7575
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Regression Modeling Regression Modeling Steps Steps
1.1. Hypothesize Deterministic ComponentHypothesize Deterministic Component
2.2. Estimate Unknown Model ParametersEstimate Unknown Model Parameters
3.3. Specify Probability Distribution of Specify Probability Distribution of Random Error TermRandom Error Term Estimate Standard Deviation of ErrorEstimate Standard Deviation of Error
4.4. Evaluate ModelEvaluate Model
5.5. Use Model for Prediction & Estimation Use Model for Prediction & Estimation
11 - 11 - 7676
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Linear Regression Linear Regression Assumptions Assumptions
1.1. Mean of Probability Distribution of Mean of Probability Distribution of Error Is 0Error Is 0
2.2. Probability Distribution of Error Has Probability Distribution of Error Has Constant VarianceConstant Variance1.1. Exercise: Constant across what?Exercise: Constant across what?
3.3. Probability Distribution of Error is Probability Distribution of Error is NormalNormal
4.4. Errors Are Independent Errors Are Independent
11 - 11 - 7777
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Error Error Probability Probability DistributionDistribution
Y
f()
X
X 1X 2
Y
f()
X
X 1X 2
^
11 - 11 - 7878
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Random Error Random Error VariationVariation
11 - 11 - 7979
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Random Error Random Error VariationVariation
1.1. Variation of Actual Variation of Actual YY from Predicted from Predicted YY
11 - 11 - 8080
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Random Error Random Error VariationVariation
1.1. Variation of Actual Variation of Actual YY from Predicted from Predicted YY
2.2. Measured by Standard Error of Measured by Standard Error of Regression ModelRegression Model Sample Standard Deviation of Sample Standard Deviation of , , ss^
11 - 11 - 8181
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Random Error Random Error VariationVariation
1.1. Variation of Actual Variation of Actual YY from Predicted from Predicted YY
2.2. Measured by Standard Error of Measured by Standard Error of Regression ModelRegression Model Sample Standard Deviation of Sample Standard Deviation of , , ss
3. 3. Affects Several FactorsAffects Several Factors Parameter SignificanceParameter Significance Prediction AccuracyPrediction Accuracy
^
11 - 11 - 8282
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Evaluating the ModelEvaluating the Model
Testing for SignificanceTesting for Significance
11 - 11 - 8383
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Regression Modeling Regression Modeling Steps Steps
1.1. Hypothesize Deterministic ComponentHypothesize Deterministic Component
2.2. Estimate Unknown Model ParametersEstimate Unknown Model Parameters
3.3. Specify Probability Distribution of Specify Probability Distribution of Random Error TermRandom Error Term Estimate Standard Deviation of ErrorEstimate Standard Deviation of Error
4.4. Evaluate ModelEvaluate Model
5.5. Use Model for Prediction & EstimationUse Model for Prediction & Estimation
11 - 11 - 8484
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Test of Slope Test of Slope CoefficientCoefficient
1.1. Shows If There Is a Linear Relationship Shows If There Is a Linear Relationship Between Between XX & & YY
2.2. Involves Population Slope Involves Population Slope 11
3.3. Hypotheses Hypotheses HH00: : 1 1 = 0 (No Linear Relationship) = 0 (No Linear Relationship)
HHaa: : 11 0 (Linear Relationship) 0 (Linear Relationship)
4.4. Theoretical Basis Is Sampling Distribution Theoretical Basis Is Sampling Distribution of Slopeof Slope
11 - 11 - 8585
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Sampling Sampling Distribution Distribution
of Sample Slopesof Sample Slopes
11 - 11 - 8686
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Y
Population LineX
Sample 1 Line
Sample 2 Line
Y
Population LineX
Sample 1 Line
Sample 2 Line
Sampling Sampling Distribution Distribution
of Sample Slopesof Sample Slopes
11 - 11 - 8787
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Y
Population LineX
Sample 1 Line
Sample 2 Line
Y
Population LineX
Sample 1 Line
Sample 2 Line
Sampling Sampling Distribution Distribution
of Sample Slopesof Sample Slopes
All Possible All Possible Sample SlopesSample Slopes
Sample 1:Sample 1: 2.52.5
Sample 2:Sample 2: 1.6 1.6
Sample 3:Sample 3: 1.81.8
Sample 4:Sample 4: 2.12.1 : : : :Very large number of Very large number of sample slopessample slopes
11 - 11 - 8888
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Y
Population LineX
Sample 1 Line
Sample 2 Line
Y
Population LineX
Sample 1 Line
Sample 2 Line
Sampling Sampling Distribution Distribution
of Sample Slopesof Sample Slopes
11
All Possible All Possible Sample SlopesSample Slopes
Sample 1:Sample 1: 2.52.5
Sample 2:Sample 2: 1.6 1.6
Sample 3:Sample 3: 1.81.8
Sample 4:Sample 4: 2.12.1 : : : :Very large number of Very large number of sample slopessample slopes
Sampling DistributionSampling Distribution
11
11SS
^
^
11 - 11 - 8989
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Slope Coefficient Slope Coefficient Test StatisticTest Statistic
n
X
X
SS
St
n
iin
ii
n
2
1
1
2
ˆ
ˆ
112
1
1
where
ˆ
n
X
X
SS
St
n
iin
ii
n
2
1
1
2
ˆ
ˆ
112
1
1
where
ˆ
11 - 11 - 9090
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Test of Slope Test of Slope Coefficient ExampleCoefficient Example
You’re a marketing analyst for Hasbro Toys. You’re a marketing analyst for Hasbro Toys. You find You find bb00 = -.1 = -.1,, bb11 = .7 = .7 & & ss = .60553= .60553..
Ad $Ad $ Sales (Units)Sales (Units)11 1122 1133 2244 2255 44
Is the relationship Is the relationship significantsignificant at the at the .05.05 level? level?
11 - 11 - 9191
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Solution TableSolution Table
Xi Yi Xi2 Yi
2 XiYi
1 1 1 1 1
2 1 4 1 2
3 2 9 4 6
4 2 16 4 8
5 4 25 16 20
15 10 55 26 37
Xi Yi Xi2 Yi
2 XiYi
1 1 1 1 1
2 1 4 1 2
3 2 9 4 6
4 2 16 4 8
5 4 25 16 20
15 10 55 26 37
11 - 11 - 9292
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Test of Slope Test of Slope Parameter Parameter
SolutionSolutionHH00: : 11 = 0 = 0
HHaa: : 11 0 0
.05.05
df df 5 - 2 = 35 - 2 = 3
Critical Value(s):Critical Value(s):
Test Statistic: Test Statistic:
Decision:Decision:
Conclusion:Conclusion:
t0 3.1824-3.1824
.025
Reject Reject
.025
t0 3.1824-3.1824
.025
Reject Reject
.025
tS
.
..
1 1
1
0 70 001915
3 656tS
.
..
1 1
1
0 70 001915
3 656
Reject at Reject at = .05 = .05
There is evidence of a There is evidence of a relationshiprelationship
11 - 11 - 9393
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Test StatisticTest StatisticSolutionSolution
1915.0
515
55
60553.0
where
656.31915.0
070.0ˆ
32
1
1
2
ˆ
ˆ
112
1
1
n
X
X
SS
St
n
iin
ii
n
1915.0
515
55
60553.0
where
656.31915.0
070.0ˆ
32
1
1
2
ˆ
ˆ
112
1
1
n
X
X
SS
St
n
iin
ii
n
11 - 11 - 9494
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Test of Slope Test of Slope ParameterParameter
Computer OutputComputer Output Parameter EstimatesParameter Estimates
Parameter Standard Parameter Standard T for H0:T for H0:
VariableVariable DF Estimate Error DF Estimate Error Param=0 Prob>|T|Param=0 Prob>|T|
INTERCEP 1 -0.1000 0.6350 -0.157 0.8849INTERCEP 1 -0.1000 0.6350 -0.157 0.8849
ADVERTADVERT 1 0.7000 0.1914 1 0.7000 0.1914 3.6563.656 0.03540.0354
t = k / S
P-Value
Skk k
^^^^
11 - 11 - 9595
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Measures of Measures of Variation Variation
in Regression in Regression 1.1. Total Sum of Squares (SSTotal Sum of Squares (SSyyyy))
Measures Variation of Observed Measures Variation of Observed YYii Around the MeanAround the MeanYY
2.2. Explained Variation (SSR)Explained Variation (SSR) Variation Due to Relationship Between Variation Due to Relationship Between
XX & & YY
3.3. Unexplained VariationUnexplained Variation (SSE) (SSE) Variation Due to Other FactorsVariation Due to Other Factors
11 - 11 - 9696
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Y
X
Y
X i
Y
X
Y
X i
Variation MeasuresVariation Measures
Y Xi i 0 1 Y Xi i 0 1
Total sum Total sum
of squares of squares
(Y(Yii - -Y)Y)22
Unexplained sum Unexplained sum
of squares (Yof squares (Yii - -
YYii))22
^
Explained sum of Explained sum of
squares (Ysquares (Yii - -Y)Y)22 ^
YYii
11 - 11 - 9797
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
1.1. ProportionProportion of Variation ‘Explained’ by of Variation ‘Explained’ by Relationship Between Relationship Between XX & & YY
Coefficient of Coefficient of DeterminationDetermination
n
ii
n
ii
n
ii
YY
YYYY
r
1
2
1
2
1
2
2
ˆ
Variation Total
Variation Explained
n
ii
n
ii
n
ii
YY
YYYY
r
1
2
1
2
1
2
2
ˆ
Variation Total
Variation Explained
0 r2 1
11 - 11 - 9898
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Y
X
Y
X
Y
X
Coefficient of Coefficient of Determination Determination
ExamplesExamplesY
X
r2 = 1 r2 = 1
r2 = .8 r2 = 0
11 - 11 - 9999
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient of Coefficient of Determination Determination
ExampleExampleYou’re a marketing analyst for Hasbro You’re a marketing analyst for Hasbro
Toys. You find Toys. You find 00 = -0.1 & = -0.1 & 11 = 0.7. = 0.7.
Ad $Ad $ Sales (Units)Sales (Units)11 1122 1133 2244 2255 44
Interpret a Interpret a coefficient of coefficient of determination determination ofof 0.8167.0.8167.
^^
11 - 11 - 100100
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
r r 22 Computer Output Computer Output
Root MSE 0.60553Root MSE 0.60553 R-square 0.8167R-square 0.8167
Dep Mean 2.00000 Dep Mean 2.00000 Adj R-sq 0.7556Adj R-sq 0.7556
C.V. 30.27650 C.V. 30.27650
r2 adjusted for number of explanatory variables & sample size
S
r2
11 - 11 - 101101
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Using the Model for Using the Model for Prediction & EstimationPrediction & Estimation
11 - 11 - 102102
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Regression Modeling Regression Modeling Steps Steps
1.1. Hypothesize Deterministic ComponentHypothesize Deterministic Component
2.2. Estimate Unknown Model ParametersEstimate Unknown Model Parameters
3.3. Specify Probability Distribution of Specify Probability Distribution of Random Error TermRandom Error Term Estimate Standard Deviation of ErrorEstimate Standard Deviation of Error
4.4. Evaluate ModelEvaluate Model
5.5. Use Model for Prediction & Estimation Use Model for Prediction & Estimation
11 - 11 - 103103
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Prediction With Prediction With Regression ModelsRegression Models
1.1. Types of PredictionsTypes of Predictions Point EstimatesPoint Estimates Interval EstimatesInterval Estimates
2.2. What Is PredictedWhat Is Predicted Population Mean Response Population Mean Response EE((YY) for ) for
Given Given XX Point on Population Regression LinePoint on Population Regression Line
Individual Response (Individual Response (YYii) for Given ) for Given XX
11 - 11 - 104104
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
What Is PredictedWhat Is Predicted
Mean Y, E(Y)
Y
Y i= 0
+ 1X
^Y Individual
Prediction, Y
E(Y) = 0 + 1X
^
XXP
^^
Mean Y, E(Y)
Y
Y i= 0
+ 1X
^Y Individual
Prediction, Y
E(Y) = 0 + 1X
^
XXP
^^
11 - 11 - 105105
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
ConfidenceConfidence Interval Interval Estimate of Mean Estimate of Mean YY
n
ii
p
Y
YnYn
XX
XX
nSS
StYYEStY
1
2
2
ˆ
ˆ2/,2ˆ2/,2
1
where
ˆ)(ˆ
n
ii
p
Y
YnYn
XX
XX
nSS
StYYEStY
1
2
2
ˆ
ˆ2/,2ˆ2/,2
1
where
ˆ)(ˆ
11 - 11 - 106106
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Factors Affecting Factors Affecting Interval WidthInterval Width
1.1. Level of Confidence (1 - Level of Confidence (1 - )) Width Increases as Confidence IncreasesWidth Increases as Confidence Increases
2.2. Data Dispersion (Data Dispersion (ss)) Width Increases as Variation IncreasesWidth Increases as Variation Increases
3.3. Sample SizeSample Size Width Decreases as Sample Size IncreasesWidth Decreases as Sample Size Increases
4.4. Distance of Distance of XXpp from Mean from MeanXX Width Increases as Distance IncreasesWidth Increases as Distance Increases
11 - 11 - 107107
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Why Distance from Why Distance from Mean?Mean?
Sample 2 Line
Y
XX1 X2
Y_ Sample 1 Line
Sample 2 Line
Y
XX1 X2
Y_ Sample 1 Line
Greater Greater dispersion dispersion than than XX11
XX
11 - 11 - 108108
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
ConfidenceConfidence Interval Interval Estimate ExampleEstimate Example
You’re a marketing analyst for Hasbro Toys. You’re a marketing analyst for Hasbro Toys. You find You find bb00 = -.1 = -.1,, bb11 = .7 = .7 & & ss = .60553= .60553..
Ad $Ad $ Sales (Units)Sales (Units)11 1122 1133 2244 2255 44
Estimate the Estimate the meanmean sales when sales when advertising is advertising is $4$4 at the at the .05.05 level. level.
11 - 11 - 109109
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Solution TableSolution Table
Xi Yi Xi2 Yi
2 XiYi
1 1 1 1 1
2 1 4 1 2
3 2 9 4 6
4 2 16 4 8
5 4 25 16 20
15 10 55 26 37
Xi Yi Xi2 Yi
2 XiYi
1 1 1 1 1
2 1 4 1 2
3 2 9 4 6
4 2 16 4 8
5 4 25 16 20
15 10 55 26 37
11 - 11 - 110110
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
ConfidenceConfidence Interval Interval Estimate SolutionEstimate Solution
7553.3)(6445.1
3316.01824.37.2)(3316.01824.37.2
3316.010
34
5
160553.
7.247.01.0ˆ
ˆ)(ˆ
2
ˆ
ˆ2/,2ˆ2/,2
YE
YE
S
Y
StYYEStY
Y
YnYn
7553.3)(6445.1
3316.01824.37.2)(3316.01824.37.2
3316.010
34
5
160553.
7.247.01.0ˆ
ˆ)(ˆ
2
ˆ
ˆ2/,2ˆ2/,2
YE
YE
S
Y
StYYEStY
Y
YnYn
XX to be predicted to be predictedXX to be predicted to be predicted
11 - 11 - 111111
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
n
ii
PYY
YYnPYYn
XX
XX
nSS
StYYStY
1
2
2
ˆ
ˆ2/,2ˆ2/,2
11
where
ˆˆ
n
ii
PYY
YYnPYYn
XX
XX
nSS
StYYStY
1
2
2
ˆ
ˆ2/,2ˆ2/,2
11
where
ˆˆ
PredictionPrediction Interval Interval of Individual of Individual
ResponseResponse
Note!Note!
11 - 11 - 112112
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Why the Extra ‘SWhy the Extra ‘S’’??
Expected(Mean) Y
Y
Y i= 0
+ 1X i
^
Y we're trying to predict
Prediction, Y
E(Y) = 0 + 1X
^
XXP
^
^Expected(Mean) Y
Y
Y i= 0
+ 1X i
^
Y we're trying to predict
Prediction, Y
E(Y) = 0 + 1X
^
XXP
^
^
11 - 11 - 113113
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Interval Estimate Interval Estimate Computer OutputComputer Output
Dep Var Pred Std Err Dep Var Pred Std Err Low95% Upp95% Low95% Upp95%Low95% Upp95% Low95% Upp95%
Obs SALES Value Predict Obs SALES Value Predict Mean Mean Predict PredictMean Mean Predict Predict
1 1.000 0.600 0.469 -0.892 2.092 -1.837 3.037 1 1.000 0.600 0.469 -0.892 2.092 -1.837 3.037
2 1.000 1.300 0.332 0.244 2.355 -0.897 3.4972 1.000 1.300 0.332 0.244 2.355 -0.897 3.497
3 2.000 2.000 0.271 1.138 2.861 -0.111 4.1113 2.000 2.000 0.271 1.138 2.861 -0.111 4.111
4 2.000 4 2.000 2.700 0.332 1.644 3.755 0.502 4.897 2.700 0.332 1.644 3.755 0.502 4.897
5 4.000 3.400 0.469 1.907 4.892 0.962 5.8375 4.000 3.400 0.469 1.907 4.892 0.962 5.837
Predicted Predicted YY when when XX = 4 = 4
Confidence Confidence IntervalInterval
SSYYPrediction Prediction IntervalInterval
11 - 11 - 114114
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Hyperbolic Interval Hyperbolic Interval BandsBands
X
Y
X
Y i= 0
+ 1X i
^
XP
_
^^
X
Y
X
Y i= 0
+ 1X i
^
XP
_
^^
11 - 11 - 115115
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Correlation ModelsCorrelation Models
11 - 11 - 116116
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Types of Types of Probabilistic ModelsProbabilistic Models
ProbabilisticModels
RegressionModels
CorrelationModels
OtherModels
ProbabilisticModels
RegressionModels
CorrelationModels
OtherModels
11 - 11 - 117117
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Correlation ModelsCorrelation Models
1.1. Answer ‘Answer ‘How Strong How Strong Is the Linear Is the Linear Relationship Between 2 Variables?’Relationship Between 2 Variables?’
2.2. Coefficient of Correlation UsedCoefficient of Correlation Used Population Correlation Coefficient Denoted Population Correlation Coefficient Denoted
(Rho) (Rho) Values Range from -1 to +1Values Range from -1 to +1 Measures Degree of AssociationMeasures Degree of Association
3.3. Used Mainly for UnderstandingUsed Mainly for Understanding
11 - 11 - 118118
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
1.1. Pearson Product Moment Coefficient of Pearson Product Moment Coefficient of Correlation, Correlation, rr::
Sample Coefficient Sample Coefficient of Correlationof Correlation
n
ii
n
ii
n
iii
YYXX
YYXX
r
1
2
1
2
1
ionDeterminat oft Coefficien
n
ii
n
ii
n
iii
YYXX
YYXX
r
1
2
1
2
1
ionDeterminat oft Coefficien
11 - 11 - 119119
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient of Correlation Coefficient of Correlation ValuesValues
-1.0-1.0 +1.0+1.000-.5-.5 +.5+.5
11 - 11 - 120120
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient of Correlation Coefficient of Correlation ValuesValues
-1.0-1.0 +1.0+1.000-.5-.5 +.5+.5
No No CorrelationCorrelation
11 - 11 - 121121
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient of Correlation Coefficient of Correlation ValuesValues
-1.0-1.0 +1.0+1.000
Increasing degree of Increasing degree of negative correlationnegative correlation
-.5-.5 +.5+.5
No No CorrelationCorrelation
11 - 11 - 122122
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient of Correlation Coefficient of Correlation ValuesValues
-1.0-1.0 +1.0+1.000-.5-.5 +.5+.5
Perfect Perfect Negative Negative
CorrelationCorrelationNo No
CorrelationCorrelation
11 - 11 - 123123
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient of Correlation Coefficient of Correlation ValuesValues
-1.0-1.0 +1.0+1.000-.5-.5 +.5+.5
Perfect Perfect Negative Negative
CorrelationCorrelationNo No
CorrelationCorrelation
Increasing degree of Increasing degree of positive correlationpositive correlation
11 - 11 - 124124
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient of Correlation Coefficient of Correlation ValuesValues
-1.0-1.0 +1.0+1.000
Perfect Perfect Positive Positive
CorrelationCorrelation
-.5-.5 +.5+.5
Perfect Perfect Negative Negative
CorrelationCorrelationNo No
CorrelationCorrelation
11 - 11 - 125125
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Coefficient of Coefficient of CorrelationCorrelation ExamplesExamples
Y
X
Y
X
Y
X
Y
X
r = 1 r = -1
r = .89 r = 0
11 - 11 - 126126
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
Test of Test of Coefficient of Coefficient of Correlation Correlation
1.1. Shows If There Is a Linear Relationship Shows If There Is a Linear Relationship Between 2 Numerical VariablesBetween 2 Numerical Variables
2.2. Same Conclusion as Testing Same Conclusion as Testing Population Slope Population Slope 11
3.3. Hypotheses Hypotheses HH00: : = 0 (No Correlation) = 0 (No Correlation)
HHaa: : 0 (Correlation) 0 (Correlation)
11 - 11 - 127127
© 2003 Pearson Prentice Hall© 2003 Pearson Prentice Hall
ConclusionConclusion
1.1. Described the Linear Regression ModelDescribed the Linear Regression Model
2.2. Stated the Regression Modeling StepsStated the Regression Modeling Steps
3.3. Explained Ordinary Least SquaresExplained Ordinary Least Squares
4.4. Computed Regression CoefficientsComputed Regression Coefficients
5.5. Predicted Response VariablePredicted Response Variable
6.6. Interpreted Computer OutputInterpreted Computer Output
End of Chapter
Any blank slides that follow are blank intentionally.