Regression analysis

REGRESSION ANALYSIS

-P H SHAMEER

• An introduction to regression model

• Performing it on SPSS

INTRODUCTION

• What is regression model?An explanatory methodForecast expressed as a function of a

certain no. of variables that influences its outcome

2 types of variables

1. DEPENDENT

-which we want to forecast

2. INDEPENDENT

-or predictor variables

• Eg:

• Predict how much an individual enjoys his/her job

• Dependent variable: job satisfaction

• Independent variables:

salary, academic qualification, age, sex,

no. of years, socio-economic status….

assumptions

1. LINEAR RELATIONSHIP exists

2. HOMOSCEDASTICITY exists

3. Residuals are INDEPENDENT of one another

4. MULTICOLLINEARITY doesn’t exist

Analysis for Linearity

Not Linear Linear

x x

Y

x

Y

x

Residual Analysis for Homoscedasticity

Non-constant variance Constant variance

x x

Y

x x

Y

resi

dua

ls

resi

dua

ls

SCATTER PLOTS• -helps to visualize, graphically the

relationship between pairs of variables

Regression Equation

where

a is y intercept

&

b1, b2,..bi are regression coefficients

1 1 2 2' i iy a b x b x b x

How a & b can be calculated?

• Method of least squares

this method determines the values in such a way that the sum of squared deviations (errors) is minimized

and hence the name least squares

b=(∑x*y/n) ─ (x * y)

( ∑x2 / n) ─ (x)2

a = y- bx where y = ∑y/n

x= ∑x/n

n is the no. of observations

forecasting

• Once the relationship is determined , it can be used to make any no. of forecasts simply by inserting the values of X’s

• y = a+b1x1+b2x2+…+bixi

• Caution: the basic relationship should be assessed periodically

terminology

b - standard regression coefficient: Measure of how strongly each predictor

variable influences the dependent variableE.g.: if b=2.5

change of one standard deviation in the predictor will change 2.5 standard deviations in the forecasting variable

terminology

RMeasure of correlation between observed

& predicted value of the dependent variable

R -1 t0 1R= n*∑xi*yi-∑xi*∑yi

√(n∑xi2- (∑xi)2) √(n∑yi

2- (∑yi)2)

Scatter Plots of Data with Various Correlation Coefficients

Y

X

Y

X

Y

X

Y

X

Y

X

r = -1 r = -.6 r = 0

r = +.3r = +1

Y

Xr = 0

Slide from: Statistics for Managers Using Microsoft® Excel 4th Edition, 2004 Prentice-Hall

terminology…..

R2

variation in Y accounted for by the set of predictors

Measure of how good a forecasting of dep. variable by knowing the independent variables.

When applied to reality, R2 over estimate the success

terminology…Adjusted R2

The adjustment takes into account the size of the sample and number of predictors

Gives most useful measure of success of our model ( goodness of fit)

R2 range:0 to 1.If R2=0.75, success will be 75%

Is each X contributing to the prediction of Y?

• Test if each regression coefficient is significantly different than zero given the variables standard error.

– T-test for each regression coefficient

Performing regression in spss

Eg:importance of several psycholinguistic variables on spelling performance

variables

Independent:

standardized spelling score(spellsc), chronological age(age), reading age(readage), standardized reading score(standsc)

Dependent variable:

percentage correct spelling(spelperc)

Performing regression in spss

• SPPS=Statistical Packages in Social Sciences

Enter the data

Cont..

>Analyze>regression> lineardialogue box appears

now enter dependent and independent variables

Selection methods:on relative contribution of independent

variables

1. simultaneous/ enter method

2. Hierarchical method

3. Statistical methods

a. Forward

b. Backward

c. Stepwise

d. Remove

Now click the statistics button

Now click ‘continue’> then ‘ok’

Output:

Cont…

• Here reading age is not a significant predictor

result:

percentage correct spelling=

-232+.406*chronological age

+.394*standardized reading score

+.786*standardized spelling score

references

Forecasting methods for management

by Spyros Makridas & Steven C Wheelwright

SPSS for psychologists

by Nicola Brace, Richard Kemp & Rosemary Snelger

Research Methods for M.Com

by L.R Potti

THANKYOU…