Econometrics I

Part 4: Partial Regression and Correlation4-1/22

Econometrics I Professor William GreeneStern School of Business

Department of Economics

Part 4: Partial Regression and Correlation4-2/22

Econometrics I

Part 4 – Partial Regression and Correlation

Part 4: Partial Regression and Correlation4-3/22

Frisch-Waugh (1933) Theorem

Context: Model contains two sets of variables: X = [ (1,time) : (other variables)] = [X1 X2] Regression model: y = X11 + X22 + (population) = X1b1 + X2b2 + e (sample)Problem: Algebraic expression for the second set

of least squares coefficients, b2

Part 4: Partial Regression and Correlation4-4/22

Partitioned SolutionDirect manipulation of normal equations produces

, ] so and

( ) =

1 1 1 2 11 2

2 1 2 2 2

1 1 1 2 1 1

2 1 2 2 2 2

1 1 1 1 2 2 1

2 1 1 2 2 2 2 2 2 2 2 2 1 1

X X b X yX X X X X y

X = [X X X X = X y =X X X X X y

X X X X b X y(X X)b = =

X X X X b X yX X b X X b X yX X b X X b X y ==> X X b X y - X X b = 2 1 1X (y - X b )

Part 4: Partial Regression and Correlation4-5/22

Partitioned Solution

Direct manipulation of normal equations produces b2 = (X2X2)-1X2(y - X1b1) What is this? Regression of (y - X1b1) on X2

If we knew b1, this is the solution for b2.Important result (perhaps not fundamental). Note

the result if X2X1 = 0.

Part 4: Partial Regression and Correlation4-6/22

Partitioned Inverse Use of the partitioned inverse result

produces a fundamental result: What is the southeast element in the inverse of the moment matrix?

1 1 1 2

2 1 2 2

-1X 'X X 'XX 'X X 'X

Part 4: Partial Regression and Correlation4-7/22

Partitioned Inverse The algebraic result is: [ ]-1

(2,2) = {[X2’X2] - X2’X1(X1’X1)-1X1’X2}-1

= [X2’(I - X1(X1’X1)-1X1’)X2]-1

= [X2’M1X2]-1 Note the appearance of an “M” matrix.

How do we interpret this result?

Part 4: Partial Regression and Correlation4-8/22

Frisch-Waugh ResultContinuing the algebraic manipulation:

b2 = [X2’M1X2]-1[X2’M1y].

This is Frisch and Waugh’s famous result - the “double residual regression.”

How do we interpret this? A regression of residuals on residuals.

Regrido Y em X1 – pego o resíduo que sobra e regrido esta parte no que sobra de X2 regredido em X1 (resíduo).

Tudo de Y que não foi explicado por X1 mas pode ser explicado por X2.

Tudo de X2 que não está sendo explicado por X1

Part 4: Partial Regression and Correlation4-9/22

Part 4: Partial Regression and Correlation4-10/22

Important Implications M1 is idempotent. (This is a very useful result.)

(i.e., X2M1’M1y = X2M1y)

(Orthogonal regression) Suppose X1 and X2 are orthogonal; X1X2 = 0. What is M1X2?

Part 4: Partial Regression and Correlation4-11/22

Applying Frisch-Waugh

Using gasoline data from Notes 3.X = [1, year, PG, Y], y = G as before.

Full least squares regression of y on X.

Part 4: Partial Regression and Correlation4-12/22

Detrending the Variables - Pg

Part 4: Partial Regression and Correlation4-13/22

Regression of Detrended G on Detrended Pg and Detrended Y

Part 4: Partial Regression and Correlation4-14/22

Partial RegressionImportant terms in this context: Partialing out the effect of X1. Netting out the effect …

“Partial regression coefficients.” To continue belaboring the point: Note the interpretation of partial

regression as “net of the effect of …”

Now, follow this through for the case in which X1 is just a constant term, column of ones.

What are the residuals in a regression on a constant. What is M1? Note that this produces the result that we can do linear regression on data

in mean deviation form.As inclinações numa regressão múltipla que contém o termo

constante são obtidas por transformar os dados como desvios em relação à média e regredir a variável y na forma de desvio em todas variáveis explicativas também como desvios em relação às suas médias.

'Partial regression coefficients' are the same as 'multiple regression coefficients.' It follows from the Frisch-Waugh theorem.

Part 4: Partial Regression and Correlation4-15/22

Partial CorrelationWorking definition. Correlation between sets of residuals. Some results on computation: Based on the M matrices. Some important considerations: Partial correlations and coefficients can have signs and magnitudes that differ greatly from gross correlations and

simple regression coefficients.

Compare the simple (gross) correlation of G and PG with the partial correlation, net of the time effect.

CALC;list;Cor(g,pg)$ Result = .7696572

CALC;list;cor(gstar,pgstar)$ Result = -.6589938

G

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

.5080 90 100 110 12070

PG

Part 4: Partial Regression and Correlation4-16/22

Partial Correlation

Part 4: Partial Regression and Correlation4-17/22

THE Application of Frisch-Waugh The Fixed Effects Model

A regression model with a dummy variable foreach individual in the sample, each observed Ti times.yi = Xi + diαi + εi, for each individual

1 1

2 2

N

=

=

N

1

2

N

y X d 0 0 0y X 0 d 0 0 β ε

αy X 0 0 0 d

β [X,D] εα Zδ ε

N may be thousands. I.e., the regression has thousands of variables (coefficients).

N columns

Part 4: Partial Regression and Correlation4-18/22

Estimating the Fixed Effects Model The FEM is a linear regression model but

with many independent variables

1

1Using the Frisch-Waugh theorem

=[ ]

D D

b XX XD Xya DX DD Dy

b XM X XM y

Part 4: Partial Regression and Correlation4-19/22

Fixed Effects Estimator (cont.)

i i i

i i i

i i i

1i

1 1 1T T T

1 1 1T T T

1 1 1T T T

(The dummy variables are orthogonal)

( ) (1/T)1- - ... -- 1- ... -... ... ... ...- - ... 1-

i i

1D

2D

D

ND

iD T i i i T i

M 0 00 M 0M

0 0 MM I d dd d =I dd

=

Part 4: Partial Regression and Correlation4-20/22

‘Within’ Transformations

i

i

Ti i i i t=1 it,k i.,k it,l i.,lk,l

Ti i i i t=1 it,k i.,k it i.k

, (x -x )(x -x )

, (x -x )(y -y )

N i iD i=1 D D

N i iD i=1 D D

XM X= XM X XM X

XM y= XM y XM y

Least Squares Dummy Variable Estimator

b is obtained by ‘within’ groups least squares (group mean deviations)