Agenda of Week VII Review of Week VI Multiple regression Canonical correlation.

Agenda of Week VII

Review of Week VI

Multiple regression

Canonical correlation

Review of Week VI

Inference on MV Mean Vector Two populations

Inference on MV Mean Vector Multi-populations: MANOVA

Multiple Regression

Basic assumptions Linearity

X is an n by K matrix with rank K

Regression

Homoscedasticity

y X

[ | ] 0E X

2[ ] [ ' | ]Var E X I

Multiple Regression

Hypotheses

0 : 0 . : 0aH vs H

0 : 0 . : 0i a iH vs H

By ANOVA

When the null hypothesis is rejected,

For all i.

Multiple Regression

' ( ) '( )

' ' ' ' ' '

' 2 ' ' '

Min y X y X

y y X y y X X X

y y X y X X

1

'2 ' 2 ' 0

( ' ) '

X y X X

X X X y

OLS model

OLS solution

Multiple RegressionANOVA table

Source Sum of squares d.f. Mean square F

Regression 2' 'b X y ny k- 1 2' '

1

b X y ny

k

Residual 'e e n- k 'e e

n k

Total 2'y y ny n- 1 2'

1

y y ny

n

2' '1

'

b X y nyk

e en k

Coefficient of

determination

22

'1

'

e eR

y y ny

Multiple Regression

2 2 1'. [ ] ( ' )

e es Est V s X X

n k

2

kk kkt

s S

t-statistics (d.f. = n-k)

Where Skk is the kth diagonal element of (X’X)-1

Standard error of the regression

Multiple Regression

2

2

kkt s S

Standard error of coefficient

Canonical Correlation

Correlation structure between a group of independent variables and a group of dependent variables A kind of multiple regression with more than two

dependent variables Example: Physical size group vs. Exercise group How much explained the variation of a group of

variables by other group of variables?

Canonical Correlation

Basic model

1

.

.i

p

X

X X

X

Group of independent variables

1

.

.i

q

Y

Y Y

Y

Group of dependent variables

( ) ( ), ( ) ( )

( , ) ( )XX XX YY YY

XY XY

Cov X S Cov Y S

Cov X Y S

Covariance matrixes

Canonical Correlation

Basic model

1 1

1 1

' ......

' ......

i i ip p

i i iq q

V X X X

W Y Y Y

LC of two groups of variables

' '( , ) ( ( , ) )

' ' ' 'XY XY

XX YY XX YY

SV W r V W

S S

Correlation between V and W

Canonical Correlation

Correlation testing

0 : 0 : 0xy a xyH vs H Hypotheses

22

1

3

1, 12 3

2 2

2 2

1

1

( 1)2

4( )1.5 ,2 5

p qp q ms

Wilks r

pqF

pqms

p qp qm n sp q

Correlation

between V and W

Canonical Correlation

,( , ) ( , )

. .

( ) ( ) 0,

( ) ( ) 1

Max V W or r V W

s t

E V E W

V V V W

Objective: To find α and β maximizing ρ or r.

Canonical Correlation

1 1' '2 2

1 1 1 1,XX YYV e S X W f S Y

1 1' '2 21 1' , 'XX YYe S f S

1st canonical variable

kth canonical variable

1 1' '2 2,k k XX k k YYV e S X W f S Y

Canonical Correlation

1 1 1( , )r r V W

1st canonical correlation coefficient

kth canonical coefficient coefficient

( , )k k kr r V W

Canonical Correlation

1 112 2

1 112 2

,

k

XX XY YY YX XX

k

YY YX XX XY YY

e eigen vector of kth eigen value of

S S S S S

f eigen vector of kth eigen value of

S S S S S

Canonical Correlation

( , ) , ( , )XX X YY Yr V X AS D r W Y BS D

Canonical loading matrix

Canonical cross loading matrix

( , ) , ( , )XY Y YX Xr V Y AS D r W X BS D