G89.2247 Lect 21 G89.2247 Lecture 2 Regression as paths and covariance structure Alternative “saturated” path models Using matrix notation to write linear.

G89.2247 Lect 2 1

G89.2247Lecture 2

• Regression as paths and covariance structure

• Alternative “saturated” path models

• Using matrix notation to write linear models

• Multivariate Expectations

• Mediation

G89.2247 Lect 2 2

Question: Does exposure to childhood foster care (X) lead to adverse outcomes (Y) ?

• Example of purported "causal model" X Y

Y = B0 + B1X + e• Regression approach

B0 and B1 can be estimated using OLS Estimates depend on sample standard deviations of Y and X, sample means,

and covariance between Y and X• B1 = SXY/S2

X

• B0 = MY -B1MX

Correlation, rXY = SXY/SXSY, can be used to estimate the variance of the residual, e, V(e).

• S2e = S2

Y(1-r2XY) = S2

Y - S2XY/S2

X

eB1

G89.2247 Lect 2 3

A Covariance Structure Approach

• If we have data on Y and X we can compute a covariance matrix

• This estimates the population covariance structure,

Y can itself be expressed as B2

12X + 2

e

Three statistics in the sample covariance matrix are available to estimate three population parameters

2

2

XXY

YXY

SS

SSS

221

2

2`

2

XX

Y

XXY

YXY

B

G89.2247 Lect 2 4

Covariance Structure Approach, Continued

• A structural model that has the same number of parameters as unique elements in the covariance matrix is "saturated".

• Saturated models always fit the sample covariance matrix.

221

21

2221

XX

XeX

B

BB

G89.2247 Lect 2 5

Another saturated model: Two explanatory variables

• The first model is likely not to yield an unbiased estimate of foster care because of selection factors (Isolation failure).

• Suppose we have a measure of family disorganization (Z) that is known to have an independent effect on Y and also to be related to who is assigned to foster care (X)

YX

Z

e

XZ

G89.2247 Lect 2 6

Covariance Structure Expression

• The model: Y=b0+b1X+b2Z+e If we assume E(X)=E(Z)=E(Y)=0 and V(X) = V(Z) = V(Y) = 1 then b0=0 and 's are standardized

• The parameters can be expressed

When sample correlations are substituted, these expressions give the OLS estimates of the regression coefficients.

211 XZ

XZYZYX

22

1 XZ

XZYXYZ

G89.2247 Lect 2 7

Covariance Structure: 2 Explanatory Variables

• In the standardized case the covariance structure is:

• Each correlation is accounted by two components, one direct and one indirect

• There are three regression parameters and three covariances.

1

1

1

1

1

1

12

21

XZXZ

XZ

XZZY

XZXY

ZYXY

G89.2247 Lect 2 8

The more general covariance matrix for two IV multiple regression

• If we do not assume variances of unity the regression model implies

212

221

2

12

21

1

1

1

ZYXZZYXZZ

XYXZX

Y

XZXZ

XZ DD

Z

X

Y

V

G89.2247 Lect 2 9

More Math Review for SEM

• Matrix notation is useful

Z

X

Y

XZXZ

XZXZ

XZXZ

Z

X

Y

Z

X

Y

V

00

00

00

1

1

1

00

00

00

12

21

1221

212

221

2

ZYXZZYXZZ

XYXZX

Y

G89.2247 Lect 2 10

A Matrix Derivation of OLS Regression

• OLS regression estimates make the sum of squared residuals as small as possible.If Model is

Then we choose B so that e'e is minimized.

• The minimum will occur when the residual vector is orthogonal to the regression planeIn that case, X'e = 0

eBXY

G89.2247 Lect 2 11

When will X'e = 0? When e is the residual from an OLS fit.

BXXYXBXYXeX ˆ''ˆ'0'

YXXXB

YXXXBXXXX

YXBXX

''ˆ

''ˆ''

'ˆ'

1

11

G89.2247 Lect 2 12

Multivariate Expectations

• There are simple multivariate generalizations of the expectation facts:E(X+k) = E(X)+k = x+kE(k*X) = k*E(X) = k*x

V(X+k) = V(X) = x2

V(k*X) = k2*V(X) = k2*x2

• Let XT=[X1 X2 X3 X4], T=[] and let k be scalar valueE(k*X) = k*E(X) = k*E(X+k* 1) = {E(X) + k* 1} = + k*1

G89.2247 Lect 2 13

Multivariate Expectations

• In the multivariate case Var(X) is a matrixV(X)=E[(X-) (X-)T]

24434241

34233231

24232221

14131221

G89.2247 Lect 2 14

Multivariate Expectations

• The multivariate generalizations of V(X+k) = V(X) = x

2

V(k*X) = k2*V(X) = k2*x2

• Are: Var(X + k*1) = Var(k* X) = k2

• Let cT = [c1 c2 c3 c4]; cT X is a linear combination of the X's. Var(cT X) = cT c

• This is a scalar value• If this positive for all values of c then is positive

definite

G89.2247 Lect 2 15

Semi Partial Regression Adjustment

• The multiple regression coefficients are estimated taking all variables into accountThe model assumes that for fixed X, Z has an effect

of magnitude Z.Sometimes people say "controlling for X"

• The model explicitly notes that Z has two kinds of association with YA direct association through Z (X fixed)An indirect association through X (magnitude

XXZ)

G89.2247 Lect 2 16

Pondering Model 1:Simple Multiple Regression

• The semi-partial regression coefficients are often different from the bivariate correlationsAdjustment effectsSuppression effects

• Randomization makes XZ = 0 in probability.

YX

Z

e

XZ

G89.2247 Lect 2 17

Mathematically Equivalent Saturated Models

• Two variations of the first model suggest that the correlation between X and Z can itself be represented structurally.

YX

Z

eY

eZ

YX

Z

eY

eX

G89.2247 Lect 2 18

Representation of Covariance Matrix

• Both models imply the same correlation structure

• The interpretation, however, is very different.

1

1

1

1

1

1

3132

231

XZZY

XZXY

ZYXY

G89.2247 Lect 2 19

Model 2:X leads to Z and Y

• X is assumed to be causally prior to Z.The association between X and Z is due to X effects.

• Z partially mediates the overall effect of X on YX has a direct effect 1 on YX has an indirect effect on Y through ZPart of the bivariate association between Z and Y is spurious (due

to common cause X)

YX

Z

eY

eZ

G89.2247 Lect 2 20

Model 3:Z leads to X and Y

• Z is assumed to be causally prior to X.The association between X and Z is due to Z effects.

• X partially mediates the overall effect of Z on YZ has a direct effect 2 on YZ has an indirect effect on Y through XPart of the bivariate association between X and Y is spurious (due

to common cause Z)

YX

Z

eY

eX

G89.2247 Lect 2 21

Choosing between models

• Often authors claim a model is good because it fits to data (sample covariance matrix)All of these models fit the same (perfectly!)

• Logic and theory must establish causal order• There are other possibilities besides 2 and 3

In some instances, X and Z are dynamic variables that are simultaneously affecting each other

In other instances both X and Z are outcomes of an additional variable, not shown.

G89.2247 Lect 2 22

Mediation: A theory approach

• Sometimes it is possible to argue on theoretical grounds thatZ is prior to X and YX is prior to YThe effect of Z on Y is completely accounted for by the

indirect path through X.

• This is an example of total mediation• If is fixed to zero, then Model 3 is no longer

saturated.Question of fit becomes informativeTotal mediation requires strong theory

G89.2247 Lect 2 23

A Flawed Example

• Someone might try to argue for total mediation of family disorganization on low self-esteem through placement in foster care

• Baron and Kenny(1986) criteria might be metZ is significantly related to YZ is significantly related to XWhen Y is regressed on Z and X, is significant but is not

significant.

• Statistical significance is a function of sample size.• Logic suggests that children not assigned to foster care

who live in a disorganized family may suffer directly.

G89.2247 Lect 2 24

A More Compelling Example of Complete Mediation

• If Z is an experimentally manipulated variable such as a prime

• X is a measured process variable• Y is an outcome logically subsequent to X

It should make sense that X affects Y for all levels of Z

E.g. Chen and Bargh (1997)• Are participants who have been subliminally primed

with negative stereotype words more likely to have partners who interact with them in a hostile manner?