Unit 2: Correlation and Causality

© Judith D. Singer, Harvard Graduate School of Education Unit #2/Slide #1

Unit 2: Correlation and Causality


The S-030 roadmap: Where’s this unit in the big picture?

Unit 2:Correlation

and causality

Unit 3:Inference for the regression model

Unit 4:Regression assumptions:

Evaluating their tenability

Unit 5:Transformations

to achieve linearity

Unit 6:The basics of

multiple regression

Unit 7:Statistical control in

depth:Correlation and

collinearity

Unit 10:Interaction and quadratic effects

Unit 8:Categorical predictors I:

Dichotomies

Unit 9:Categorical predictors II:

Polychotomies

Unit 11:Regression modeling

in practice

Unit 1:Introduction to

simple linear regression

Building a solid

foundation

Mastering the

subtleties

Adding additional predictors

Generalizing to other types of

predictors and effects

Pulling it all

together


In this unit, we’re going to learn about…

• Developing a heuristic understanding of the correlation coefficient (r)– Understanding correlation as regression on standardized variables– The relationship between r and R2—how large is large?

• From correlation to causality– Randomized experiments: The “gold standard” for establishing causality– What can you do when randomized experiments aren’t possible?– When might an observed correlation not indicate a causal relationship? – Spurious correlation, confounding, Simpson’s paradox, reciprocal

causation and ecological correlation– Conditions for establishing causality


RQ: Is there a link between TV exposure and attention deficit problems?

Read the Pediatrics article Listen to the NPR Interview

Imagine you’re a peer reviewer for the paper:1. What’s particularly good about the study?2. What’s potentially problematic?

Christakis, D.A., et al. (2004). Early television

exposure and subsequent attentional problems in

children. Pediatrics, 113(4): 708-713


Developing a measure of “co-relation”: Meet Karl Pearson(27 March 1857 – 27 April 1936)

Heredity: relationships between siblings and spouses (Pearson & Lee, 1903, On the laws of inheritance in

man, Biometrika)

• Pearson, Galton’s advisee and the first Galton Professor of Eugenics at University College, London (shown here with Galton at right)

• He and RA Fisher hated each other and founded separate journals to publish their papers

• Developed, or named, many of the basic tools of modern statistics, including standard deviation, 2 goodness of fit, and correlationLearn more about Karl Pearson

Pearson’s “problems” to solve:

(1) Neither variable is an “outcome” or a “predictor”

(2) The measure of correlation should be dimensionless, (eg., applicable for inches or feet, digit span or stature)

His solution: Re-express (transform) both variables on new “standard” scales that essentially eliminate the particular metrics of the original scales


Transformation and standardization: Re-expressing a variable’s scale

ID OwnIQ SOwnIQ FostIQ SFostIQ

1 68 -1.9985 63 -2.3080 2 71 -1.7943 76 -1.4535 . . .25 95 -0.1606 96 -0.138926 96 -0.0925 93 -0.3361. . .52 129 2.1539 117 1.2415 53 131 2.2900 132 2.2274

Variablesd

VariableVariableSVariable

Standardization: A particular transformation that

yields a new variable with mean = 0 and sd = 1

Transformation:Any re-expression of a variable’s

scale

8.1

32

F

C

CentigradeFarenheit

Standardization...Forces the sample mean of the new

variable to be 0 and its sd to be 1The new values measure an individual’s

distance from the sample mean in sd unitsDoesn’t change anyone’s relative rankDoesn’t create a normally distributed variable

69.14

36.97

OwnIQSOwnIQ

21.15

11.98

FostIQSFostIQ

Mean- 2sd

Mean- 1sd

Mean Mean+ 1sd

Mean+ 2sd

Mean

Mean- 1sd

Mean- 2sd

Mean+ 2sd

Mean+ 1sd

Mean = 98.11

sd = 15.21

Mean = 97.36

sd = 14.69


^ SOwnIQSFostIQ 8767.00000.0

Using a regression on standardized variables to understand correlation

Does 0.8767 seem familiar?0.87672 =

Standardized regression line goes

precisely through (0,0)

At average X (SOwnIQ=0), we predict average Y (SFostIQ=0)

Slope of the standardized

regression line assesses the estimated

difference in FostIQ(measured in standard

deviation units)per standard deviation

in OwnIQ

Pearson product-moment coefficient, r

r2 = R2

0.7686


How do we interpret r?


Plots to help develop your intuition for interpreting r

Interactive applet for learning more about correlation


Understanding the relationship between r and R2 (and their use as measures of “effect size”)

1-R2R2r

96%4%.2

99%1%.1Not uncommon in social sciences, but when r < .2, you have very

little explanatory power (R2 < 4%)

75%25%.5

84%16%.4

91%9%.3Covers most “statistically significant” correlations in social sciences, but even when r = .5, you’re only explaining 25% of the

variance in Y

64%36%.6

51%49%.7

Rare in the social sciences and even when r = .7, you’re still

explaining less than ½ the variance in Y

36%64%.8

19%81%.9

10%90%.95

Extremely rare in the social sciences, unless you have

aggregate data or a coding problem(!)

Small: r=.10

Medium: r=.30

Large: r=.50

Cohen’s guidelines

Another way of thinking about r is

as a measure of effect size


Plots to help develop your intuition for interpreting r

F.J. Anscombe. (1973). Graphs in statistical analysis. American Statistician, 27, 17-21.

Anscombe’s Quartet(r = 0.816)


From correlation to causality

Identified mechanism You have a sound theory to explain how a change in the predictor produces a change in the outcome

No plausible alternative explanationThere’s no rival predictor that can explain away the

observed correlation

Responsiveness You demonstrate that a change in the predictor

actually produces a change in the outcome

ConsistencyYou find the same result in other populations, with different characteristics,

at different times“I interpreted…Galton…to mean that there was a category broader

than causation, namely correlation…and that this new

conception of correlation brought psychology, anthropology,

medicine, and sociology in large parts into the field of mathematical treatment. It was Galton who first freed me from the prejudice that sound mathematics could only be applied to natural phenomenon

under the category of causation. Here for the first time was a possibility, I will not say a

certainty, of reaching knowledge—as valid as physical knowledge was then thought to be—in the

field of living forms and above all in the field of human conduct.”

Karl Pearson, 1889

What do we really mean when we say: “Associated with”“Related to”“Explained by”“Varies with”“Covaries with”

Four criteria for establishing causality

Highest priorities for design and analysis

and often the hardest to establish – focus on finding some “exogenous

variation”


From correlation to causality: The Counterfactual Causal Model

Counterfactual reasoning provides a powerful lens for thinking about these

questions

You’d like to know what outcome values these individuals would have had if they had received a

“different treatment” (i.e., if they had different predictor values)?

Treatment

NoTreatment

COMPARE

Outcome

Treatment

Control

Treatment

ControlA

ssig

nm

en

t

The Challenge of the Counterfactual

X

X

???

???


Why randomized experiments are the “gold standard”

Four important attributes of randomized experiments

The researcher actively

intervened in the system, actually changing X (the treatment) and

seeing what happens to Y

Any difference found in Y must be

due to the changing of X (the

treatment) because there’s no other plausible

explanation

Because of random assignment, groups are

guaranteed to be initially

equivalent, on average, on all

observable (and unobservable) characteristics

4.82(4.02)

5.00(3.50)

October(pretest)

BTLArthur

5.69(3.87)

7.04(3.56)

February (midterm)

7.16(3.75)

9.13(4.35)

May/June (posttest)

(n=51)(n=57)

RQ: Can narrative skills be ‘taught’ via TV to English Language Learners?

Uchikoshi, Y. (2005). Narrative development in bilingual kindergarteners: Can Arthur help? Developmental Psychology,

41(3): 464-478.

The control group provides the ideal counterfactual—our best estimate

of what the treatment group

would have looked like if it didn’t

receive the treatment


There will always be studies where researchers have the burden of proof

Ethics: Morally, there are some treatments to which you can’t expose peopleDoes radiation cause cancer?

Time and cost: Practically, some information is better than no informationDoes quality child care cause better life outcomes?Availability of data: With so much data, shouldn’t we analyze it?

How might you try to establish responsiveness? The key question is:

How are predictor values assigned?

•They’re not: they’re immutable characteristics of people

•Participants choose them

•Researchers assign them (but not randomly)

•Outside forces inadvertently change them

•External raters assign them using a ranking criterion (e.g., identifying those above a cut-score)How might you eliminate alternative explanations? The key question is:

Can the findings be explained away?

•Can you establish that the groups were equivalent initially?

•Can you isolate that portion of the variation in X that’s exogenous (not subject to selection bias) ?

•Can you rule out other explanations for the observed association?

“You can’t fix by analysis what you bungled by design…”

Light, Singer and Willett (1990)

When participants (or even researchers) choose, the conclusions are weaker because they’re subject to selection bias

Natural experiments

Regression discontinuitydesigns

Matching (especially propensity score matching is very popular now)

Instrumental variables

Feasibility: Logistically, there are some treatments to which you can’t assign peopleDoes education cause increased income?

Many would argue that these can’t be “causes”

Let’s think about how you might go about doing this


Non-experimental data: Might the correlation be causal?

Correlation

X Y X Y

Causation

But, just because we haven’t done an experiment doesn’t mean the correlation isn’t causal

Sample Tobacco Industry Statements

– “[the causes of diseases] are complex, and the mechanism of causation, as well as the possible role of any cigarette smoke constituent in causation, have not been scientifically established”

– “[At] least two standards for establishing causation exist. An epidemiological standard of causation, based primarily on statistical evidence, … [and] the more rigorous traditional scientific standard…[which] requires, among other things… well-designed and conducted … experiments.”

Read the Waxman (2002) report Tobacco industry statements in the Department of Justice

Lawsuit

Does Imply ?


Spurious Correlation: Common Response to a Third Variable

There’s a 3rd variable, Z, which causes changes in

X and may—or may not—also

cause changes in Y

X Y

Z

SES is often the “3rd variable”It is easy to prove that the wearing of tall hats

and the carrying of umbrellas enlarges the chest, prolongs life, and confers comparative immunity from disease… A university degree, a daily bath,

the owning of thirty pairs of trousers, a knowledge of Wagner’s music, a pew in church, anything, in short, that implies more means and better nurture…can be statistically palmed off as a magic spell conferring all sorts of privileges…The mathematician whose correlations would fill a Newton with admiration, may, in collecting and

accepting data and drawing conclusions from them, fall into quite crude errors by just such

popular oversights --George Bernard Shaw (1906)

?

But not all spurious correlations are nonsense

Polio Incidence

Soft DrinkConsumption

Height Reading testscores

Age


Confounding: A “confusion of effects”:A third variable may (or may not) explain away (or reduce)

the correlation

X Y

Z

There’s a 3rd variable, Z, which is correlated with X

and which causes changes in Y, but we don’t know if this

explains away the correlation between X & Y

Pauperism Poorhouses

Yule (1899) An investigation into the causes of changes in pauperism in England

Gov’t EfficiencyPigou (1899)

Yule’s footnote 25“Strictly speaking, for

‘due to’ read ‘associated with.’ ”

?

?


Simpson’s paradox: A third variable may reverse(!) the correlation

Some confounders don’t just ‘explain

away’ the association, they

reveal a reversal in the direction of the

effect

X Y

Z

+

Sex bias in graduate admissions:UC Berkeley (1973)

N apps % accepted

Men 8442 44%

Women 4321 34%

-

Learn more about Simpson’s paradox

AdmissionMale +

Department

-

r = -0.56


Reciprocal causation: Do happy mothers make happy babies? Or it is the other way around?

X may cause Y or Y may cause

X—with the data we have, we just

can’t tell

X Y

?

?

Cross-sectional observational studies are particularly susceptible to questions of

reciprocal causation• Motherhood and suicide risk: married women

with children are at lower risk of suicide than unmarried women; the more children the lower the risk

• Depression and smoking: teens who are daily smokers are more likely to be seriously depressed


Ecological correlation: Aggregate and individual correlations may differ

Aggregate data describe aggregate relationships, not individual level relationships

Xaggregate

Yaggregate

?-.526

-.619

48 states

9 regions

Robinson, W.S. (1950). Ecological Correlations and the Behavior of Individuals. American Sociological Review 15:

351–357.

.11897,272 individuals

Foreign Born

Unit of analysis

Correlations with Illiteracy

(Rural, low foreign born, but lots of

illiterates)

(Urban, lots of foreign born, but also lots of very literate folks)


Where to go to learn more about establishing causality

In recent years, there has been an explosion of interest in the conditions necessary for establishing causal inferences. Different disciplines use different standards and approaches, and there is much to

learn by reading broadly. Here are some sources if you want to explore this topic in more depth:

General overviews (warning: some of these are very

technical)• Shadish, WR, Cook, TF and Campbell, ST

(2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference, Boston, MA: Houghton Mifflin.

• Holland PW (1986) Statistics and causal inference (with discussion). Journal of the American Statistical Association, 81, 945-970. [Link to search results for article and discussion.]

• Freedman, DA (2004). Graphical models for causation and the identification problem. Evaluation Review, 28(4), 267-293.

• Rubin, DB (2005) Causal inference using potential outcomes: Design, Modeling, Decisions. Journal of the American Statistical Association, 100, 322-331.

Discussions focused on sociology• Winship, C & Morgan, S (1999). The

estimation of causal effects from observational data. Annual Review of Sociology, 25, 659-707.

• Morgan, SL & Winship, C (2007). Counterfactuals and Causal Inference: Methods and Principles for Social Research. NY: Cambridge U Press.

Discussions focused on education• Shavelson, RJ & Towne, L eds (2002) Scientific

Research in Education. Washington, DC: National Academy Press.

• Angrist, JD (2004). American education research changes tack. Oxford Review of Economic Policy, 20(2), 198-212

• Gamse, BC & Singer, JD (2005) Lessons from the Red Sox Playbook. Harvard Education Letter, 21(2), 7-8

• Cook, TD (2001) Sciencephobia: Why education researchers reject randomized experiments. Education Next, Fall, 63-68

Discussions focused on psychology• Rutter M (2007) Proceeding from Observed

Correlation to Causal Inference: The Use of Natural Experiments, Perspectives on Psychological Science, 2(4) 377-395.

Discussions focused on epidemiology• Rothman, KJ & Greenland, S (2005). Causation

and causal inference in epidemiology, American Journal of Public Health, Supplement 1, 95(1), S144-S150

• Maldonado, G & Greenland, S (2002). Estimating causal effects. International Journal of Epidemiology, 31, 422-429.

• Krieger, N (1994). Epidemiology and the web of causation: Has anyone seen the spider? Social Science and Medicine. 39(7) 887-903.


What’s the big takeaway from this unit?

• Correlation coefficients are helpful tools when used correctly– Having a scale free measure of association is a powerful concept; you

can develop your intuition about the meaning of correlations and that intuition will carry across all types of variables

– The size of a correlation tells you about the strength of a relationship, not its magnitude. For the magnitude, you need the slope

• Correlation Causality– Randomized experiments are the gold standard for establishing

causality, but even with them there can be limits to your inferences– There are many reasons why you might find a correlation between an

outcome and a predictor; learn how to think about alternative explanations and evaluate whether they’re likely to hold in any given context

– When analyzing data, consider the steps involved in going from correlation to causality and decide how far your inferences can go

• There are many other issues involved in moving from correlation to causality– But before being able to tackle these more technical treatments, you need to

know much more about the basic regression approach– We offer an entire class on this issue – S-290: Quantitative methods

for improving causal inference


options nodate nocenter nonumber;

title1 "Unit 2: IQs of Cyril Burt's identical twins";footnote1 "Program: Unit 2--Burt analysis.sas";

*-----------------------------------------------------*Be sure to update the infile reference to the file's location on your computer*-----------------------------------------------------*;

*---------------------------------------------------- *Input Burt data and name variables in dataset*-----------------------------------------------------*;

data one; infile 'm:\datasets\Burt.txt'; input ID 1-2 OwnIQ 4-6 FostIQ 8-10;

*-----------------------------------------------------*Estimate bivariate correlation between owniq & fostiq (Pearson correlation coefficient)*-----------------------------------------------------*;

proc corr data=one; var owniq fostiq;

run;

quit;

Appendix 1: Annotated PC SAS code for Unit 2, Burt data

Don’t forget the semicolon at the end of every statement;Don’t forget the semicolon at the end of every statement;

Don’t forget to specify the location of the raw data, and check that you are indicating the appropriate drive

Don’t forget to specify the location of the raw data, and check that you are indicating the appropriate drive

proc corr estimates bivariate correlations between variables you specify. Its var statement syntax is var1 var2 var3 … varn (note that it has neither an * (like proc gplot) or an = (like proc reg)

proc corr estimates bivariate correlations between variables you specify. Its var statement syntax is var1 var2 var3 … varn (note that it has neither an * (like proc gplot) or an = (like proc reg)


Glossary terms included in Unit 2

• Aggregate data• Causality• Confounder• Correlation• Reciprocal causation• Spurious• Standardization• Transformation

Unit 2: Correlation and Causality

Documents