Simple Correlation Scatterplots & r –scatterplots for continuous - binary relationships –H0: & RH: –Non-linearity Interpreting r Outcomes vs. RH: –Supporting.

Simple Correlation

• Scatterplots & r– scatterplots for continuous - binary relationships– H0: & RH:– Non-linearity

• Interpreting r• Outcomes vs. RH:

– Supporting vs. “contrary” results

• Outcomes vs. Population – Correct vs. Error results

• Differences between correlations in a group• Difference between correlations in different groups

A scatterplot a graphical depiction of the relationship between two quantitative (or binary) variables

• each participant’s x & y values depicted as a point in x-y space

Pearson’s correlation coefficient (r value) summarizes the direction and strength of the linear relationship between two quantitative variables into a single number (range from -1.00 to 1.00)

• you should always examine the scatterplot before considering the correlation between two variable

• NHST can be applied to test if the correlation in the sample is sufficiently large to reject H0: of no linear relationship between the variables in

the population

A linear regression formula allows us to take advantage of this relationship to estimate or predict the value of one variable (the criterion) from the other (the predictor).

• prediction should only be applied if the relationship between the variables is “linear” and “substantial”

Example of a “scatterplot”

Age of Puppy (weeks)

Am

ount

Pup

py E

ats

(pou

nds)

5

4

3

2

1

04 8 12 16 20 24

Puppy Age (x) Eats (y)

Sam

Ding

Ralf

Pit

Seff

…

Toby

8

20

12

4

24

..

16

2

4

2

1

4

..

3

When examining a scatterplot, we look for three things...• relationship

• no relationship• linear • non-linear

• direction (if linear)• positive• negative

• strength• strong• moderate• weak

linear, positive, weak

linear, negative, moderate

No relationship

Hi

Hi

Hi

Hi

Hi

Hi

Lo

Lo

LoLo

LoLo

linear, positive, strong

Hi

Hi

Lo

Lo

nonlinear, strong

Hi

HiLoLo

We can use correlation to examine the relationship between a quantitative predictor variable and a quantitative criterion variable.

Y

X

A positive r tells us those higher X values tend to have higher Y values

A negative r tells us those with lower X values tend to have higher Y values

A nonsignificant r tells us there is no linear relationship between X & Y

Y

Y

Y

Y

Y

strong + weak + +1.00

strong - weak - .00

X X

X X X

We can also use correlation to examine the relationship between a binary predictor variable and a quantitative criterion variable.

Y

grp 1 grp 2 grp 1 grp 2 grp 1 grp 2

grp 1 grp 2 grp 1 grp 2 grp 1 grp 2

A positive r tells us the group with the higher X code as the higher mean Y

A negative r tells us the group with the lower X code as the higher mean Y

A nonsignificant r tells us the groups have “equivalent” means on Y

Y

Y

Y

Y

Y

strong + weak + +1.00

strong - weak - .00

For each of the following show the envelope for the H0: and the RH:

People who have more depression before therapy will be those who have more depression after therapy.

Instructor isn’t related to practice.

Depression before

Dep

ress

ion

afte

r

H0:

RH:

Study

# E

rror

s

0 1Instructor

Pra

ctic

e

Those who study more have fewer errors on the spelling test

H0:

RH:

H0:

RH:

For each of the following show the envelope for the H0: and the RH:

People who score better on the pretest will be those who tend to score worse on the posttest

I predict that snapping turtles (coded 1) will eat more crickets than painted turtles (coded 0).

Pretest

Pos

t-te

st

H0:

RH:

# Sessions

Dep

ress

ion

0 1Species

# C

rick

ets

You can’t predict depression from the number of therapy sessions

H0:

RH:

H0:

RH:

The Pearson’s correlation ( r ) summarizes the direction and strength of the linear relationship shown in the scatterplot• r has a range from -1.00 to 1.00

• 1.00 a perfect positive linear relationship

• 0.00 no linear relationship at all

• -1.00 a perfect negative linear relationship

• r assumes that the relationship is linear• if the relationship is not linear, then the r-value is an underestimate

of the strength of the relationship at best and meaningless at worst

For a non-linear relationship, r will be based on a “rounded out” envelope -- leading to a misrepresentative r

Extreme Non-linear relationship

• r value is “misinformative”

actual scatterplot

notice... there is an x-y relationship

Scatterplot as correlation “sees it”

regression line has 0 slope & r = 0 -- no linear relationship

Moderate Non-linear relationship

• r value is an underestimate of the strength of the nonlinear relationship

actual scatterplot

notice... there is an x-y relationship

Scatterplot as correlation “sees it”

regression line has non-0 slope & r ~= 0 but, the regression line not a great representation of the bivariate relationship

NHST testing with r

H0: r = 0.00 is the same as r2 = 0.00

• get used to working with both r (the correlation between the 2 vars ) and r2 (the “variance shared between the 2 vars)”

Performing the significance test• software will usually provide an exact p-value (use p < .05)• a general formula is …

r² N = sample size F = ------------------------ (1 - r²) / (N - 2) Find F-critical using df = 1 & N-2

What “retaining H0:” and “Rejecting H0:” means...

• When you retain H0: you’re concluding…– The linear relationship between these variables in the

sample is notis not strong enough to allow me to conclude there is a linear relationship between them in the population represented by the sample.

• When you reject H0: you’re concluding…– The linear relationship between these variables in the

sample isis strong enough to allow me to conclude there is a linear relationship between them in the population represented by the sample.

The p-value (value range 1.0 – 0) tells the probability of making a Type I error if you reject the H0: based on the sample data• e.g., p = .10 means “if we reject H0: based on these data there is a 10%

chance that there really is no relationship between the variables in the population represented by the sample”

• The usual “acceptable risk” is less than 5% or p < .05

r (range -1.0 – 1.0) tells strength and direction of the bivariate relationship between Y & X• “large enough to be interesting” value vary across research areas , but a

common guideline is .10 = small, .30 = medium and .50 = large

r2 (range 0 – 1.0) tells how much of the Y variability is “accounted for,” “predicted from” or “caused by” X• e.g., r=.30 means that .302 (9%) of the Y variability is accounted for by X• “large enough to be interesting” will vary across research areas , but a

common guideline is 1% = small, 10% = medium and 25% = large

effect significance vs. effect size vs. shared variance

Interpret each of the following (significance, strength & direction)

For age & social skills: r = .25, p = .043.

For practice and performance errors: r = -.52, p = .015

For age and performance: r = -.33, p = .231

For gender (m=1, f=2) and social skills: r = .14, p = .004

For gender (m=1, f=2) and performance: r = -.31, p = .029

For gender (m=1, f=2) and practice: r = .11, p = .098

Sig – medium – positive Older adolescents tend to have higher social skills scores

Sig – large – negative Those who practiced more tended to have fewer errors

Nonsig – medium? - negative ? There is no linear relationship between age and performance

Sig – small – positive Females had higher mean on social skills scores

Sig – medium – negative Males had higher mean performance

Nonsig – small? – positive? No mean practice difference between males & females

Statistical Conclusion Errors

In the population there are only three possibilities...

In the Population

-r r = 0 +r

… and three possible statistical decisions

Type I error

Type I error

Type II error

Type II error

Correctly retained H0:

Correctly rejected H0:

Correctly rejected H0:

Type III error

Type III error

Please note that this is a different question than whether the results “match” the RH: This is about whether the results from the sample are “correct” – whether the results are “represent the population. This is about statistical conclusion validity

Outcomes

-r

r = 0

+r

The 9 outcomes come in 5 types …

Type I error -- “false alarm” - finding a significant mean difference between the conditions in the study when there really isn’t a difference between the populations

Type II error -- “miss” - finding no difference between the conditions of the study when there really is a difference between the populations

Type III error -- “misspecification” - finding a difference between the conditions of the study that is different from the the difference between the populations

Correctly retained H0: -- finding no difference between the conditions of the study when there really is no difference between the populations

Correctly rejected H0: -- finding a difference between the conditions of the study that is the same as the the difference between the populations

Practice with statistical decision errors ...

Type II

Correct rejection

Type III

Type I

Correct retention

We found that students who did more homework problems tended to have higher exam scores, which is what the other studies have found.

We found that students who did more homework problems tended to have lower exam scores. All other studies found the opposite effect.

We found that students who did more homework problems and those who did fewer problems tended to have about the same exam scores, which is what the other studies have found.

We found that students who did more homework problems tended to have lower exam scores. Ours is the only study with this finding, others find no relationship.

We found that students who did more homework problems and those who did fewer problems tended to have about the same exam scores. Everybody else has found that homework helps.

We found that students who did more homework problems tended to have lower exam scores. Ours is the only study with this finding.

Can’t tell -- what DID the other studies find?

correlation RH: vs. outcomes

Research Hypotheses -r r = 0 +r

… and three possible statistical outcomes

Outcomes

-r

r = 0

+r

So, there are only 9 possible combinations of RH: & Outcomes …of 4 types “effect as expected”

“unexpected null” “unexpected effect” “backward effect”

? ?? ?

? ?? ?

There are only three possible Research Hypotheses

Results contrary to RH:

Keep in mind that rejecting H0: does not guarantee support for the research hypothesis?

Why not ???

• The direction of the r might be opposite that of the RH:

• The RH: might be that’s there’s no correlation (RH: = H0:) ? ?

? ?

Remember !!!

Our purpose is not to “Reject the H0:” …

nor even to “support our RH:” …

Our real purpose is for our results to represent the relationship between the constructs in the target population !!!!!

A quick focus on the two that are most often confused …

Type III Statistical Decision Error– When our significant findings have a direction or pattern

different from that found in the population– A difference between “the effect we found” and “the effect

we should have found”

“Results contrary to our RH:”– When our findings have a direction or pattern different from

what we had hypothesized– A difference between “the effect we found” and “the effect

we hypothesized”

A result can be BOTH!!!!! (Or neither, or one, or the other !!!)

RH:, statistical conclusions & statistical decision errors ...

RH:

+ direction/pattern H0: - direction/pattern

Statistical Decision

+ direction/pattern

(p < .05)

H0:

(p > .05)

- direction/pattern

(p < .05

Correct rejection, Type I or Type III

Correct retention or Type II

RH: supported Unexpected H0:

Correct rejection, Type I or Type III

Correct rejection, Type I or Type III

Correct rejection, Type I or Type III

Correct rejection, Type I or Type III

Correct rejection, Type I or Type III

Correct retention or Type II

Correct retention or Type II

Unexpected effect“backward” Results

Lets practice …

Our RH: was that there will be a negative correlation between performance on the GRE and cumulative GPA.

We found r = .47, p = .016.

These results are “contrary to our RH:” -- a significant relationship in the opposite direction from the RH:

The consistent results of these other studies suggests that our finding was a correct rejection – what we found “does describe the relationship between these variables in the population”.

A literature review revealed 105 other studies involving these two variables, each of which found a correlation between .43 and .61 (all p < .05).

Our RH: was incorrect, not supported, but our results were right!!!

Another … Our RH: was that there will be a positive correlation between the severity of depression at the beginning of therapy and the amount of improvement a patient shows during the first six weeks of therapy.

We found r = .27, p = .085.

These results are “contrary to our RH:” -- a nonsignificant relationship isn’t the RH: +r

The consistent findings of these other studies suggests that our finding was a Type II error – what we found “doesn’t describe the likely relationship between these variables in the population”.

The 14 studies of these two variables which followed ours each found a correlation between -.33 and -.41 (all p < .05).

Our RH: was incorrect, not supported & our results were wrong!!!

Try this one … Our RH: was that there will be a positive correlation between social skills and comfort in an unfamiliar social situation.

We found r (82) = .37, p = .016.These results “support our RH:” - a significant relationship in

the RH: direction

The consistent results of these other studies suggests that our finding was a Type I error – what we found “does not describe the relationship between these variables in the population”.

Our RH: was incorrect but supported & our results were wrong !!!

A literature review revealed 22 other studies involving these two variables, each of which found a correlation between -.13 and .11 (all p > .05)

Last one …

Our RH: is that there will be a positive correlation between how much a person likes to compliment people and the number of close friends a person reports.

We found r (58) = .30, p < .05.These results “support our RH:” -- a significant, positive relationship, as hypothesized

Our finding was consistent with earlier research!

A literature review revealed 8 other studies of these two variables, each of which found a correlation between .25 and .32 (all p < .05).

The “researchers Trifecta” RH: is correct & supported and the results are correct 1!!!

Keep in mind … There are 27 combinations of RH: (+ 0 -), Results (+ 0 -) and Population value (+ 0 -).

“Success” depends more on a consistent agreement of the last two than of the first two!

Comparing “Correlated Correlations”

Are two correlations (that share a variable) different, within a single population/group?

Common versions of this question include…

• “Which is better correlated with performance, practice or prior skill?”

• “Which is better correlated with practice, performance or confidence?”

H0: The correlation between practice and performance is the same as between prior skill and performance.

H0: rperf, prac = rperf, pskill

Tested using Hotelling’s t-test, Steiger’s Z-test, or one of several variations…

Comparing Correlations across Populations/Groups

Is a correlation different in one pop/group than in another?

Common versions of this question include…

• “Is performance better correlated with practice for novices or for experts?”

• ““Is pracice better correlated with confidence for novices or for experts?”

H0: The correlation between practice and performance is the same for novice and for experienced participants

H0: rperf, prac for novices = rperf, pract for experts

Tested using Fisher’s Z-test

Identify the kinds of “correlation question” for each …

Is age a better predictor of social skills for boys than girls?

Is age a better predictor of social skills than SES?

Does age predict social skills?

Does IQ predict school performance?

Does IQ predict school performance better than SES?

Does SES predict IQ better for children or adults?

“Is a correlation different across two populations?”

“Are two correlations different, within a single population?”

“Is a correlation different across two populations?”

“Are two correlations different, within a single population?”

“Are two variables correlated, within a specific population?”

“Are two variables correlated, within a specific population?”

1st moment of caution when comparing correlations!

You have to decide if you are going to compare ….the “correlations” of the two predictors (including sign + or -)

or the “strength”, r2, |r|, or “predictive utility” of the two predictors (ignoring the sign)

For example:r(98) = .35 for # correct and confidence ratingsr(98) = -.25 for # correct and time to complete the task

(r = -.45 for confidence and time to complete)

Comparing .35 & -.25 yields Z = 3.55, p < .01 different rComparing .35 & .25 yields Z = .63, p > .05 same r2

Notice that these questions are equivalent if the signs of the two correlations are the same!

2nd moment of caution when comparing correlations!Don’t confuse asking…• if each variable is significantly correlated with the criterion vs. • if the variables are differentially correlated with the criterion

Example…r(28) = .37, p < .05 for # correct and time to complete the taskr(28) = .33, p > .05 for confidence and time to complete the task

Although # correct is significantly correlated with time to complete the task and confidence is not significantly correlated with time to complete, it is a different question to ask if the two correlations are significantly different!

Said differently There may not be a significant difference between a significant correlation and a non-significant correlation.

An important variation of comparing correlations…

While it is most common to apply these models to ask which of two variables is the better predictor of a given criterion…

… it is possible to apply them to ask for which criterion a given variable is the better predictor.

Often we collect multiple variables that are considered “outcome” or criterion variables. If so, when we talk about how good a predictor is, it is important to know if the effectiveness of the predictor depends upon the criterion we are using.

Remember – like in the other applications of these models …

• it is different to say that a predictor is correlated with one criterion and not correlated with another, than to say it is differentially correlated with the two!

• it is different to ask if two correlations are significantly different than to ask if the two r2 or |r| are different