Unit 5: Inference for categorical variables Lecture 2: Inference for 2-sample proportions Statistics 101 Thomas Leininger June 12, 2013
Unit 5: Inference for categorical variablesLecture 2: Inference for 2-sample proportions
Statistics 101
Thomas Leininger
June 12, 2013
Announcements
Announcements
Quiz tomorrow
Project reports...
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 2 / 33
Difference of two proportions
Melting ice cap
Question
Scientists predict that global warming may have big effects on the polarregions within the next 100 years. One of the possible effects is thatthe northern ice cap may completely melt. Would this bother you agreat deal, some, a little, or not at all if it actually happened?
(a) A great deal
(b) Some
(c) A little
(d) Not at all
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 3 / 33
Difference of two proportions
Results from the GSS
The GSS asks the same question, below is the distribution ofresponses from the 2010 survey:
A great deal 454Some 124A little 52Not at all 50Total 680
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 4 / 33
Difference of two proportions
Parameter and point estimate
Parameter of interest: Difference between the proportions of allDuke students and all Americans who would be bothered a greatdeal by the northern ice cap completely melting.
pDuke − pUS
Point estimate: Difference between the proportions of sampledDuke students and sampled Americans who would be bothereda great deal by the northern ice cap completely melting.
p̂Duke − p̂US
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 5 / 33
Difference of two proportions
Inference for comparing proportions
The details are the same as before...
CI: point estimate ±margin of error
HT: Use Z = point estimate−null valueSE to find appropriate p-value.
We just need the appropriate standard error of the point estimate(SEp̂Duke−p̂US ), which is the only new concept.
Standard error of the difference between two sample proportions
SE(p̂1−p̂2) =
√p1(1 − p1)
n1+
p2(1 − p2)
n2
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 6 / 33
Difference of two proportions Confidence intervals for difference of proportions
Conditions for CI for difference of proportions
1 Independencewithin groups:
The US group is sampled randomly and we’re assuming that theDuke group represents a random sample as well.nDuke < 10% of all Duke students and 680 < 10% of all Americans.
We can assume that the attitudes of Duke students in the sampleare independent of each other, and attitudes of US residents inthe sample are independent of each other as well.between groups: The sampled Duke students and the USresidents are independent of each other.
2 Success-failure:At least 10 observed successes and 10 observed failures in thetwo groups.
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 7 / 33
Difference of two proportions Confidence intervals for difference of proportions
Application exercise:CI for difference of proportions
Construct a 95% confidence interval for the difference between theproportions of Duke students and Americans who would be bothereda great deal by the melting of the northern ice cap (pDuke − pUS ).
Data Duke USA great deal 454Not a great deal 226Total 680
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 8 / 33
Difference of two proportions HT for comparing proportions
Question
Which of the following is the correct set of hypotheses for testing if theproportion of all Duke students who would be bothered a great dealby the melting of the northern ice cap differs from the proportion of allAmericans who do?
(a) H0 : pDuke = pUS
HA : pDuke , pUS
(b) H0 : p̂Duke = p̂US
HA : p̂Duke , p̂US
(c) H0 : pDuke − pUS = 0HA : pDuke − pUS , 0
(d) H0 : pDuke = pUS
HA : pDuke < pUS
Both (a) and (c) are correct.
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 9 / 33
Difference of two proportions HT for comparing proportions
Flashback to working with one proportion
When constructing a confidence interval for a populationproportion, we check if the observed number of successes andfailures are at least 10.
np̂ ≥ 10 n(1 − p̂) ≥ 10
When conducting a hypothesis test for a population proportion,we check if the expected number of successes and failures areat least 10.
np0 ≥ 10 n(1 − p0) ≥ 10
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 10 / 33
Difference of two proportions HT for comparing proportions
Pooled estimate of a proportion
In the case of comparing two proportions where H0 : p1 = p2,there isn’t a given null value we can use to calculated theexpected number of successes and failures in each sample.
Therefore, we need to first find a common (pooled) proportion forthe two groups, and use that in our analysis.
This simply means finding the proportion of total successesamong the total number of observations.
Pooled estimate of a proportion
p̂ =# of successes1 +# of successes2
n1 + n2
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 11 / 33
Difference of two proportions HT for comparing proportions
Application exercise:Pooled estimate of a proportion - in context
Calculate the estimated pooled proportion of Duke students and Amer-icans who would be bothered a great deal by the melting of the north-ern ice cap. Which sample proportion (p̂Duke or p̂US ) the pooled esti-mate is closer to? Why?
Data Duke USA great deal 454Not a great deal 226Total 680
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 12 / 33
Difference of two proportions HT for comparing proportions
Application exercise:HT for comparing proportions
Do these data suggest that the proportion of all Duke students whowould be bothered a great deal by the melting of the northern ice capdiffers from the proportion of all Americans who do? Calculate the teststatistic, the p-value, and interpret your conclusion in context of thedata.
Data Duke USp̂ 0.668n 680
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 13 / 33
Recap
Recap - inference for one proportion
Population parameter: p, point estimate: p̂Conditions:
independence- random sampleat least 10 successes and failures- if not→ randomization
Standard error: SE =
√p(1−p)
n
for CI: use p̂for HT: use p0
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 14 / 33
Recap
Recap - comparing two proportions
Population parameter: (p1 − p2), point estimate: (p̂1 − p̂2)
Conditions:independence within groups- random sample and 10% condition met for both groupsindependence between groupsat least 10 successes and failures in each group- if not→ randomization
SE(p̂1−p̂2) =√
p1(1−p1)n1
+p2(1−p2)
n2
for CI: use p̂1 and p̂2for HT:
when H0 : p1 = p2: use p̂pool =# suc1+#suc2
n1+n2
when H0 : p1 − p2 = (some value other than 0): use p̂1 and p̂2
- this is pretty rare
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 15 / 33
Recap
Reference - standard error calculations
one sample two samples
mean SE = s√n
SE =
√s2
1n1
+s2
2n2
proportion SE =
√p(1−p)
n SE =√
p1(1−p1)n1
+p2(1−p2)
n2
When working with means, it’s very rare that σ is known, so weusually use s.When working with proportions,
if doing a hypothesis test, p comes from the null hypothesisif constructing a confidence interval, use p̂ instead
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 16 / 33
Small sample inference for difference between two proportions Back of the hand
Back of the hand
There is a saying “know something like the back of your hand.” De-scribe an experiment to test if people really do know the backs of theirhands.
In the MythBusters episode, 11 out of 12 people guesses the backs oftheir hands correctly.
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 17 / 33
Small sample inference for difference between two proportions Back of the hand
Comparing back of the hand to palm of the hand
MythBusters also asked these people to guess the palms of theirhands. This time 7 out of the 12 people guesses correctly. The dataare summarized below.
Back Palm TotalCorrect 11 7 18Wrong 1 5 6Total 12 12 24
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 18 / 33
Small sample inference for difference between two proportions Back of the hand
Proportion of correct guesses
Palm Back TotalCorrect 11 7 18Wrong 1 5 6Total 12 12 24
Proportion of correct in the back group: 1112 = 0.916
Proportion of correct in the palm group: 712 = 0.583
Difference: 33.3% more correct in the back of the hand group.
Based on the proportions we calculated, do you think the chance ofguessing the back of the hand correctly is higher than palm of thehand?
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 19 / 33
Small sample inference for difference between two proportions Back of the hand
Hypotheses
What are the hypotheses for comparing if the proportion of people whocan guess the backs of their hands correctly is greater than the pro-portion of people who can guess the palm of their hands correctly?
H0: pback = ppalm
HA : pback > ppalm
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 20 / 33
Small sample inference for difference between two proportions Back of the hand
Conditions?
Independence - within groups, between groups?Within each group we can assume that the guess of one subjectis independent of another.Between groups independence is not satisfied - we have thesame people guessing. However we’ll assume they’reindependent guesses to continue with the analysis.
Sample size?p̂pool =
11+712+12 = 18
24 = 0.75Expected successes in back group: 12 × 0.75 = 9, failures = 3Expected successes in palm group: 12 × 0.75 = 9, failures = 3Since S/F condition fails, we need to use simulation to comparethe proportions.
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 21 / 33
Small sample inference for difference between two proportions Randomization HT for comparing two proportions
Simulation scheme
1 Use 24 index cards, where each card represents a subject.2 Mark 18 of the cards as “correct” and the remaining 6 as “wrong”.3 Shuffle the cards and split into two groups of size 12, for back
and palm.4 Calculate the difference between the proportions of “correct” in
the back and palm decks, and record this number.5 Repeat steps (3) and (4) many times to build a randomization
distribution of differences in simulated proportions.
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 22 / 33
Small sample inference for difference between two proportions Randomization HT for comparing two proportions
Interpreting the simulation results
When simulating the experiment under the assumption ofindependence, i.e. leaving things up to chance.
If results from the simulations based on the chance model look likethe data, then we can determine that the difference between theproportions correct guesses in the two groups was simply due tochance.
If the results from the simulations based on the chance model do notlook like the data, then we can determine that the difference betweenthe proportions correct guesses in the two groups was not due tochance, but because people actually know the backs of their handsbetter.
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 23 / 33
Small sample inference for difference between two proportions Randomization HT for comparing two proportions
Simulation results
In the next slide you can see the result of a hypothesis test(using only 100 simulations to keep the results simple).
Each dot represents a difference in simulated proportion ofsuccesses. We can see that the distribution is centered at 0 (thenull value).
We can also see that 9 out of the 100 simulations yieldedsimulated differences at least as large as the observed difference(p-value = 0.09).
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 24 / 33
Small sample inference for difference between two proportions Randomization HT for comparing two proportions
inference(hand, gr, est = "proportion", type = "ht", null = 0,
alternative = "greater", order = c("back","palm"), success = "correct",
method = "simulation", seed = 879, nsim = 100)
Response variable: categorical, Explanatory variable: categorical
Two categorical variables
Difference between two proportions -- success: correct
Summary statistics:
group
data back palm Sum
correct 11 7 18
wrong 1 5 6
Sum 12 12 24
Observed difference between proportions (back-palm) = 0.3333
H0: p_back - p_palm = 0 ; HA: p_back - p_palm > 0
p-value = 0.09
group
data
back palm
correct
wrong
Randomization distribution
-0.2 0.0 0.2 0.4
010
2030
40 observed 0.3333
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 25 / 33
Small sample inference for difference between two proportions Randomization HT for comparing two proportions
What if we want to compare more than two proportions?
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 26 / 33
Intro to the Chi-square test Weldon’s dice
Weldon’s dice
Walter Frank Raphael Weldon (1860 -1906), was an English evolutionary biologistand a founder of biometry. He was the jointfounding editor of Biometrika, with FrancisGalton and Karl Pearson.
In 1894, he rolled 12 dice 26,306 times, andrecorded the number of 5s or 6s (which heconsidered to be a success).
It was observed that 5s or 6s occurred more often than expected,and Pearson hypothesized that this was probably due to theconstruction of the dice. Most inexpensive dice havehollowed-out pips, and since opposite sides add to 7, the facewith 6 pips is lighter than its opposing face, which has only 1 pip.
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 27 / 33
Intro to the Chi-square test Weldon’s dice
Labby’s dice
In 2009, Zacariah Labby (U ofChicago), repeated Weldon’sexperiment using ahomemade dice-throwing, pipcounting machine.
http:// www.youtube.com/watch?v=95EErdouO2w
The rolling-imaging processtook about 20 seconds perroll.
Each day there were ∼150 images to process manually.At this rate Weldon’s experiment was repeated in a little morethan six full days.Recommended reading:http:// galton.uchicago.edu/ about/ docs/ labby09dice.pdf
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 28 / 33
Intro to the Chi-square test Weldon’s dice
Labby’s dice (cont.)
Labby did not actually observe the same phenomenon thatWeldon observed (higher frequency of 5s and 6s).Automation allowed Labby to collect more data than Weldon didin 1894, instead of recording “successes” and “failures”, Labbyrecorded the individual number of pips on each die.
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 29 / 33
Intro to the Chi-square test Creating a test statistic for one-way tables
Expected counts
Question
Labby rolled 12 dice 26,306 times. If each side is equally likely to comeup, how many 1s, 2s, · · · , 6s would he expect to have observed?
(a) 16
(b) 126
(c) 26,3066
(d) 12×26,3066 = 52, 612
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 30 / 33
Intro to the Chi-square test Creating a test statistic for one-way tables
Summarizing Labby’s results
The table below shows the observed and expected counts fromLabby’s experiment.
Outcome Observed Expected
1 53,222 52,612
2 52,118 52,612
3 52,465 52,612
4 52,338 52,612
5 52,244 52,612
6 53,285 52,612
Total 315,672 315,672
At first glance, does there appear to be an inconsistency between theobserved and expected counts?
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 31 / 33
Chi-square test of GOF Creating a test statistic for one-way tables
Setting the hypotheses
Do these data provide convincing evidence to suggest an inconsis-tency between the observed and expected counts?
H0: There is no inconsistency between the observed and theexpected counts. The observed counts follow the samedistribution as the expected counts.
HA : There is an inconsistency between the observed and theexpected counts. The observed counts do not follow the samedistribution as the expected counts. There is a bias in which sidecomes up on the roll of a die.
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 32 / 33
Chi-square test of GOF Creating a test statistic for one-way tables
Evaluating the hypotheses
To evaluate these hypotheses, we quantify how different theobserved counts are from the expected counts.
Large deviations from what would be expected based onsampling variation (chance) alone provide strong evidence forthe alternative hypothesis.
This is called a goodness of fit test since we’re evaluating howwell the observed data fit the expected distribution.
Statistics 101 (Thomas Leininger) U5 - L2: Inf. for 2-sample prop. June 12, 2013 33 / 33