1 Stat 5100 Handout #8 – Notes: Simple Inference After verifying assumptions, look at “inference” Hypothesis testing – basic steps 1. Null Hypothesis (“no effect”) Alternative Hypothesis (“some effect”) 2. Test statistic -- depends on model and H0 3. Determine “sampling distribution” If H0 is true, and we “drew” many samples of size n from this population, calculating t for each sample, what would be distribution of these t values? When model assumptions are true and H0 is true, statistical theory says: Then t = 4. Get P-value
5
Embed
Stat 5100 Handout #8 Notes: Simple Inference · Stat 5100 Handout #8 – Notes: Simple Inference After verifying assumptions, look at “inference” Hypothesis testing – basic
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Stat 5100 Handout #8 – Notes: Simple Inference
After verifying assumptions, look at “inference”
Hypothesis testing – basic steps
1. Null Hypothesis (“no effect”)
Alternative Hypothesis (“some effect”)
2. Test statistic
-- depends on model and H0
3. Determine “sampling distribution”
If H0 is true, and we “drew” many samples of size n from this population, calculating t
for each sample, what would be distribution of these t values?
When model assumptions are true and H0 is true, statistical theory says:
Then t =
4. Get P-value
2
P-value is probability of observing a difference (t) at least as extreme as what was seen,
just by chance, when H0 is true.
5. Make conclusion in context
[Historical note regarding .05 threshold] To Ronald Fisher, the significance test only made sense
in the context of a sequence of experiments, all aimed at clarifying the same effect. The closest
he ever came to defining a specific p-value cut-off was in a 1929 article to the Society for
Psychical Research:
“An observation is judged significant, if it would rarely have been produced, in the
absence of a real cause of the kind we are seeking.”
“It is a common practice to judge a result significant, if it is of such a magnitude that it
would have been produced by chance not more frequently than once in twenty trials.
This is an arbitrary, but convenient, level of significance for the practical investigator ...”
“He should only claim that a phenomenon is experimentally demonstrable when he
knows how to design an experiment so that it will rarely fail to give a significant result.
Consequently, isolated significant results which he does not know how to reproduce are
left in suspense pending further investigation.”
(As on p. 99 of ``The Lady Tasting Tea'' (2001) by David Salsburg; similar discussion in “Truth,
Damn Truth, and Statistics”, by Paul F. Velleman in July 2008 Journal of Statistics Education: