Administrative Matters Midterm II Results Take max of two midterm scores:

Administrative Matters

Midterm II Results

Take max of two midterm scores:

Administrative Matters

Midterm II Results

Take max of two midterm scores

Approx grades:

92-100 A

82-92 B

70-82 C

60-70 D

0 – 60 F

Last Time

• Confidence Intervals– For proportions (Binomial)

• Choice of sample size– For Normal Mean– For proportions (Binomial)

• Interpretation of Confidence Intervals

Reading In Textbook

Approximate Reading for Today’s Material:

Pages 493-501, 422-435, 447-467

Approximate Reading for Next Class:

Pages 422-435, 372-390

Sample Size for Proportions

i.e. find so that:

Now solve to get:

(good candidate for list of formulas)

n )1,0,975.0(1

NORMINV

npp

m

m

ppNORMINVn

11,0,975.0

ppm

NORMINVn

1

1,0,975.02


i.e. find so that:

Now solve to get:

Problem: don’t know

n )1,0,975.0(1

NORMINV

npp

m

m

ppNORMINVn

11,0,975.0

p

ppm

NORMINVn

1

1,0,975.02


Solution 1: Best Guess

Use from:

– Earlier Study

– Previous Experience

– Prior Idea

p̂


Solution 2: Conservative

Recall

So “safe” to use:

4

11max1,0

ppp

411,0,975.0

2

mNORMINV

n

Interpretation of Conf. Intervals

Mathematically:

pic 1 pic 2

3rd interpretation

"",.. bracketsmXmXICtheP

mXPmXmP 95.0

mXmXP


Frequentist View: If repeat the

experiment many times



experiment many times,

About 95% of the time, CI’s will contain μ



experiment many times,

About 95% of the time, CI’s will contain μ

(and 5% of the time it won’t)


Nice Illustration:

Publisher’s Website

• Statistical Applets

• Confidence Intervals











Nice Illustration:




Shows proper interpretation


Nice Illustration:




Shows proper interpretation:– If repeat drawing the sample


Nice Illustration:




Shows proper interpretation:– If repeat drawing the sample

– Interval will cover truth 95% of time


Nice Illustration:




Lower Confidence Level (95% 80%)


Nice Illustration:




Lower Confidence Level (95% 80%):– Shorter confidence intervals


Nice Illustration:




Lower Confidence Level (95% 80%):– Shorter confidence intervals

– Leads to lower hit rate


Recall Class HW:

Estimate % of Male Students at UNC


Recall Class HW:


Revisit Class Example 7http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg7.xls




Recall Class HW:


Recall:

Q1: Sample of 25 from Class


Recall Class HW:


Recall:


Q2: Sample of 25 from any doorway


Recall Class HW:


Recall:



Q3: Sample of 25 think of names


Recall Class HW:


Recall:



Q3: Sample of 25 think of names

Q4: Random sample (from phone book)


Histogram analysis: Class Example 7http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg7.xls

Q1:

Sample from Class



Q1:

Sample from Class:

- Compare to

theoretical



Q1:

Sample from Class:

- Compare to

theoretical

- Some bias



Q1:

Sample from Class:

- Compare to

theoretical

- Some bias

- less variation



Q2:

From Doorways



Q2:

From Doorways:

- No bias



Q2:

From Doorways:

- No bias

- More variation



Q3:

Think up names



Q3:

Think up names:

- Upwards bias



Q3:

Think up names:

- Upwards bias

- More variation



Q4:

Random Sample



Q4:

Random Sample:

- Looks better?



Q4:

Random Sample:

- Looks better?

- Reasonable

variation?



Q4:

Random Sample:

- Looks better?

- Reasonable

variation?

- Really need

CIs etc.


Now consider C.I. View:

Class Example 13http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg13.xls


Now consider C.I. View:

Class Example 13http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg13.xls

Explore idea:

CI should cover 90% of time


Class Example 13


Class Example 13


Class Example 13


Class Example 13


Class Example 13


Class Example 13

Q1: Summarize Coverage


Class Example 13


94% > 90%

(since sd too small)


Class Example 13


77% < 90%

(since too

variable)


Class Example 13


77% < 90%

(since too

biased)


Class Example 13


87% ≈ 90%

(seems OK?)


Class Example 13

Simulate from

Binomial

87% ≈ 90%

(shows within

expected

range)


Class Example 13:http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg13.xls

Q1: SD too small Too many cover

Q2: SD too big Too few cover

Q3: Big Bias Too few cover

Q4: Good sampling About right

Q5: Simulated Bi Shows “natural var’n”


HW: 6.20 ($1260, $1540), 6.21

6.28 (but use Excel & make histogram)

Research Corner

Another SiZer analysis:

British Incomes Data

Research Corner



o Annual Survey (1985)

Research Corner




o Done in Great Britain

Research Corner





o Variable of Interest:

Family Income

Research Corner





o Variable of Interest:

Family Income

o Distribution?

Research Corner


SiZer Results:

1 bump at coarse scale

(expected)

Research Corner


SiZer Results:


2 bumps at medium scale

Research Corner


SiZer Results:



(Quite a radical

statement)

Research Corner


SiZer Results:



Finer scale bumps not

statistically significant

Research Corner



Usual models for Incomes

(one bump only)

Research Corner




(one bump only)

2 bumps were verified

Research Corner




(one bump only)


(in PhD dissertation)

Research Corner




(one bump only)


(in PhD dissertation)

But when worth looking?

Next time

• Add multiple year plots as well

In:

IncomesAllKDE.mpg

Deeper look at Inference

Recall: “inference” = CIs and Hypo Tests



Main Issue: In sampling distribution

nNX /,0~




Usually σ is unknown

nNX /,0~




Usually σ is unknown, so replace with an estimate, s.

nNX /,0~





For n large, should be “OK”

nNX /,0~





For n large, should be “OK”, but what about:

• n small?

nNX /,0~





For n large, should be “OK”, but what about:

• n small?

• How large is n “large”?

nNX /,0~

Unknown SD

Goal: Account for “extra variability in the

s ≈ σ approximation”

Unknown SD



Mathematics: Assume individual ,~ NX i

Unknown SD



Mathematics: Assume individual

I.e.

• Data have mound shaped histogram

,~ NX i

Unknown SD




I.e.


• Recall averages generally normal

,~ NX i

Unknown SD




I.e.


• Recall averages generally normal

• But now must focus on individuals

,~ NX i

Unknown SD

Then nNX /,~

Unknown SD

Then

So can write:

nNX /,~

1,0~ N

n

X

Unknown SD

Then

So can write:

(recall: standardization (Z-score)

idea)

nNX /,~

1,0~ N

n

X

Unknown SD

Then

So can write:

(recall: standardization (Z-score)

idea,

used in an important way here)

nNX /,~

1,0~ N

n

X

Unknown SD

Then

So can write:

Replace

nNX /,~

1,0~ N

n

X

Unknown SD

Then

So can write:

Replace by

nNX /,~

1,0~ N

n

X

s

Unknown SD

Then

So can write:

Replace by , then

nNX /,~

1,0~ N

n

X

sn

sX

Unknown SD

Then

So can write:

Replace by , then

has a distribution named

nNX /,~

1,0~ N

n

X

sn

sX

Unknown SD

Then

So can write:

Replace by , then

has a distribution named:

“t-distribution with n-1 degrees of freedom”

nNX /,~

1,0~ N

n

X

sn

sX

t - Distribution

Notes:

1. n is a parameter

t - Distribution

Notes:

1. n is a parameter (like ),,, p

t - Distribution

Notes:

1. n is a parameter (like )

(Recall: these index families

of probability distributions)

,,, p

t - Distribution

Notes:

1. n is a parameter (like ) that controls “added variability that comes from the s ≈ σ approximation”

,,, p

t - Distribution

Notes:

1. n is a parameter (like ) that controls “added variability that comes from the s ≈ σ approximation”

View: Study Densities,

over degrees of freedom…http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/EgTDist.mpg

,,, p

t - Distribution

Compare N(0,1) distribution, to t-distribution,

d.f. = 7

t - Distribution


d.f. = 7

t - Distribution


d.f. = 7

t is more spread

t - Distribution


d.f. = 7

t is more spread:

- Lower Peak

t - Distribution


d.f. = 7

t is more spread:

- Lower Peak

- Fatter Tails

t - Distribution


d.f. = 7

t is more spread

smaller 5%-tile

t - Distribution


d.f. = 7

t is more spread

smaller 5%-tile

larger 99%-tile

t - Distribution


d.f. = 7

t is more spread

Makes sense,

since s ≈ σ

more variation

t - Distribution


d.f. = 3

All effects are

magnified

Since s ≈ σ approx

gets worse

t - Distribution


d.f. = 1

Extreme Case

Have terrible

s ≈ σ approx

t - Distribution


d.f. = 7

Now try larger d.f.

t - Distribution


d.f. = 14

All approximations

are better

t - Distribution


d.f. = 25

Even better

t - Distribution


d.f. = 25

Even better

- Densities almost

on top

t - Distribution


d.f. = 25

Even better

- Densities almost

on top

- Quantiles very

close

t - Distribution


d.f. = 100

Hard to see any

difference

t - Distribution


d.f. = 100

Hard to see any

difference

Since excellent

s ≈ σ approx

t - Distribution

Notes:

2. Careful: set “degrees of freedom” =

= n – 1

t - Distribution

Notes:


= n – 1 (not n)

t - Distribution

Notes:


= n – 1 (not n)

• Easy to forget later

t - Distribution

Notes:


= n – 1 (not n)

• Easy to forget later

• Good to add to sheet of notes for exam

t - Distribution

Notes:

3. Must work with standardized version of

X

t - Distribution

Notes:


i.e.

nsX X

t - Distribution

Notes:


i.e.

(will affect how we compute probs….)

nsX X

t - Distribution

Notes:


i.e.

• No longer can plug mean and SD

into EXCEL formulas

nsX X

t - Distribution

Notes:


i.e.


into EXCEL formulas

• In text standardization was already done

nsX X

t - Distribution

Notes:


i.e.


into EXCEL formulas

• In text standardization was already done,

since used in Normal table calc’ns

nsX X

t - Distribution

Notes:

4. Calculate t probs (e.g. areas & cutoffs),

t - Distribution

Notes:


using TDIST

t - Distribution

Notes:


using TDIST & TINV

t - Distribution

Notes:


using TDIST & TINV

Caution: these are set up differently from NORMDIST & NORMINV

t - Distribution

Notes:


using TDIST & TINV

Caution: these are set up differently from NORMDIST & NORMINV

See Class Example 14http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg14.xls

t - Distribution

Class Example 14:

t - Distribution

Class Example 14:

Calculate Upper Prob

t - Distribution

Class Example 14:


t - Distribution

Class Example 14:


Using TDIST

t - Distribution

Class Example 14:


Using TDIST

(Check TDIST menu)

t - Distribution

Class Example 14:


Using TDIST

- cutoff

t - Distribution

Class Example 14:


Using TDIST

- cutoff

- d. f.

t - Distribution

Class Example 14:


Using TDIST

- cutoff

- d. f.

- upper prob. only

t - Distribution

Class Example 14:

Careful: opposite from NORMDIST

Using TDIST

- cutoff

- d. f.

- upper prob. only

t - Distribution

Class Example 14:


use upper

Using TDIST

- cutoff

- d. f.

- upper prob. only

t - Distribution

Class Example 14:


use upper, NOT lower probs

Using TDIST

- cutoff

- d. f.

- upper prob. only

t - Distribution

Class Example 14:

To compute lower prob

t - Distribution

Class Example 14:

To compute lower prob

Use “1 – trick”, i.e. Not Rule of probability

t - Distribution

Class Example 14:

How about upper prob of negative?

t - Distribution

Class Example 14:


Give it a try

t - Distribution

Class Example 14:


Give it a try

Get an error message in response

t - Distribution

Class Example 14:


Give it a try

Get an error message in response

(Click this for sometimes useful info)

t - Distribution

Class Example 14:

Reason:

TDIST tuned for 2-tailed

t - Distribution

Class Example 14:

Reason:


(where need cutoff > 0)

t - Distribution

Class Example 14:

Reason:


(where need cutoff > 0)

(correct version for CIs and H. tests)

t - Distribution

Class Example 14:

Approach:

t - Distribution

Class Example 14:

Approach:

Use “1 – trick”

t - Distribution

Class Example 14:

Approach:


(to write as prob. can compute)

t - Distribution

Class Example 14:

For Two-Tailed Prob

t - Distribution

Class Example 14:

For Two-Tailed Prob

TDIST is very convenient

t - Distribution

Class Example 14:

For Two-Tailed Prob

TDIST is very convenient

(much better than NORMDIST)

t - Distribution

Class Example 14:

For Interior Prob

t - Distribution

Class Example 14:

For Interior Prob


t - Distribution

Class Example 14:

For Interior Prob


TDIST again very convenient

t - Distribution

Class Example 14:

For Interior Prob


TDIST again very convenient

(again better than NORMDIST)

t - Distribution

Class Example 14:

Now try increasing d.f.

t - Distribution

Class Example 14:


Big difference for small n

t - Distribution

Class Example 14:



But converges for larger n

t - Distribution

Class Example 14:




To Normal(0,1)

t - Distribution

Class Example 14:




To Normal(0,1)

(as expected)

t - Distribution

HW: C23

For T ~ t, with degrees of freedom:

(a) 3 (b) 12 (c) 150 (d) N(0,1)

Find:

i. P{T> 1.7} (0.094, 0.057, 0.046, 0.045)

ii. P{T < 2.14} (0.939, 0.973, 0.983, 0.984)

iii. P{T < -0.74} (0.256, 0.237, 0.230, 0.230)

iv. P{T > -1.83} (0.918, 0.954, 0.965, 0.966)

t - Distribution

HW: C23

v. P{|T| > 1.18} (0.323, 0.261, 0.240, 0.238)

vi. P{|T| < 2.39} (0.903, 0.966, 0.982, 0.983)

vii. P{|T| < -2.74} (0, 0, 0, 0)

And now for something completely different

“Thinking Outside the Box”

Also Called:“Lateral Thinking”


Find the word or simple phrase suggested:

death ..... life



death ..... life

life after death



ecnalg



ecnalg

backward

glance



He's X himself



He's X himself

He's by himself



THINK



THINK

think big ! !



ababaaabbbbaaaabbbb ababaabbaaabbbb. .



ababaaabbbbaaaabbbb ababaabbaaabbbb. .

long time no 'C'

t - Distribution

Class Example 14:

Next explore TINV

(Inverse function)

t - Distribution

Class Example 14:

Next explore TINV

(Inverse function)

(Given cutoff, find area)

t - Distribution

Class Example 14:

Next explore TINV

t - Distribution

Class Example 14:

Next explore TINV

Given prob. (area)

t - Distribution

Class Example 14:

Next explore TINV

Given prob. (area) & d.f.

t - Distribution

Class Example 14:

Next explore TINV

Given prob. (area) & d.f., find cutoff

t - Distribution

Class Example 14:

Next explore TINV

Given prob. (area) & d.f., find cutoff

(next think carefully about interpretation)

t - Distribution

Class Example 14:

Next explore TINV

Recall TDIST e.g. from above:

t - Distribution

Class Example 14:

Next explore TINV


Now invert this,

t - Distribution

Class Example 14:

Next explore TINV


Now invert this, i.e. given prob.

t - Distribution

Class Example 14:

Next explore TINV


Now invert this, i.e. given prob., find cutoff

t - Distribution

Class Example 14:

Next explore TINV


For same d.f.

t - Distribution

Class Example 14:

Next explore TINV


For same d.f., use resulting prob. as input

t - Distribution

Class Example 14:

Next explore TINV


For same d.f., use resulting prob. as input

But new answer is different

t - Distribution

Class Example 14:

Next explore TINV


Maybe due to rounding?

t - Distribution

Class Example 14:

Next explore TINV


Maybe due to rounding? Try exact value

t - Distribution

Class Example 14:

Next explore TINV



t - Distribution

Class Example 14:

Next explore TINV



Still get wrong answer

t - Distribution

Class Example 14:

Next explore TINV

Reason for inconsistency:

t - Distribution

Class Example 14:

Next explore TINV


Works via 2-tailed

t - Distribution

Class Example 14:

Next explore TINV


Works via 2-tailed, not 1-tailed, probability

t - Distribution

Class Example 14: Explore TINV


t - Distribution



Check by inverting 2-tailed answer above:

t - Distribution




t - Distribution




Get:

t - Distribution




Get: plug in above output

t - Distribution




Get: plug in above output, to return to input

EXCEL Functions

Summary:

Normal:

EXCEL Functions

Summary:

Normal:

plug in: get out:

EXCEL Functions

Summary:

Normal:

plug in: get out:

NORMDIST: cutoff

EXCEL Functions

Summary:

Normal:

plug in: get out:

NORMDIST: cutoff area

EXCEL Functions

Summary:

Normal:

plug in: get out:


NORMINV: area

EXCEL Functions

Summary:

Normal:

plug in: get out:


NORMINV: area cutoff

EXCEL Functions

Summary:

Normal:

plug in: get out:


NORMINV: area cutoff

(but TDIST is set up really differently)

EXCEL Functionst distribution:

1 tail:


1 tail:

plug in: get out:


1 tail:

plug in: get out:

TDIST: cutoff


1 tail:

plug in: get out:

TDIST: cutoff area


1 tail:

plug in: get out:

TDIST: cutoff area

EXCEL notes: - no explicit inverse


1 tail:

plug in: get out:

TDIST: cutoff area

EXCEL notes: - no explicit inverse

- backwards from Normal…

EXCEL Functions

t distribution:

2 tail:

EXCEL Functions

t distribution:

2 tail:

plug in: get out:

EXCEL Functions

t distribution:

2 tail:

plug in: get out:

TDIST: cutoff

EXCEL Functions

t distribution:

Area

2 tail:

plug in: get out:

TDIST: cutoff area

EXCEL Functions

t distribution:

Area

2 tail:

plug in: get out:

TDIST: cutoff area

TINV: area

EXCEL Functions

t distribution:

Area

2 tail:

plug in: get out:

TDIST: cutoff area

TINV: area cutoff

EXCEL Functions

t distribution:

Area

2 tail:

plug in: get out:

TDIST: cutoff area

TINV: area cutoff

(EXCEL note: this one has the inverse)

EXCEL Functions

Note: when need to invert the 1-tail TDIST,

Use twice the area.

EXCEL Functions


Use twice the area.

Area = A

EXCEL Functions


Use twice the area.

Area = A Area = 2 A

t - Distribution

HW: C23 (cont.)

viii. C so that 0.05 = P{|T| > C}

(3.18, 2.17, 1.98, 1.96)

ix. C so that 0.99 = P{|T| < C}

(5.84, 3.05, 2.61, 2.58)

Administrative Matters Midterm II Results Take max of two midterm scores:

Documents