Expected Value (Mean), Variance, Independence Transformations of Random Variables Last Time:

Expected Value (Mean), Variance,Expected Value (Mean), Variance,

IndependenceIndependence

Transformations of Random VariablesTransformations of Random Variables

Last Time: Last Time:

Law of Large Numbers

statistics sampleOther

:Mean Sample X

,~ NX

X

:better andbetter mean population the

esapproximatmean sample then the

:grows size sample theIf

n

X

Probability distribution of a die1/6 1/6 1/6 1/6 1/6 1/6

1 2 3 4 5 6

X

Expected value of X is 3.5

Expected Value of a RV

x

X xXPxXE

of values possible all

)()(

A measure of the center of the distribution

Note: The expected value need not be an eventExample:

The expected value of a die is 1(1/6)+2(1/6)+3(1/6)+4(1/6)+5(1/6)+6(1/6) = 21/6 = 3.5

which is not a possible outcome (of a die roll)

Example: One Coin Toss, No Crying

“We’ll toss a coin once. If it is heads, you get $10 million. If it is tails, you’ll have to pay me $1 million”

0

0.1

0.2

0.3

0.4

0.5

0.6

-2M

-1M 0M 1M 2M 3M 4M 5M 6M 7M 8M 9M 10M

11M

X



How much do you expect to get after the coin toss?

50% chance for $10 million50% chance for -$1 million

$4.5 million on average

Recall: X denotes the change in your net value (in millions of dollars) after the coin toss.

So, the expected value of X is

X = (-1) ·P(X= -1) + (10) ·P(X= 10)

= (-1) · (.5) + (10) · (.5) = -.5 + 5 = 4.5



0

0.1

0.2

0.3

0.4

0.5

0.6

-2M

-1M 0M 1M 2M 3M 4M 5M 6M 7M 8M 9M 10M

11M

X

4.500.000 = (.5)(-1.000.000) + (.5)(10.000.000)

Fair Games:

1,1in values takes that Suppose X

21

21 1P,1P that Suppose XX

11P11PX XX

011 21

21

X

Fair Game:Expected Value of Zero,

i.e., you `expect’ to neither win nor lose (on average)



0

0.1

0.2

0.3

0.4

0.5

0.6

-2M

-1M 0M 1M 2M 3M 4M 5M 6M 7M 8M 9M 10M

11M

X

4.500.000 = (.5)(-1.000.000) + (.5)(10.000.000)

This game isbetter than fair!

But…VERY risky!!

Variance of a RV

xX

XX

xXPx

X

possible all

2

22

)()(

of Value Expected

A measure of the spread of the distribution(Some people, e.g., on Wall Street, use this as a measure of risk)

Standard Deviation of a RV

XXX of Variance2

A measure of the spread of the distributionin original units

$in measured

$in measured

$in measured

$in measured

X

22X

X

X

Probability distribution of the squared deviations for a die

1/3 1/3 1/3

0.25

0.75

1.25

1.75

2.25

2.75

3.25

3.75

4.25

4.75

5.25

5.75

6.25

1/6 1/6 1/6 1/6 1/6 1/6

1 2 3 4 5 6

E(X)

+/- .5

+/- 1.5

+/- 2.5

Deviations from E(X):

Understandingthe variance of a die

Variance of X is 2.91

2)( XX

)(

2)(

2)(

)(

find

)Spread"(" )()(find

)Average"(" )()(find

Xg

xXgXg

xXg

xXPxg

xXPxg

Puzzle: for general functions g

Sorry!No simple formula for general functions g!

Formulae for the linear case: g(x) = ax+b

XbaX

2X

22baX shift)by unaffected (Spread

matters)(Shift

a

a

ba XbaX

Formulae for linear functions in two variables

cba

YX

YX cbYaX

then, variablesrandom are , If

Joint Random Variables.Joint Random Variables.wrapping up some loose ends wrapping up some loose ends

with probability and random variableswith probability and random variables

e.g. Bayes’ Theoreme.g. Bayes’ Theorem

Today: Today:

JOINTLY DISTRIBUTED RANDOM VARIABLESJOINTLY DISTRIBUTED RANDOM VARIABLES

values of two (or more) random variables might be interrelated

Examples:• (X,Y) = (height, weight) (person)

• (X,Y) = (ft.2, # bedrooms) (house) • (X,Y) = (risk, return) (investment)

• (X,Y) = (Homework Score, Exam 1 Score) • (X,Y) = (Amount of sleep before exam, Score on Exam)

JOINTLY DISTRIBUTED RANDOM VARIABLESJOINTLY DISTRIBUTED RANDOM VARIABLES

assigning two (or more) numerical values to each outcome

Joint distribution of X and Y:

Discrete case: for each pair of values (x,y) have to specify

Joint probability: P(X=x,Y=y)

Note: P(X=x,Y=y) = P({X=x}{Y=y})

Note:

Continuous case: for each pair of values (x,y) have to specify

Joint density: f(x,y)

),(

1),(yxall

yYxXP

MARGINAL PROBABILITIESMARGINAL PROBABILITIES (discrete case)

How to calculate the probability distribution of X (or of Y) from thejoint probability distribution:

• Marginal probability for any value x0 of X is:

•Marginal probability for any value y0 of Y is:

xall

yYxXPyYP ),()( 00

yall

yYxXPxXP ),()( 00

CONDITIONAL PROBABILITYCONDITIONAL PROBABILITY (nothing new)

)(

}){}({)|(

yYP

yYxXPyYxXP

)(

}){}({)|(

xXP

yYxXPxXyYP

INDEPENDENCE OF RANDOM VARIABLESINDEPENDENCE OF RANDOM VARIABLES

Recall: events A and B are independent if any of the following holds:

• P(AB)=P(A)·P(B) • P(A|B)=P(A) • P(B|A)=P(B)

Random variables X and Y are independent if, for all possible values x of X and y of Y, events {X=x} and {Y=y} are independent

In other words,

X and Y are independent if and only if for all x and y:

P(X=x,Y=y) = P(X=x) ·P(Y=y)

INDEPENDENT RANDOM VARIABLESINDEPENDENT RANDOM VARIABLES

X and Y are independent if and only if for all x and y:

P(X=x,Y=y) = P(X=x) ·P(Y=y)

Good news:

If X and Y are independent then:

222)( YXYX

222)( YXYX

REMEMBER: INDEPENDENCEREMEMBER: INDEPENDENCETwo events are independent if the information about one of them occurring (or not occurring) does not change the probability of the other one.

In other words, A and B are independent if P(A|B) = P(A)

How to recognize independent events?How to recognize independent events?

A and B are independent if any of the following is true:

P(A|B)=P(A)

P(B|A)=P(B)

P(AB)=P(A)·P(B)

(If any of the above equalities is true, then all three are true)

Let’s take a deck of cards and draw a (single) card (just once) at random from the deck.

The sample space S is the set of all cards.Let A be the set of jacks, i.e., the event that I draw a jack

Let B be the set of face cards, i.e., the event that I draw a face card.

P(A)

P(B)

If A occurs, what is the (conditional) probability that B occurs?

If B occurs, what is the (conditional) probability that A occurs?

Are the events A,B independent?

= 1/13

= 3/13

= 1

=1/3

No


The sample space S is the set of all cards.Let A be the set of red cards, i.e., the event that I draw a red card


P(A)

P(B)

If A occurs, what is the (conditional) probability that B occurs?

If B occurs, what is the (conditional) probability that A occurs?

Are the events A, B independent?

= 1/2

= 3/13

= 3/13

=1/2

Yes.

REMEMBER: THE UNIONREMEMBER: THE UNION

of events A and B is the event consisting of all outcomes that are in A or B (or both).

Notation: AB (book notation: A or B)

A union B

P(A B) = P(A) + P(B) - P(AB)

A BA B

THE UNION OF THE UNION OF DISJOINTDISJOINT

(MUTUALLY EXCLUSIVE) EVENTS(MUTUALLY EXCLUSIVE) EVENTS

P(A B) = P(A) + P(B) - P(AB)

A

B

A

B

Here: P(A B) = P(A) + P(B) - 0

Here: P(A B C) = P(A) + P(B) + P(C)

Some Rules of Probability 10 AP 1SP

BPAPBAPBA then , If

APAP c 1

CPBPAPCBAP

CBCABA

then

,, If

BAPBPAPBAP

BA

then

, If

BPAPBAP

BA

then

t independen are , If

t?independennot

are , ifWhat BA

More formulae:

• P(B|A) = =

Thus, P(B|A) is not the same as P(A|B).

• P(AB) = P(A|B)·P(B)

• P(AB) = P(B|A)·P(A)

CONDITIONAL PROBABILITYCONDITIONAL PROBABILITY

P(A)A)P(B

P(A)B)P(A

)()()|(

BPBAPBAP

AIDS Testing Example

ELISA test: + : HIV positive

- : HIV negative

Correctness: 99% on HIV positive person (1% false negative)

95% on HIV negative person

(5% false alarm)

Mandatory ELISA testing for people applying for marriage licenses in MA.

“low risk” population: 1 in 500 HIV positive

Suppose a person got ELISA = +.Q: HIV positive?

What we know: present) when antibodies HIVdetect o(Ability t 99.| HIVP

absent) when antibodies HIVreject (Ability 95.| noHIVP

)PopulationRisk (Low 500

1HIVP

test)positivegiven present antibodies (HIV ?| HIVP

|

P

HIVPHIVP

?HIVP ?P

P(HIV +) = P(+ | HIV) P(HIV)= (.99) (.002)= .00198

P(+) = P(+ HIV) + P(+ noHIV)= P(+ | HIV) P(HIV)

+ P(+ | noHIV) P(noHIV)

= .00198 + (.05)(.998)= .00198 + .0499 = .05188

|

P

HIVPHIVP

.03816 05188.

00198.| HIVP

The probability that a person with a positive ELISA test result actually has the HIV anti-bodies

is less than 4%

Number ofPeople with

HIV and positive result

Number ofPeople with

falsepositive result

Conclusion:• False Positive overwhelms Correct

Positive

• Lesson to learn:

• When it comes to conditional probabilities

on rare events (with updated information) do not trust your intuition.

• Conditional Probabilities can be counter-intuitive

AIDS TestingSensitivity Analysis

(Relax, this is just for illustration.)

0.9

9

0.9

5

0.9

1

0.8

7

0.8

3 0.99

0.870

0.2

0.4

0.6

0.8

1

P(HIV | +)

P(- | no HIV)

P(+ | HIV)

P(HIV | +) with Base Rate 1/1000

0.9

9

0.9

5

0.9

1

0.8

7

0.8

3 0.99

0.870

0.2

0.4

0.6

0.8

1

P(HIV | +)

P(- | no HIV)

P(+ | HIV)


0.9

9

0.9

5

0.9

1

0.8

7

0.8

3 0.99

0.870

0.2

0.4

0.6

0.8

1

P(HIV | +)

P(- | no HIV)

P(+ | HIV)


0.9

9

0.9

5

0.9

1

0.8

7

0.8

3 0.99

0.870

0.2

0.4

0.6

0.8

1

P(HIV | +)

P(- | no HIV)

P(+ | HIV)


0.9

9

0.9

5

0.9

1

0.8

7

0.8

3 0.99

0.870

0.2

0.4

0.6

0.8

1

P(HIV | +)

P(- | no HIV)

P(+ | HIV)

P(HIV | +) with Base Rate .025

0.9

9

0.9

5

0.9

1

0.8

7

0.8

3 0.99

0.870

0.2

0.4

0.6

0.8

1

P(HIV | +)

P(- | no HIV)

P(+ | HIV)


0.9

9

0.9

5

0.9

1

0.8

7

0.8

3 0.99

0.870

0.2

0.4

0.6

0.8

1

P(HIV | +)

P(- | no HIV)

P(+ | HIV)


0.9

9

0.9

5

0.9

1

0.8

7

0.8

3 0.99

0.870

0.2

0.4

0.6

0.8

1

P(HIV | +)

P(- | no HIV)

P(+ | HIV)


0.9

9

0.9

5

0.9

1

0.8

7

0.8

3 0.99

0.870

0.2

0.4

0.6

0.8

1

P(HIV | +)

P(- | no HIV)

P(+ | HIV)


0.9

9

0.9

5

0.9

1

0.8

7

0.8

3 0.99

0.870

0.2

0.4

0.6

0.8

1

P(HIV | +)

P(- | no HIV)

P(+ | HIV)


0.9

9

0.9

5

0.9

1

0.8

7

0.8

3 0.99

0.870

0.2

0.4

0.6

0.8

1

P(HIV | +)

P(- | no HIV)

P(+ | HIV)


Enough Fun! It’s time to work!

?|out figureyou then Can

|,|, knowyou that Suppose

BAP

ABPABPAP c

Example: A is HIV B is +

P(A) = 1/500P(B | A) = .99P(B | notA) = .05

Bayes’ Theorem

cc APABPAPABP

APABPBAP

||

||

…some peoplemake a living outof this formula

Try Michael Birnbaum’s (former UIUC psych faculty) Bayesian calculatorhttp://psych.fullerton.edu/mbirnbaum/bayes/BayesCalc.htm

http://psych.fullerton.edu/mbirnbaum/bayes/BayesCalc.htm

Bayes’ Theorem

cc APABPAPABP

APABPBAP

||

||

notHIVPnotHIVPHIVPHIVP

HIVPHIVPHIVP

||

||

500

4995001

5001

05.99.

99.|

HIVP

038.| HIVP 99.| HIVP

Bayes’ Theorem

cc APABPAPABP

APABPBAP

||

||


The sample space S is the set of all cards.Let A be the set of jacks, i.e., the event that I draw a jack


P(A|B) = 1/3 P(B|A) = 1

An Excursion into Logic…

If you are an undergraduate major in Psychology at UIUC, then you have to complete

a course requirement in Statistics.

“If p, then q”therefore

“If not q, then not p” Contrapositive

An Excursion into Logic…If you don’t have to complete

a course requirement in Statistics,then you are not an undergraduate major in

Psychology at UIUC.

“If p, then q”therefore

“If not q, then not p” Contrapositive


If you are not an undergraduate major in Psychology at UIUC, then you do not have to complete a course requirement in Statistics.

“If p, then q”does not imply

“if not p, then not q”

FALLACY!!(denying the antecedent)


If you have to complete a course requirement in Statistics then you are

an undergraduate major in Psychology at UIUC.

“If p, then q”does not imply“If q, then p”

FALLACY!!(affirming the consequent)

Probabilistic Reasoning…

BPABP |

APBAP | implies

)'antecedent (`denying

| also implies BPABP

)'consequent g(`affirmin

| and APBAP

Expected Value (Mean), Variance, Independence Transformations of Random Variables Last Time:

Documents

y slide

value y

expected value of x

exam slide

score x

average slide

probability distribution

new slide