Probability

1

MEL761: Statistics for Decision Making

Dr S G DeshmukhMechanical Department

Indian Institute of Technology

Probability

• Classical view• Experimental view• Various rules• Examples

2

Learning Objectives

• Comprehend the different ways of assigning probability.

• Understand and apply marginal, union, joint, and conditional probabilities.

• Select the appropriate law of probability to use in solving problems.

• Solve problems using the laws of probability including the laws of addition, multiplication and conditional probability

• Revise probabilities using Bayes’ rule.

3

Methods of Assigning Probabilities

• Classical method of assigning probability (rules and laws)

• Relative frequency of occurrence (cumulated historical data)

• Subjective Probability (personal intuition or reasoning)

4

Classical Probability• Number of outcomes leading to

the event divided by the total number of outcomes possible

• Each outcome is equally likely• Determined a priori -- before

performing the experiment• Applicable to games of chance• Objective -- everyone correctly

using the method assigns an identical probability

P EN

Where

N

en( )

:

total number of outcomes

number of outcomes in Een

5

Relative Frequency Probability

• Based on historical data

• Computed after performing the experiment

• Number of times an event occurred divided by the number of trials

• Objective -- everyone correctly using the method assigns an identical probability

P EN

Where

N

en( )

:

total number of trials

number of outcomes

producing Een

6

Subjective Probability• Comes from a person’s intuition or

reasoning• Subjective -- different individuals may

(correctly) assign different numeric probabilities to the same event

• Degree of belief• Useful for unique (single-trial)

experiments– New product introduction– Initial public offering of common stock– Site selection decisions– Sporting events

7

Structure of Probability

• Experiment• Event• Elementary Events• Sample Space• Unions and Intersections• Mutually Exclusive Events• Independent Events• Collectively Exhaustive Events• Complementary Events

8

Experiment• Experiment: a process that produces outcomes

– More than one possible outcome– Only one outcome per trial

• Trial: one repetition of the process• Elementary Event: cannot be decomposed or

broken down into other events• Event: an outcome of an experiment

– may be an elementary event, or– may be an aggregate of elementary events– usually represented by an uppercase letter,

e.g., A, E1

9

An Example Experiment

Experiment: randomly select, without replacement, two families from the residents of Tiny Town

Elementary Event: the sample includes families A and C

Event: each family in the sample has children in the household

Event: the sample families own a total of four automobiles

Family Children in Household

Number of Automobiles

ABCD

YesYesNoYes

3212

Tiny Town Population

10

Sample Space

• The set of all elementary events for an experiment

• Methods for describing a sample space– roster or listing– tree diagram– set builder notation– Venn diagram

11

Sample Space: Example

• Experiment: randomly select, without replacement, two families from the residents of Tiny Town

• Each ordered pair in the sample space is an elementary event, for example -- (D,C)

Family Children in Household

Number of Automobiles

ABCD

YesYesNo

Yes

3212

Listing of Sample Space

(A,B), (A,C), (A,D),(B,A), (B,C), (B,D),(C,A), (C,B), (C,D),(D,A), (D,B), (D,C)

12

Sample Space: Tree Diagram for Random Sample of Two

Families A

B

C

D

D

BC

D

A

C

D

A

B

C

A

B

13

Sample Space: Set Notation for Random Sample of Two

Families

• S = {(x,y) | x is the family selected on the first draw, and y is the family selected on the second draw}

• Concise description of large sample spaces

14

Sample Space

• Useful for discussion of general principles and concepts

Listing of Sample Space

(A,B), (A,C), (A,D),(B,A), (B,C), (B,D),(C,A), (C,B), (C,D),(D,A), (D,B), (D,C)

Venn Diagram

15

Union of Sets• The union of two sets contains an

instance of each element of the two sets.

X

Y

X Y

1 4 7 9

2 3 4 5 6

1 2 3 4 5 6 7 9

, , ,

, , , ,

, , , , , , ,

C IBM DEC Apple

F Apple Grape Lime

C F IBM DEC Apple Grape Lime

, ,

, ,

, , , ,

YX

16

Intersection of Sets• The intersection of two sets contains only

those element common to the two sets.

X

Y

X Y

1 4 7 9

2 3 4 5 6

4

, , ,

, , , ,

C IBM DEC Apple

F Apple Grape Lime

C F Apple

, ,

, ,

YX

17

Mutually Exclusive Events

• Events with no common outcomes

• Occurrence of one event precludes the occurrence of the other event

X

Y

X Y

1 7 9

2 3 4 5 6

, ,

, , , ,

C IBM DEC Apple

F Grape Lime

C F

, ,

,

YX

P X Y( ) 0

18

Independent Events

• Occurrence of one event does not affect the occurrence or nonoccurrence of the other event

• The conditional probability of X given Y is equal to the marginal probability of X.

• The conditional probability of Y given X is equal to the marginal probability of Y.

P X Y P X and P Y X P Y( | ) ( ) ( | ) ( )

19

Collectively Exhaustive Events

• Contains all elementary events for an experiment

E1 E2 E3

Sample Space with three collectively exhaustive events

20

Complementary Events

• All elementary events not in the event ‘A’ are in its complementary event.

SampleSpace A

P Sample Space( ) 1

P A P A( ) ( ) 1A

21

Counting the Possibilities

• mn Rule

• Sampling from a Population with Replacement

• Combinations: Sampling from a Population without Replacement

22

mn Rule

• If an operation can be done m ways and a second operation can be done n ways, then there are mn ways for the two operations to occur in order.

• A cafeteria offers 5 salads, 4 meats, 8 vegetables, 3 breads, 4 desserts, and 3 drinks. A meal is two servings of vegetables, which may be identical, and one serving each of the other items. How many meals are available?

23

Sampling from a Population with Replacement

• A tray contains 1,000 individual tax returns. If 3 returns are randomly selected with replacement from the tray, how many possible samples are there?

• (N)n = (1,000)3 = 1,000,000,000

24

Combinations

• A tray contains 1,000 individual tax returns. If 3 returns are randomly selected without replacement from the tray, how many possible samples are there?

0166,167,00)!31000(!3

!1000

)!(!

!

nNn

N

n

N

25

Four Types of Probability

• Marginal Probability

• Union Probability

• Joint Probability

• Conditional Probability

26

Four Types of Probability

Marginal

The probability of X occurring

Union

The probability of X or Y occurring

Joint

The probability of X and Y occurring

Conditional

The probability of X occurring given that Y has occurred

YX YX

Y

X

P X( ) P X Y( ) P X Y( ) P X Y( | )

27

General Law of Addition

P X Y P X P Y P X Y( ) ( ) ( ) ( )

YX

28

General Law of Addition -- Example

P N S P N P S P N S( ) ( ) ( ) ( )

SN

.56 .67.70

P N

P S

P N S

P N S

( ) .

( ) .

( ) .

( ) . . .

.

70

67

56

70 67 56

0 81

29

Office Design ProblemProbability Matrix

.11 .19 .30

.56 .14 .70

.67 .33 1.00

Increase Storage SpaceYes No Total

Yes

No

Total

Noise Reduction

30

Office Design ProblemProbability Matrix

.11 .19 .30

.56 .14 .70

.67 .33 1.00

Increase Storage Space

Yes No TotalYes

No

Total

Noise Reduction

P N S P N P S P N S( ) ( ) ( ) ( )

. . .

.

70 67 56

81

31

Office Design ProblemProbability Matrix

.11 .19 .30

.56 .14 .70

.67 .33 1.00

Increase Storage Space

Yes No TotalYes

No

Total

Noise Reduction

P N S( ) . . .

.

56 14 11

81

32

Venn Diagram of the X or Y but not Both Case

YX

33

The Neither/Nor Region

YX

P X Y P X Y( ) ( ) 1

34

The Neither/Nor Region

SN

P N S P N S( ) ( )

.

.

1

1 81

19

35

Special Law of Addition

If X and Y are mutually exclusive,

P X Y P X P Y( ) ( ) ( )

X

Y

36

Example..

Type of GenderPosition Male Female TotalManagerial 8 3 11Professional 31 13 44Technical 52 17 69Clerical 9 22 31Total 100 55 155

P T C P T P C( ) ( ) ( )

.

69

155

31

155645

37

Example..

Type of GenderPosition Male Female TotalManagerial 8 3 11Professional 31 13 44Technical 52 17 69Clerical 9 22 31Total 100 55 155

P P C P P P C( ) ( ) ( )

.

44

155

31

155484

38

Law of Multiplication

P X Y P X P Y X P Y P X Y( ) ( ) ( | ) ( ) ( | )

P M

P S M

P M S P M P S M

( ) .

( | ) .

( ) ( ) ( | )

( . )( . ) .

80

1400 5714

0 20

0 5714 0 20 0 1143

39

Law of MultiplicationExample..

Total

.7857

Yes No

.4571 .3286

.1143 .1000 .2143

.5714 .4286 1.00

Married

YesNo

Total

Supervisor

Probability Matrixof Employees

20.0)|(

5714.0140

80)(

2143.0140

30)(

MSP

MP

SP

P M S P M P S M( ) ( ) ( | )

( . )( . ) .

0 5714 0 20 0 1143

P M S P M P M S

P M S P S P M S

P M P M

( ) ( ) ( )

. . .

( ) ( ) ( )

. . .

( ) ( )

. .

0 5714 0 1143 0 4571

0 2143 0 1143 0 1000

1

1 0 5714 0 4286

P S P S

P M S P S P M S

( ) ( )

. .

( ) ( ) ( )

. . .

1

1 0 2143 0 7857

0 7857 0 4571 0 3286

40

Special Law of Multiplication for Independent Events

• General Law

• Special Law

P X Y P X P Y X P Y P X Y( ) ( ) ( | ) ( ) ( | )

If events X and Y are independent,

and P X P X Y P Y P Y X

Consequently

P X Y P X P Y

( ) ( | ), ( ) ( | ).

,

( ) ( ) ( )

41

Law of Conditional Probability

• The conditional probability of X given Y is the joint probability of X and Y divided by the marginal probability of Y.

P X YP X Y

P Y

P Y X P X

P Y( | )

( )

( )

( | ) ( )

( )

42

Law of Conditional Probability

NS

.56 .70

P N

P N S

P S NP N S

P N

( ) .

( ) .

( | )( )

( )

.

..

70

56

56

7080

43

Office Design Problem

164.

67.

11.

)(

)()|(

SP

SNPSNP

.19 .30

.14 .70

.33 1.00

Increase Storage SpaceYes No Total

YesNo

Total

Noise Reduction .11

.56

.67

Reduced Sample Space for “Increase

Storage Space” = “Yes”

44

Independent Events

• If X and Y are independent events, the occurrence of Y does not affect the probability of X occurring.

• If X and Y are independent events, the occurrence of X does not affect the probability of Y occurring.

If X and Y are independent events,

, and P X Y P X

P Y X P Y

( | ) ( )

( | ) ( ).

45

Independent EventsDemonstration Problem

Geographic Location

NortheastD

SoutheastE

MidwestF

WestG

Finance A .12 .05 .04 .07 .28

Manufacturing B .15 .03 .11 .06 .35

Communications C .14 .09 .06 .08 .37

.41 .17 .21 .21 1.00

P A GP A G

P GP A

P A G P A

( | )( )

( )

.

.. ( ) .

( | ) . ( ) .

0 07

0 210 33 0 28

0 33 0 28

46

Independent EventsDemonstration Problem

D E

A 8 12 20

B 20 30 50

C 6 9 15

34 51 85

P A D

P A

P A D P A

( | ) .

( ) .

( | ) ( ) .

8

342353

20

852353

0 2353

47

Revision of Probabilities: Bayes’ Rule

• An extension to the conditional law of probabilities

• Enables revision of original probabilities with new information

P X YP Y X P X

P Y X P X P Y X P X P Y X P Xi

i i

n n( | )

( | ) ( )

( | ) ( ) ( | ) ( ) ( | ) ( )

1 1 2 2

48

Revision of Probabilities with Bayes' Rule: Example

447.0)35.0)(12.0()65.0)(08.0(

)35.0)(12.0(

)()|()()|(

)()|()|(

553.0)35.0)(12.0()65.0)(08.0(

)65.0)(08.0(

)()|()()|(

)()|()|(

12.0)|(

08.0)|(

35.0)(

65.0)(

SPSdPAPAdP

SPSdPdSP

SPSdPAPAdP

APAdPdAP

SdP

AdP

SP

AP

49

Revision of Probabilities with Bayes’ Rule:

Conditional Probability

0.052

0.042

0.094

0.65

0.35

0.08

0.12

0.0520.094

=0.553

0.0420.094

=0.447

Alamo

South Jersey

Event

Prior Probability

P Ei( )

Joint Probability

P E di( )

Revised Probability

P E di( | )P d Ei( | )

Conditional Probability

0.052

0.042

0.094

0.65

0.35

0.08

0.12

0.0520.094

=0.553

0.0420.094

=0.447

A

S

Event

Prior Probability

P Ei( )

Joint Probability

P E di( )

Revised Probability

P E di( | )P d Ei( | )

50

Revision of Probabilities with Bayes' Rule:Example

A0.65

S0.35

Defective0.08

Defective0.12

Acceptable0.92

Acceptable0.88

0.052

0.042

+ 0.094

51

Probability for a Sequence of Independent Trials

• 25 percent of a bank’s customers are commercial (C) and 75 percent are retail (R).

• Experiment: Record the category (C or R) for each of the next three customers arriving at the bank.

• Sequences with 1 commercial and 2 retail customers.

– C1 R2 R3

– R1 C2 R3

– R1 R2 C3

52

Probability for a Sequenceof Independent Trials

• Probability of specific sequences containing 1 commercial and 2 retail customers, assuming the events C and R are independent

P C R R P C P R P R

P R C R P R P C P R

P R R C P R P R P C

( ) ( ) ( ) ( )

( ) ( ) ( ) ( )

( ) ( ) ( ) ( )

1 2 3

1 2 3

1 2 3

1

4

3

4

3

4

9

64

3

4

1

4

3

4

9

64

3

4

3

4

1

4

9

64

53

Probability for a Sequence of Independent Trials

• Probability of observing a sequence containing 1 commercial and 2 retail customers, assuming the events C and R are independent

P C R R R C R R R C

P C R R P R C R P R R C

( ) ( ) ( )

( ) ( ) ( )

1 2 3 1 2 3 1 2 3

1 2 3 1 2 3 1 2 3

9

64

9

64

9

64

27

64

54

Probability for a Sequence of Independent Trials

• Probability of a specific sequence with 1 commercial and 2 retail customers, assuming the events C and R are independent

• Number of sequences containing 1 commercial and 2 retail customers

• Probability of a sequence containing 1 commercial and 2 retail customers

P C R R P C P R P R ( ) ( ) ( )9

64

n rCn

rn

r n r

!

! !

!

! !

3

1 3 13

39

64

27

64

55

Probability for a Sequence of Dependent Trials

• Twenty percent of a batch of 40 tax returns contain errors.

• Experiment: Randomly select 4 of the 40 tax returns and record whether each return contains an error (E) or not (N).

• Outcomes with exactly 2 erroneous tax returnsE1 E2 N3 N4

E1 N2 E3 N4

E1 N2 N3 E4

N1 E2 E3 N4

N1 E2 N3 E4

N1 N2 E3 E4

56

Probability for a Sequence of Dependent Trials

• Probability of specific sequences containing 2 erroneous tax returns (three of the six sequences)

P E E N N P E P E E P N E E P N E E N

P E N E N P E P N E P E E N P N E N E

( ) ( ) ( | ) ( | ) ( | )

,

, ,.

( ) ( ) ( | ) ( | ) ( | )

1 2 3 4 1 2 1 3 1 2 4 1 2 3

1 2 3 4 1 2 1 3 1 2 4 1 2 3

8

50

7

49

32

48

31

47

55 552

5 527 2000 01

8

50

32

49

7

48

31

47

55 552

5 527 2000 01

8

50

32

49

31

48

7

47

55 552

5 527 2000 01

1 2 3 4 1 2 1 3 1 2 4 1 2 3

,

, ,.

( ) ( ) ( | ) ( | ) ( | )

,

, ,.

P E N N E P E P N E P N E N P E E N N

57

Probability for a Sequence of Independent Trials

• Probability of observing a sequence containing exactly 2 erroneous tax returns

P E E N N E N E N E N N E

N E E N N E N E N N E E

P E E N N P E N E N P E N N E

P N E E N P N E N E P N N E E

(( ) ( ) ( )

( ) ( ) ( ))

( ) ( ) ( )

( ) ( ) ( )

,

1 2 3 4 1 2 3 4 1 2 3 4

1 2 3 4 1 2 3 4 1 2 3 4

1 2 3 4 1 2 3 4 1 2 3 4

1 2 3 4 1 2 3 4 1 2 3 4

55 552

5

, ,

,

, ,

,

, ,

,

, ,

,

, ,

,

, ,

.

527 200

55 552

5 527 200

55 552

5 527 200

55 552

5 527 200

55 552

5 527 200

55 552

5 527 200

0 06

58

Probability for a Sequence of Dependent Trials

• Probability of a specific sequence with exactly 2 erroneous tax returns

• Number of sequences containing exactly 2 erroneous tax returns

• Probability of a sequence containing exactly 2 erroneous tax returns

n r r

nC

n

rn

r n rC

!

! !

!

! !

4

2 4 26

655 552

5 527 2000 06

,

, ,.

P E E N N( ),

, ,.1 2 3 4

8

50

7

49

32

48

31

47

55 552

5 527 2000 01

59

A bag contains 5 white balls & 4 black balls. One ball is drawn at random. What is the probability of drawing alternative white and black ball?

Ans:

=

Examples on Probability

121

32

42

53

63

74

84

95

1261

=

60

Examples on Probability

If on an average rain falls on 12 days in every 30 day. Find probability 1) that first 3 of a given week will be fine and

remainder wet. 2) that rain will fall on just 3 days of a given

week.

Ans. Here p= 12/30 =0.40

1) (0.40) (0.40) (0.40) (0.60)4

= (0.40)3 (0.60)4

= 0.0038 2) 7c3 (0.40)3 (0.60)4 = 0.2903

61

2

22

2 2 2

1 2 3

1

1 ( )

2 ( ) ( )

3 ( )

R R R

p x kk

x f x

x f x x f x x f x

R x k

R k x k

R x k

Chebyshev’s Inequality

62

2 22

1 3

2 2 2

1 3

2

... 1

... 3

... 1 3

( ) ( )

1[ ( ) ( )]

R R

R R

R x f x x f x

But x k inR

x k inR

or x k inR or

k f x f x

f x f xk

Chebyshev’s Inequality (Cont…)