Mth263 lecture 4

MTH 263 Probability and Random Variables

Lecture 4, Chapter 2Dr. Sobia Baig

Electrical Engineering DepartmentCOMSATS Institute of Information Technology, Lahore

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig 2

Contents

• Conditional probability• Bayes theorem• Independent Events


Basic Concepts of Probability Theory

• set theory is used to specify the sample space and the events of a random experiment

• the axioms of probability specify rules for computing the probabilities of events

• the notion of conditional probability allows us to determine how partial information about the outcome of an experiment affects the probabilities of events

• Conditional probability also allows us to formulate the notion of “independence” of events and of experiments.

• We consider “sequential” random experiments that consist of performing a sequence of simple random subexperiments.


Conditional Probability

• we are interested in determining whether two events A and B, are related in the sense that knowledge about the occurrence of one, say B, alters the likelihood of occurrence of the other A.

• This requires that we find the conditional probability, of event A given that event B has occurred.

• The conditional probability is defined by


Computing Conditional Probability



Conditional Probability• For instance, one picks a card at random from a 52-card deck. • One knows that the card is black. What is the probability that it is

the ace of clubs?

• The sensible answer is that if one only knows that the card is black, then that card is equally likely to be any one of the 26 black cards.

• Therefore, the probability that it is the ace of clubs is 1/26.

• Similarly, given that the card is black, the probability that it is an ace is 2/26, because there are 2 black aces (spades and clubs)


Examples of Conditional ProbabilityA digital communication channel has an error rate of one bit per every thousand

transmitted. Errors are rare, but when they occur, they tend to occur in bursts that affect many consecutive bits. If a single bit is transmitted, we might model the probability of an error as 1/1000. However, if the previous bit was in error, because of the bursts, we might believe that the probability that the next bit is in error is greater than 1/1000.

In a thin film manufacturing process, the proportion of parts that are not acceptable is 2%. However, the process is sensitive to contamination problems that can increase the rate of parts that are not acceptable. If we knew that during a particular shift there were problems with the filters used to control contamination, we would assess the probability of a part being unacceptable as higher than 2%.




Example• An urn contains two black balls and three white balls. Two balls are

selected at random from the urn without replacement and the sequence of colors is noted. Find the probability that both balls are black.


• Let B1 and B2 be the events that the outcome is a black ball in the first and second draw,

• respectively.


Binary Communication System• Many communication systems can be modeled in the following way.• First, the user inputs a 0 or a 1 into the system, and a corresponding signal

is transmitted.• Second, the receiver makes a decision about what was the input to the

system, based on the signal it received. • Suppose that the user sends 0s with probability (1-p)and 1s with

probability p,

Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

14

Total Probability Rule• For any event B, we can write B

as the union of the part of B in A and the part of B in A’.

Because A and A’ are mutually exclusive, and A intersection B and A’ intersection B are mutually exclusive.

• Therefore, from the probability of the union of mutually exclusive events and the Multiplication Rule total probability rule is obtained.



Total Probability Rule- Generalized to n Partitions

• Let B1, B2, … Bn be mutually exclusive events whose union equals the sample space S

• We refer to these sets as a partition of S.• Any event A can be represented as the union of mutually exclusive events

• By Corollary 4,


Example-Total Probability Rule

• in Example 2.25, find the probability of the event that the second ball is white.


Example

• A manufacturing process produces a mix of “good” memory chips and “bad” memory chips.

• The lifetime of good chips follows the exponential law introduced in Example 2.13, with a rate of failure α

• The lifetime of bad chips also follows the exponential law, but the rate of failure is 1000α

• Suppose that the fraction of good chips is 1-p and of bad chips, p.

• Find the probability that a randomly selected chip is still functioning after t seconds.


Example Solution


Bayes’ Rule

• We might know one conditional probability but would like to calculate a different one.

• From the definition of conditional probability,

• Therefore, it can be stated,


22

When to Apply Bayes’ Rule

• We have some random experiment in which the events of interest form a partition.

• The “a priori probabilities” of these events P[Bj], are the probabilities of the events before the experiment is performed.

• Now suppose that the experiment is performed, and we are informed that event A occurred; the “a posteriori probabilities” are the probabilities of the events in the partition P[Bj ], given this additional information.


23

Binary Communication System

• In the binary communication system in Example 2.26, find which input is more probable given that the receiver has output a 1.

• Assume that, a priori, the input is equally likely to be 0 or 1



25

Example- Bayes’ Rule• Let C be the event “chip still functioning after t seconds,” and let G be the

event “chip is good,” and B be the event “chip is bad.” The problem requires that we find the value of t for which



Example Quality Control

• Consider the memory chips discussed in Example 2.28.• a fraction p of the chips are bad and tend to fail much more

quickly than good chips. • Suppose that in order to “weed out” the bad chips, every chip

is tested for t seconds prior to leaving the factory. • The chips that fail are discarded and the remaining chips are

sent out to customers.• Find the value of t for which 99% of the chips sent out to

customers are good.



INDEPENDENCE OF EVENTS

• If knowledge of the occurrence of an event B does not alter the probability of some other event A, then it would be natural to say that event A is independent of B.

• In terms of probabilities this situation occurs when


• Two events, A and B are said to be independent,


Example-Independent Events

• A ball is selected from an urn containing two black balls, numbered 1 and 2, and two white balls, numbered 3 and 4.

Let the events A, B, and C be defined as follows:

• Are events A and B independent? Are events A and C independent?

31

• First, consider events A and B.• The probabilities are


32

Example Solution

• First, consider events A and B.• The probabilities are

• and the events A and B are independent.Probability and Random Variables, Lecture 4, by Dr. Sobia Baig

33

• These two equations imply that because the proportion of outcomes in S

that lead to the occurrence of A is equal to the proportion of outcomes in B that lead to A.

• Thus knowledge of the occurrence of B does not alter the probability of the occurrence of A.


34

Mutually Exclusive & Independence

• In general if two events have nonzero probability and are mutually exclusive, then they cannot be independent.

• For example, suppose they were independent and mutually exclusive; then

• which implies that at least one of the events must have zero probability.




• If two events A and B are independent, then P[A and B] = Pr[A]Pr[B]; that is, the probability that both A and B occur is equal to the probability that A occurs times the probability that B occurs.

• If A and B are mutually exclusive, • then P[A and B] = 0; that is, the probability that both

A and B occur is zero. • Clearly, if A and B are nontrivial events (Pr[A] and

Pr[B] are nonzero), then they cannot be both independent and mutually exclusive.



• Consider a fair coin and a fair six-sided die.• Let event A be obtaining heads, and event B be

rolling a 6. • Then we can reasonably assume that events A and B

are independent, because the outcome of one does not affect the outcome of the other

• The probability that both A and B occur is P[A and B] = P[A]P[B] = (1/2)(1/6) = 1/12. Since this value is not zero, then events A and B cannot be mutually exclusive.



• Consider a fair six-sided die as before, only in addition to the numbers 1 through 6 on each face, we have the property that the even-numbered faces are colored red, and the odd-numbered faces are colored green.

• Let event A be rolling a green face, and event B be rolling a 6. Then P[A] = 1/2 P[B] = 1/6 as in our previous example.

• But it is obvious that events A and B cannot simultaneously occur, since rolling a 6 means the face is red, and rolling a green face means the number showing is odd.

• Therefore Pr[A and B] = 0. Therefore, we see that a mutually exclusive pair of nontrivial events are also necessarily dependent events.

• if A and B are mutually exclusive, then if A occurs, then B cannot also occur; and vice versa. This stands in contrast to saying the outcome of A does not affect the outcome of B, which is independence of events.


38

Example• Consider the experiment

discussed in Example 2.32 where two numbers are selected at random from the unit interval. Let the events B, D, and F be defined as follows:


• It can be easily verified that any pair of these events is independent:


n Independent Events

• In order for a set of n events to be independent, the probability of an event should be unchanged when we are given the joint occurrence of any subset of the other events.

• This requirement naturally leads to the following definition of independence.


Example 2.35 System Reliability

• A system consists of a controller and three peripheral units.

• The system is said to be “up” if the controller and at least two of the peripherals are functioning.

• Find the probability that the system is up, assuming that all components fail independently.


43

Sequential Experiments• Many random experiments can be viewed as sequential experiments that

consist of a sequence of simpler subexperiments• These subexperiments may or may not be independent.

Sequences of Independent Experiments


44

• Let be events such that concerns only the outcome of the kth subexperiment.

• If the subexperiments are independent, then it is reasonable to assume that the above events are independent.

• Thus



Example


Example


Summary

• A conditional probability quantifies the effect of partial knowledge about the outcome of an experiment on the probabilities of events.

• It is particularly useful in sequential experiments where the outcomes of subexperiments constitute the “partial knowledge.”

Mth263 lecture 4

Documents

mutually exclusive

binary communication system

total probability rule

mutually exclusive events

mutually exclusive

bayes rule

sided die

sample space