COMP 182 Algorithmic Thinking Discrete Probability Luay ...nakhleh/COMP182/DiscreteProbability.pdfBernoulli Trials Suppose an experiment can have only two possible outcomes, e.g.,

COMP 182 Algorithmic Thinking

Discrete Probability Luay NakhlehComputer ScienceRice University

Reading Material

❖ Chapter 7, Sections 1-4

Experiment, Sample Space, and Event❖ An experiment is a procedure that yields one of a given

set of possible outcomes (tossing a coin, rolling a die five times, etc.).

❖ The sample space of the experiment is the set of possible outcomes (for the coin toss experiment, the sample space is {H,T}).

❖ An event is a subset of the sample space (the possible events that correspond to the coin toss experiment are {}, {H}, {T}, {H,T}).

Experiment, Sample Space, and Event

❖ For each of the following, what is the sample space? What is an example of an event? ❖ Experiment 1: Rolling a 6-sided die twice.❖ Experiment 2: Generating an Erdos-Renyi

graph with parameters n and p.

Assigning Probabilities

❖ Let S be a sample space of an experiment with finite number of outcomes. We assign a probability p(s) to every outcome s, so that

1. 0≤p(s)≤1 for each s∈S, and

2. X

s2S

p(s) = 1.

Assigning Probabilities

❖The function p from the set of all outcomes of the sample space S is called a probability distribution.

Uniform Distribution

❖The uniform distribution on a set S with n elements assigns probability 1/n to each element of S (all n elements are equally likely).

Probability of an Event: Laplace’s Definition

❖ If S is a finite sample space of equally likely outcomes, and E is an event, that is, a subset of S, then the probability of E is p(E)=|E|/|S|.

Pierre-Simon Laplace

Probability of an Event

❖ Back to the experiment of rolling a die twice:❖ What is the probability of the event “the

sum of the two numbers is 8”?


❖ When the elements of the sample space are not all equally likely, Laplace’s definition doesn’t work!

❖ The probability of an event E is then defined in terms of the probabilities of its elements.


❖ The probability of the event E is

p(E) =X

s2E

p(s)


❖ Back to the experiment of ER graphs with n and p:❖ What is the probability of the event “the

generated graph has exactly k edges”?

Combinations of Events

❖ If E1,E2,… is a sequence of pairwise disjoint events in a sample space S, then

p

[

i

Ei

!=X

i

p(Ei)

Conditional Probability

❖ Let E and F be two events with p(F)>0. The conditional probability of E given F, denoted by p(E|F), is defined as

p(E|F ) =p(E \ F )

p(F )

Independence

❖ The events E and F are independent if and only if p(E∩F)=p(E)p(F).

❖ For example, consider the space of randomly generated bit strings of length four (all 16 have the same probability), and consider: ❖ E: the string begins with 1❖ F: the string contains an even number of 1s.

Pairwise and Mutual Independence❖ The events E1,E2,…,En are pairwise

independent if and only if p(Ei∩Ej)=p(Ei)p(Ej) for all pairs i and j with i≠j.

❖ The events E1,E2,…,En are mutually independent if and only if for every subset E’⊆{E1,E2,…,En} with |E’|≥2 we have

p

\

Ei2E0

Ei

!=

Y

Ei2E0

p(Ei)

❖ Mutually independent events are also pairwise independent events.

❖ Pairwise independent events are not necessarily mutually independent events. ❖ Experiment: Toss a fair coin twice.❖ Events:

❖ E1: The first toss is H.❖ E2: The second toss is H.❖ E3: Both tosses give the same outcome.

Bernoulli Trials

❖ Suppose an experiment can have only two possible outcomes, e.g., the flipping of a coin or the random generation of a bit (recall the random ER graphs?).

❖ Each performance of the experiment is called a Bernoulli trial.

❖ One outcome is called a success and the other a failure.

❖ If p and q are the probabilities of success and failure, respectively, then p+q=1.

The Binomial Distribution B(k:n,p)

❖ The probability of exactly k successes in n independent Bernoulli trials, with probability of success p and probability of failure q=1-p, is ✓

n

k

◆pkqn�k

Bayes’ Theorem

Bayes’ Theorem

❖ Suppose that E and F are events from a sample space S such that p(E)≠0 and p(F)≠0. Then:

p(F |E) =p(E|F )p(F )

p(E|F )p(F ) + p(E|F )p(F )

Bayes’ Theorem

❖ Suppose that E and F are events from a sample space S such that p(E)≠0 and p(F)≠0. Then:

p(F |E) =p(E|F )p(F )

p(E|F )p(F ) + p(E|F )p(F )

p(E)

❖ Suppose that one person in 100,000 has a particular disease. There is a test for the disease that gives a positive result 99% of the time when given to someone with the disease. When given to someone without the disease, 99.5% of the time it gives a negative result. Find

1. the probability that a person who tests positive has the disease.

2. the probability that a person who tests negative does not have the disease.

Generalized Bayes’ Theorem

❖Suppose that {F1,F2,…,Fn} is a partition of the sample space S. Then, for an event E, we have

p(Fj |E) =p(E|Fj)p(Fj)Pni=1 p(E|Fi)p(Fi)

Random Variables, Expected Value and Variance

Random Variables❖ A random variable is a function from the sample space

of an experiment to the set of real numbers.

❖ A coin is tossed twice. Let X(t) be the random variable that equals that number of heads that appear when t is the outcome. Then X(t) takes on the following values:

❖ X(HH)=2

❖ X(HT)=X(TH)=1

❖ X(TT)=0

Random Variables

❖ The distribution of a random variable X on a sample space S is the set of pairs (r,p(X=r)) for all r∈X(S), where p(X=r) is the probability that X takes the value r.

❖ Example: A fair coin is tossed twice and X(t) is the number of heads in outcome t. The distribution of X is

❖ {(0,0.25),(1,0.5),(2,0.25)}

Random Variables: Illustration on Random Graphs

❖ Experiment: Given a set of n nodes, for every set of two nodes, connect them with an edge with probability p (recall Erdos-Renyi?)

❖ What is the sample space? What is its size?

❖ Define a random variable X(g) that equals the number of edges in g.

❖ What are the possible values of X(g)?

❖ What is the probability distribution of X(g)?



❖ What is the sample space? What is its size?

❖ S is the set of all graphs with n nodes.

❖ S contains the set of all n-node graphs with 0 edges, 1 edge, 2 edges,…, n(n-1)/2 edges. So, the size of S is

|S| =n(n�1)/2X

k=0

✓n(n� 1)/2

k

◆= 2n(n�1)/2




❖ What are the possible values of X(g)?

❖ X(g)∈{0,1,…,n(n-1)/2}.




❖ What is the probability distribution of X(g)?

p(X(g) = k) =

✓n(n� 1)/2

k

◆pk(1� p)

n(n�1)2 �k k 2

⇢0, 1, . . . ,

n(n� 1)

2

�

Expected Values

❖ The expected value (also called the expectation or mean) of a (discrete) random variable X on the sample space S is

E(X) =X

s2S

p(s)X(s)

Expected Values

❖ Example: A fair coin is tossed twice and X(t) is the number of heads.

E(X) = p(HH) ·X(HH) + p(HT ) ·X(HT ) + p(TT ) ·X(TT ) + p(TH) ·X(TH)= 0.25 · 2 + 0.25 · 1 + 0.25 · 0 + 0.25 · 1= 1

Expected Values❖ What is the expected number of edges in a random

graph with n nodes and probability p (using the ER procedure)?

❖ Based on the results we saw before, we have

E(X(g)) =Pn(n�1)/2

k=0 k · p(X(g) = k)

=Pn(n�1)/2

k=0 k�n(n�1)/2

k

�pk(1� p)

n(n�1)2 �k

= n(n�1)2 p

Linearity of Expectations

❖ If Xi, i=1,2,…,n, are random variables on S, and if a and b are real numbers, then

E(X1 +X2 + · · ·+Xn) = E(X1) + E(X2) + · · ·+ E(Xn)

E(aXi + b) = aE(Xi) + b

Average-case Analysis❖ What is the average-case running time of LinearSearch

(for finding whether an element x exists in an array of n elements)?

❖ Assume x is in the input array with probability p.

❖ Assume that if x is in the array, it can be in any position with equal probability.

❖ What random variable do we define? What is the expected value of the variable?

The Geometric Distribution

❖ So far, we have discussed random variables with a finite number of possible outcomes.

❖ Consider the following experiment: A coin with probability of tails being p is tossed repeatedly until it comes up tails. What is the expected number of tosses until this coin comes up tails?

❖ The sample space here is {T,HT,HHT,HHHT,…}, which is infinite.


❖ Let X be the random variable equal to the number of tosses in an element in the sample space.

❖ We have p(X=k)=(1-p)k-1p.

❖ Then,

E(X) =1X

k=1

kp(X = k) =1X

k=1

k(1� p)k�1p = p1X

k=1

k(1� p)k�1 = p1

p2=

1

p


❖ Let X be the random variable equal to the number of tosses in an element in the sample space.

❖ We have p(X=k)=(1-p)k-1p.

❖ Then,

E(X) =1X

k=1

kp(X = k) =1X

k=1

k(1� p)k�1p = p1X

k=1

k(1� p)k�1 = p1

p2=

1

p

1X

k=1

kxk�1 =1

(1� x)2(|x| < 1)


❖ A random variable X has a geometric distribution with parameter p if p(X=k)=(1-p)k-1p for k=1,2,3,…, where p is a real number with 0≤p≤1.

❖ If , then X ⇠ Geometric(p) E(X) = 1/p.


0 1

p

q

1-p 1-q

How is the duration in state 0 distributed?

Independent Random Variables

❖ The random variables X and Y on a sample space S are independent if

p(X = x and Y = y) = p(X = x)p(Y = y)

Independent Random Variables

❖ If random variables X and Y on sample space S are independent, then

E(XY ) = E(X)E(Y )

Variance

❖ The expected value of a random variable tells us its average value, but nothing about how widely its values are distributed.

❖ Contrast X and Y on S={1,2,3,4,5,6}, where

❖ X(s)=0 for all s∈S

❖ Y(s)=-1 for s∈{1,2,3} and Y(s)=1 for s∈{4,5,6}

Variance

❖ Let X be a random variable on a sample space S. The variance of X, denoted by V(X), is

V (X) =X

s2S

(X(s)� E(X))2p(s)

❖ The standard deviation of X, denoted by σ(X), is defined to be

�(X) =pV (X)

Variance

V (X) =P

s2S(X(s)� E(X))2p(s)=

Ps2S X(s)2p(s)� 2E(X)

Ps2S X(s)p(s) + E(X)2

Ps2S p(s)

= E(X2)� 2E(X)E(X) + E(X)2

= E(X2)� E(X)2

Variance

❖ If X is a random variable on a sample space S and , then E(X) = µ

V (X) = E((X � µ)2)

Bienayme’s Formula

❖ If Xi, i=1,2,…,n, are pairwise independent random variables on S, then

V (nX

i=1

Xi) =nX

i=1

V (Xi)

Variance

❖ If X is a random variable and c is a constant then,

V (cX) = c2V (X)

Bienayme’s Formula

❖ If Xi, i=1,2,…,n, are random variables (not necessarily independent) on S, then

where

Cov(X,Y ) = E(XY )� E(X)E(Y )

V

nX

i=1

Xi

!=

nX

i=1

V (Xi) + 2nX

i=1

X

j>i

Cov(Xi, Xj)

Tail Bounds

What Is This About?

❖ How large can a random variable get?

❖ In other words, how far from the mean can a value that the random variable takes be?

mean

mean

tails

mean

tails

how large?

Why Do We Care?

❖ Example:

❖ is the number of steps an algorithm takes.

❖ is the average-case running-time of the algorithm.

❖ Can the algorithm, on average, take 2n steps, but on some inputs take, say, 500n2 steps?

E(X)<latexit sha1_base64="kTVjQRU6iUov3VUghL8k1P3M2A8=">AAAB9XicbVDLSgMxFL1TX7W+qi7dBItQN2VGBF0WRXBZwT6gHUsmzbShmWRIMkoZ+h9uXCji1n9x59+YaWehrQcCh3Pu5Z6cIOZMG9f9dgorq2vrG8XN0tb2zu5eef+gpWWiCG0SyaXqBFhTzgRtGmY47cSK4ijgtB2MrzO//UiVZlLcm0lM/QgPBQsZwcZKD2kvwmYUBOhmWu2c9ssVt+bOgJaJl5MK5Gj0y1+9gSRJRIUhHGvd9dzY+ClWhhFOp6VeommMyRgPaddSgSOq/XSWeopOrDJAoVT2CYNm6u+NFEdaT6LATmYh9aKXif953cSEl37KRJwYKsj8UJhwZCTKKkADpigxfGIJJorZrIiMsMLE2KJKtgRv8cvLpHVW89yad3deqV/ldRThCI6hCh5cQB1uoQFNIKDgGV7hzXlyXpx352M+WnDynUP4A+fzB395kdY=</latexit><latexit sha1_base64="kTVjQRU6iUov3VUghL8k1P3M2A8=">AAAB9XicbVDLSgMxFL1TX7W+qi7dBItQN2VGBF0WRXBZwT6gHUsmzbShmWRIMkoZ+h9uXCji1n9x59+YaWehrQcCh3Pu5Z6cIOZMG9f9dgorq2vrG8XN0tb2zu5eef+gpWWiCG0SyaXqBFhTzgRtGmY47cSK4ijgtB2MrzO//UiVZlLcm0lM/QgPBQsZwcZKD2kvwmYUBOhmWu2c9ssVt+bOgJaJl5MK5Gj0y1+9gSRJRIUhHGvd9dzY+ClWhhFOp6VeommMyRgPaddSgSOq/XSWeopOrDJAoVT2CYNm6u+NFEdaT6LATmYh9aKXif953cSEl37KRJwYKsj8UJhwZCTKKkADpigxfGIJJorZrIiMsMLE2KJKtgRv8cvLpHVW89yad3deqV/ldRThCI6hCh5cQB1uoQFNIKDgGV7hzXlyXpx352M+WnDynUP4A+fzB395kdY=</latexit><latexit sha1_base64="kTVjQRU6iUov3VUghL8k1P3M2A8=">AAAB9XicbVDLSgMxFL1TX7W+qi7dBItQN2VGBF0WRXBZwT6gHUsmzbShmWRIMkoZ+h9uXCji1n9x59+YaWehrQcCh3Pu5Z6cIOZMG9f9dgorq2vrG8XN0tb2zu5eef+gpWWiCG0SyaXqBFhTzgRtGmY47cSK4ijgtB2MrzO//UiVZlLcm0lM/QgPBQsZwcZKD2kvwmYUBOhmWu2c9ssVt+bOgJaJl5MK5Gj0y1+9gSRJRIUhHGvd9dzY+ClWhhFOp6VeommMyRgPaddSgSOq/XSWeopOrDJAoVT2CYNm6u+NFEdaT6LATmYh9aKXif953cSEl37KRJwYKsj8UJhwZCTKKkADpigxfGIJJorZrIiMsMLE2KJKtgRv8cvLpHVW89yad3deqV/ldRThCI6hCh5cQB1uoQFNIKDgGV7hzXlyXpx352M+WnDynUP4A+fzB395kdY=</latexit><latexit sha1_base64="kTVjQRU6iUov3VUghL8k1P3M2A8=">AAAB9XicbVDLSgMxFL1TX7W+qi7dBItQN2VGBF0WRXBZwT6gHUsmzbShmWRIMkoZ+h9uXCji1n9x59+YaWehrQcCh3Pu5Z6cIOZMG9f9dgorq2vrG8XN0tb2zu5eef+gpWWiCG0SyaXqBFhTzgRtGmY47cSK4ijgtB2MrzO//UiVZlLcm0lM/QgPBQsZwcZKD2kvwmYUBOhmWu2c9ssVt+bOgJaJl5MK5Gj0y1+9gSRJRIUhHGvd9dzY+ClWhhFOp6VeommMyRgPaddSgSOq/XSWeopOrDJAoVT2CYNm6u+NFEdaT6LATmYh9aKXif953cSEl37KRJwYKsj8UJhwZCTKKkADpigxfGIJJorZrIiMsMLE2KJKtgRv8cvLpHVW89yad3deqV/ldRThCI6hCh5cQB1uoQFNIKDgGV7hzXlyXpx352M+WnDynUP4A+fzB395kdY=</latexit>

X<latexit sha1_base64="hf6hOeTjseL13iz+i/MO/ptaY5E=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==</latexit><latexit sha1_base64="hf6hOeTjseL13iz+i/MO/ptaY5E=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==</latexit><latexit sha1_base64="hf6hOeTjseL13iz+i/MO/ptaY5E=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==</latexit><latexit sha1_base64="hf6hOeTjseL13iz+i/MO/ptaY5E=">AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==</latexit>

Markov’s Inequality

❖ Let X be a random variable that takes only nonnegative values. Then, for every real number a>0 we have

P (X � a) E(X)

a<latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit><latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit><latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit><latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit>

Markov’s Inequality

❖ Let X be a random variable that takes only nonnegative values. Then, for every real number a>0 we have

How large a value can X take?

P (X � a) E(X)

a<latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit><latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit><latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit><latexit sha1_base64="31UkQukhjmsnC9djG5Wb82HiHoE=">AAACEnicbVDLSsNAFJ3UV62vqEs3g0VoNyURQZdFEVxWsA9oQrmZTtqhk4czE6GEfIMbf8WNC0XcunLn3zhps9DWA8MczrmXe+/xYs6ksqxvo7Syura+Ud6sbG3v7O6Z+wcdGSWC0DaJeCR6HkjKWUjbiilOe7GgEHicdr3JVe53H6iQLArv1DSmbgCjkPmMgNLSwKy3aj3sjOg9hjp2uP4dXwBJUycANfY8fJ3VevUshWxgVq2GNQNeJnZBqqhAa2B+OcOIJAENFeEgZd+2YuWmIBQjnGYVJ5E0BjKBEe1rGkJApZvOTsrwiVaG2I+EfqHCM/V3RwqBlNPA05X5onLRy8X/vH6i/As3ZWGcKBqS+SA/4VhFOM8HD5mgRPGpJkAE07tiMgYdidIpVnQI9uLJy6Rz2rCthn17Vm1eFnGU0RE6RjVko3PURDeohdqIoEf0jF7Rm/FkvBjvxse8tGQUPYfoD4zPH/qYnG8=</latexit>

Markov’s Inequality: Proof

E(X) =P

x xP (x)=

Px<a xP (x) +

Px�a xP (x)

�P

x<a 0P (x) +P

x�a aP (x)= a

Px�a P (x)

= aP (x � a)<latexit sha1_base64="zdBcIo93Nxa1qsLtbPixCAD4rZU=">AAAC03icbVJdi9NAFJ1k/VjjV10ffblYXFqEkoigDwqLIvhYwe4WmlJuprfdYSeTODNZWkLYRXz1z/nmX/BXOGnistt6YeDMOffcO3NnklwKY8Pwt+fv3bp95+7+veD+g4ePHneeHBybrNCcRjyTmR4naEgKRSMrrKRxrgnTRNJJcvax1k/OSRuRqa92ndM0xaUSC8HROmrW+RMntBSqRK1xXZVSyioo4xTtaZLAp6o37sMhvHcrNkU6K1cVrGDYW/UhjgO4KcE7rFYb7eUVFS/pG/yja8thQ10zORnCpmaw68Srbk0v3E7Y0d2+1fqOjUnN28vNOt1wEG4CdkHUgi5rYzjr/IrnGS9SUpZLNGYShbmdumpWcElVEBeGcuRnuKSJgwpTMtNy8yYVvHDMHBaZdktZ2LDXHSWmxqzTxGXW0zbbWk3+T5sUdvF2WgqVF5YUbxotCgk2g/qBYS40cSvXDiDXwp0V+Clq5NZ9g8ANIdq+8i44fjWIwkH05XX36EM7jn32jD1nPRaxN+yIfWZDNmLcG3rn3oV36Y/80v/u/2hSfa/1PGU3wv/5F1RA1K8=</latexit><latexit sha1_base64="zdBcIo93Nxa1qsLtbPixCAD4rZU=">AAAC03icbVJdi9NAFJ1k/VjjV10ffblYXFqEkoigDwqLIvhYwe4WmlJuprfdYSeTODNZWkLYRXz1z/nmX/BXOGnistt6YeDMOffcO3NnklwKY8Pwt+fv3bp95+7+veD+g4ePHneeHBybrNCcRjyTmR4naEgKRSMrrKRxrgnTRNJJcvax1k/OSRuRqa92ndM0xaUSC8HROmrW+RMntBSqRK1xXZVSyioo4xTtaZLAp6o37sMhvHcrNkU6K1cVrGDYW/UhjgO4KcE7rFYb7eUVFS/pG/yja8thQ10zORnCpmaw68Srbk0v3E7Y0d2+1fqOjUnN28vNOt1wEG4CdkHUgi5rYzjr/IrnGS9SUpZLNGYShbmdumpWcElVEBeGcuRnuKSJgwpTMtNy8yYVvHDMHBaZdktZ2LDXHSWmxqzTxGXW0zbbWk3+T5sUdvF2WgqVF5YUbxotCgk2g/qBYS40cSvXDiDXwp0V+Clq5NZ9g8ANIdq+8i44fjWIwkH05XX36EM7jn32jD1nPRaxN+yIfWZDNmLcG3rn3oV36Y/80v/u/2hSfa/1PGU3wv/5F1RA1K8=</latexit><latexit sha1_base64="zdBcIo93Nxa1qsLtbPixCAD4rZU=">AAAC03icbVJdi9NAFJ1k/VjjV10ffblYXFqEkoigDwqLIvhYwe4WmlJuprfdYSeTODNZWkLYRXz1z/nmX/BXOGnistt6YeDMOffcO3NnklwKY8Pwt+fv3bp95+7+veD+g4ePHneeHBybrNCcRjyTmR4naEgKRSMrrKRxrgnTRNJJcvax1k/OSRuRqa92ndM0xaUSC8HROmrW+RMntBSqRK1xXZVSyioo4xTtaZLAp6o37sMhvHcrNkU6K1cVrGDYW/UhjgO4KcE7rFYb7eUVFS/pG/yja8thQ10zORnCpmaw68Srbk0v3E7Y0d2+1fqOjUnN28vNOt1wEG4CdkHUgi5rYzjr/IrnGS9SUpZLNGYShbmdumpWcElVEBeGcuRnuKSJgwpTMtNy8yYVvHDMHBaZdktZ2LDXHSWmxqzTxGXW0zbbWk3+T5sUdvF2WgqVF5YUbxotCgk2g/qBYS40cSvXDiDXwp0V+Clq5NZ9g8ANIdq+8i44fjWIwkH05XX36EM7jn32jD1nPRaxN+yIfWZDNmLcG3rn3oV36Y/80v/u/2hSfa/1PGU3wv/5F1RA1K8=</latexit><latexit sha1_base64="zdBcIo93Nxa1qsLtbPixCAD4rZU=">AAAC03icbVJdi9NAFJ1k/VjjV10ffblYXFqEkoigDwqLIvhYwe4WmlJuprfdYSeTODNZWkLYRXz1z/nmX/BXOGnistt6YeDMOffcO3NnklwKY8Pwt+fv3bp95+7+veD+g4ePHneeHBybrNCcRjyTmR4naEgKRSMrrKRxrgnTRNJJcvax1k/OSRuRqa92ndM0xaUSC8HROmrW+RMntBSqRK1xXZVSyioo4xTtaZLAp6o37sMhvHcrNkU6K1cVrGDYW/UhjgO4KcE7rFYb7eUVFS/pG/yja8thQ10zORnCpmaw68Srbk0v3E7Y0d2+1fqOjUnN28vNOt1wEG4CdkHUgi5rYzjr/IrnGS9SUpZLNGYShbmdumpWcElVEBeGcuRnuKSJgwpTMtNy8yYVvHDMHBaZdktZ2LDXHSWmxqzTxGXW0zbbWk3+T5sUdvF2WgqVF5YUbxotCgk2g/qBYS40cSvXDiDXwp0V+Clq5NZ9g8ANIdq+8i44fjWIwkH05XX36EM7jn32jD1nPRaxN+yIfWZDNmLcG3rn3oV36Y/80v/u/2hSfa/1PGU3wv/5F1RA1K8=</latexit>

Markov’s Inequality: An Example

❖ A fair coin is tossed n times. Give an upper bound on the probability that at least 3n/4 of the tosses yield heads.

P (X � 3n

4) E(X)

3n/4=

n/2

3n/4=

2

3<latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="cJVIq6fDH9Dtb1vDnmGkmMNTnL8=">AAACNnicbZBPS8MwGMbfzn9z/qtevQRFmJfZzoFeBEEEjxPcLKxlpFk6g2lak1QYpV/Ni9/Am3cvHhTxE5huQ3T6QODh97wheZ8w5Uxpx3m2KnPzC4tL1eXayura+oa9udpVSSYJ7ZCEJ9ILsaKcCdrRTHPqpZLiOOT0Orw9K/PreyoVS8SVHqU0iPFQsIgRrA3q21677iF/SO+QH0lM8kNR5K1iH/n8G+V+jPVNGKLzou7tF2bkoFWgk2kqDpqzqARF3951Gs5Y6K9xp2YXpmr37Sd/kJAspkITjpXquU6qgxxLzQinRc3PFE0xucVD2jNW4JiqIB83UKA9QwYoSqQ5QqMx/Xkjx7FSozg0k+UyajYr4X9ZL9PRcZAzkWaaCjJ5KMo40gkq60QDJinRfGQMJpKZvyJyg00L2pReMyW4syv/Nd1mw3Ua7qUDVdiGHaiDC0dwChfQhg4QeIAXeIN369F6tT4mdVWsaW9b8EvW5xfDmKya</latexit><latexit sha1_base64="cJVIq6fDH9Dtb1vDnmGkmMNTnL8=">AAACNnicbZBPS8MwGMbfzn9z/qtevQRFmJfZzoFeBEEEjxPcLKxlpFk6g2lak1QYpV/Ni9/Am3cvHhTxE5huQ3T6QODh97wheZ8w5Uxpx3m2KnPzC4tL1eXayura+oa9udpVSSYJ7ZCEJ9ILsaKcCdrRTHPqpZLiOOT0Orw9K/PreyoVS8SVHqU0iPFQsIgRrA3q21677iF/SO+QH0lM8kNR5K1iH/n8G+V+jPVNGKLzou7tF2bkoFWgk2kqDpqzqARF3951Gs5Y6K9xp2YXpmr37Sd/kJAspkITjpXquU6qgxxLzQinRc3PFE0xucVD2jNW4JiqIB83UKA9QwYoSqQ5QqMx/Xkjx7FSozg0k+UyajYr4X9ZL9PRcZAzkWaaCjJ5KMo40gkq60QDJinRfGQMJpKZvyJyg00L2pReMyW4syv/Nd1mw3Ua7qUDVdiGHaiDC0dwChfQhg4QeIAXeIN369F6tT4mdVWsaW9b8EvW5xfDmKya</latexit><latexit sha1_base64="hSgK9dyjj0PqU3eWocHf+/5PwWA=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVjFhsXWRw5cAiOQAlY4BQ0wDVoghbA4AG8gDfwbjwar8aH8TlrXTKymQPwp4yvbzVMrkI=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit><latexit sha1_base64="HcuvOFcpjVlkBhuZxxBdEiP5M9o=">AAACQXicbZDNS8MwGMZTP+f8mnr0EhzCdtnaOdCLMBTB4wS3FdYx0izdwtK0JqkwSv81L/4H3rx78aCIVy+mWxHdfCHw8Hvelzfv44aMSmWaz8bS8srq2npuI7+5tb2zW9jbb8sgEpi0cMACYbtIEkY5aSmqGLFDQZDvMtJxx5ep37knQtKA36pJSHo+GnLqUYyURv2C3SzZ0BmSO+h4AuH4hCdxPSlDh/2g2PGRGrkuvEpKdjnRLdV6As8zl1dr8ygFSb9QNCvmtOCisDJRBFk1+4UnZxDgyCdcYYak7FpmqHoxEopiRpK8E0kSIjxGQ9LVkiOfyF48TSCBx5oMoBcI/biCU/p7Ika+lBPf1Z3pMXLeS+F/XjdS3lkvpjyMFOF4tsiLGFQBTOOEAyoIVmyiBcKC6r9CPEI6BaVDz+sQrPmTF0W7VrHMinVTLzYusjhy4BAcgRKwwClogGvQBC2AwQN4AW/g3Xg0Xo0P43PWumRkMwfgTxlf3zaMrkY=</latexit>

❖ For distributions encountered in practice, Markov’s inequality gives a very loose bound.❖ Why?

Chebyshev’s Inequality

❖ Let X be a random variable. For every real number r>0,

P (|X � E(X)| � a) V (X)

a2<latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit><latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit><latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit><latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit>

Chebyshev’s Inequality

❖ Let X be a random variable. For every real number r>0,

How likely is it that RV X takes a valuethat’s at least distance a from its expected value?

P (|X � E(X)| � a) V (X)

a2<latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit><latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit><latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit><latexit sha1_base64="EmG2ec8vSIpiqKdxp47b53bNDhY=">AAACHXicbVDNS8MwHE3n15xfVY9egkPYDo52DPQ4FMHjBLcV1jrSLN3C0g+TVBhd/xEv/itePCjiwYv435huPejmg5DHe78fyXtuxKiQhvGtFVZW19Y3ipulre2d3T19/6Ajwphj0sYhC7nlIkEYDUhbUsmIFXGCfJeRrju+zPzuA+GChsGtnETE8dEwoB7FSCqprzdalakFT2Fi+0iOXBdepRWrOoX2kNxDVIU2U7ftcYSTjjLSBN3V075eNmrGDHCZmDkpgxytvv5pD0Ic+ySQmCEheqYRSSdBXFLMSFqyY0EihMdoSHqKBsgnwklm6VJ4opQB9EKuTiDhTP29kSBfiInvqsksg1j0MvE/rxdL79xJaBDFkgR4/pAXMyhDmFUFB5QTLNlEEYQ5VX+FeIRUFVIVWlIlmIuRl0mnXjONmnnTKDcv8jqK4AgcgwowwRlogmvQAm2AwSN4Bq/gTXvSXrR37WM+WtDynUPwB9rXD6nWn9E=</latexit>

Chebyshev’s Inequality: Proof

Chebyshev’s Inequality: ProofObserve that

P (|X � E(X)| � r) = P ((X � E(X))2 � r2)

Chebyshev’s Inequality: ProofObserve that

P (|X � E(X)| � r) = P ((X � E(X))2 � r2)

Applying Markov’s inequality, we get

P ((X � E(X))2 � r2) E((X � E(X))2)

r2=

V (X)

r2

Markov vs Chebyshev

P (X � kµ) 1

k<latexit sha1_base64="Dt7dwsm5ITLnxO4mtUy/zMlL8So=">AAACCHicdVDLSgMxFM34rPU16tKFwSLUTUl81l3RjcsK9gGdoWTSTBuaeZhkhDJ06cZfceNCEbd+gjv/xnQ6gooeuHA4517uvceLBVcaoQ9rZnZufmGxsFRcXlldW7c3NpsqSiRlDRqJSLY9opjgIWtorgVrx5KRwBOs5Q0vJn7rlknFo/Baj2LmBqQfcp9Too3UtXfq5TZ0+uwGDp0g2YeOMNTxJaEpHqfDcdcuocoxwmcnCKIKypCRKj7EEOdKCeSod+13pxfRJGChpoIo1cEo1m5KpOZUsHHRSRSLCR2SPusYGpKAKTfNHhnDPaP0oB9JU6GGmfp9IiWBUqPAM50B0QP125uIf3mdRPtVN+VhnGgW0ukiPxFQR3CSCuxxyagWI0MIldzcCumAmBS0ya5oQvj6FP5PmgcVjCr46qhUO8/jKIBtsAvKAINTUAOXoA4agII78ACewLN1bz1aL9brtHXGyme2wA9Yb59AVZjg</latexit><latexit sha1_base64="Dt7dwsm5ITLnxO4mtUy/zMlL8So=">AAACCHicdVDLSgMxFM34rPU16tKFwSLUTUl81l3RjcsK9gGdoWTSTBuaeZhkhDJ06cZfceNCEbd+gjv/xnQ6gooeuHA4517uvceLBVcaoQ9rZnZufmGxsFRcXlldW7c3NpsqSiRlDRqJSLY9opjgIWtorgVrx5KRwBOs5Q0vJn7rlknFo/Baj2LmBqQfcp9Too3UtXfq5TZ0+uwGDp0g2YeOMNTxJaEpHqfDcdcuocoxwmcnCKIKypCRKj7EEOdKCeSod+13pxfRJGChpoIo1cEo1m5KpOZUsHHRSRSLCR2SPusYGpKAKTfNHhnDPaP0oB9JU6GGmfp9IiWBUqPAM50B0QP125uIf3mdRPtVN+VhnGgW0ukiPxFQR3CSCuxxyagWI0MIldzcCumAmBS0ya5oQvj6FP5PmgcVjCr46qhUO8/jKIBtsAvKAINTUAOXoA4agII78ACewLN1bz1aL9brtHXGyme2wA9Yb59AVZjg</latexit><latexit sha1_base64="Dt7dwsm5ITLnxO4mtUy/zMlL8So=">AAACCHicdVDLSgMxFM34rPU16tKFwSLUTUl81l3RjcsK9gGdoWTSTBuaeZhkhDJ06cZfceNCEbd+gjv/xnQ6gooeuHA4517uvceLBVcaoQ9rZnZufmGxsFRcXlldW7c3NpsqSiRlDRqJSLY9opjgIWtorgVrx5KRwBOs5Q0vJn7rlknFo/Baj2LmBqQfcp9Too3UtXfq5TZ0+uwGDp0g2YeOMNTxJaEpHqfDcdcuocoxwmcnCKIKypCRKj7EEOdKCeSod+13pxfRJGChpoIo1cEo1m5KpOZUsHHRSRSLCR2SPusYGpKAKTfNHhnDPaP0oB9JU6GGmfp9IiWBUqPAM50B0QP125uIf3mdRPtVN+VhnGgW0ukiPxFQR3CSCuxxyagWI0MIldzcCumAmBS0ya5oQvj6FP5PmgcVjCr46qhUO8/jKIBtsAvKAINTUAOXoA4agII78ACewLN1bz1aL9brtHXGyme2wA9Yb59AVZjg</latexit><latexit sha1_base64="Dt7dwsm5ITLnxO4mtUy/zMlL8So=">AAACCHicdVDLSgMxFM34rPU16tKFwSLUTUl81l3RjcsK9gGdoWTSTBuaeZhkhDJ06cZfceNCEbd+gjv/xnQ6gooeuHA4517uvceLBVcaoQ9rZnZufmGxsFRcXlldW7c3NpsqSiRlDRqJSLY9opjgIWtorgVrx5KRwBOs5Q0vJn7rlknFo/Baj2LmBqQfcp9Too3UtXfq5TZ0+uwGDp0g2YeOMNTxJaEpHqfDcdcuocoxwmcnCKIKypCRKj7EEOdKCeSod+13pxfRJGChpoIo1cEo1m5KpOZUsHHRSRSLCR2SPusYGpKAKTfNHhnDPaP0oB9JU6GGmfp9IiWBUqPAM50B0QP125uIf3mdRPtVN+VhnGgW0ukiPxFQR3CSCuxxyagWI0MIldzcCumAmBS0ya5oQvj6FP5PmgcVjCr46qhUO8/jKIBtsAvKAINTUAOXoA4agII78ACewLN1bz1aL9brtHXGyme2wA9Yb59AVZjg</latexit>

vsP (|X � µ| � k�) 1

k2<latexit sha1_base64="dz4UVgVlLMEUmb1W6XtV9Jxh6mE=">AAACE3icdVDLSgMxFM3UV62vqks3wSJUwTIpPndFNy4r2Ad0xpJJM9PQZGZMMkKZ9h/c+CtuXCji1o07/8a0VqiiBy6cnHMvufd4MWdK2/aHlZmZnZtfyC7mlpZXVtfy6xt1FSWS0BqJeCSbHlaUs5DWNNOcNmNJsfA4bXi985HfuKVSsSi80v2YugIHIfMZwdpI7fxetTho7jsiGUAnoDew5ygWCLwLHW5eji8xSdEw7V2Xh+18wS7ZY8Apcmij0yME0UQpgAmq7fy704lIImioCcdKtZAdazfFUjPC6TDnJIrGmPRwQFuGhlhQ5abjm4Zwxygd6EfSVKjhWJ2eSLFQqi880ymw7qrf3kj8y2sl2j9xUxbGiaYh+frITzjUERwFBDtMUqJ53xBMJDO7QtLFJgdtYsyZEL4vhf+TermE7BK6PChUziZxZMEW2AZFgMAxqIALUAU1QMAdeABP4Nm6tx6tF+v1qzVjTWY2wQ9Yb5+g1p1k</latexit><latexit sha1_base64="dz4UVgVlLMEUmb1W6XtV9Jxh6mE=">AAACE3icdVDLSgMxFM3UV62vqks3wSJUwTIpPndFNy4r2Ad0xpJJM9PQZGZMMkKZ9h/c+CtuXCji1o07/8a0VqiiBy6cnHMvufd4MWdK2/aHlZmZnZtfyC7mlpZXVtfy6xt1FSWS0BqJeCSbHlaUs5DWNNOcNmNJsfA4bXi985HfuKVSsSi80v2YugIHIfMZwdpI7fxetTho7jsiGUAnoDew5ygWCLwLHW5eji8xSdEw7V2Xh+18wS7ZY8Apcmij0yME0UQpgAmq7fy704lIImioCcdKtZAdazfFUjPC6TDnJIrGmPRwQFuGhlhQ5abjm4Zwxygd6EfSVKjhWJ2eSLFQqi880ymw7qrf3kj8y2sl2j9xUxbGiaYh+frITzjUERwFBDtMUqJ53xBMJDO7QtLFJgdtYsyZEL4vhf+TermE7BK6PChUziZxZMEW2AZFgMAxqIALUAU1QMAdeABP4Nm6tx6tF+v1qzVjTWY2wQ9Yb5+g1p1k</latexit><latexit sha1_base64="dz4UVgVlLMEUmb1W6XtV9Jxh6mE=">AAACE3icdVDLSgMxFM3UV62vqks3wSJUwTIpPndFNy4r2Ad0xpJJM9PQZGZMMkKZ9h/c+CtuXCji1o07/8a0VqiiBy6cnHMvufd4MWdK2/aHlZmZnZtfyC7mlpZXVtfy6xt1FSWS0BqJeCSbHlaUs5DWNNOcNmNJsfA4bXi985HfuKVSsSi80v2YugIHIfMZwdpI7fxetTho7jsiGUAnoDew5ygWCLwLHW5eji8xSdEw7V2Xh+18wS7ZY8Apcmij0yME0UQpgAmq7fy704lIImioCcdKtZAdazfFUjPC6TDnJIrGmPRwQFuGhlhQ5abjm4Zwxygd6EfSVKjhWJ2eSLFQqi880ymw7qrf3kj8y2sl2j9xUxbGiaYh+frITzjUERwFBDtMUqJ53xBMJDO7QtLFJgdtYsyZEL4vhf+TermE7BK6PChUziZxZMEW2AZFgMAxqIALUAU1QMAdeABP4Nm6tx6tF+v1qzVjTWY2wQ9Yb5+g1p1k</latexit><latexit sha1_base64="dz4UVgVlLMEUmb1W6XtV9Jxh6mE=">AAACE3icdVDLSgMxFM3UV62vqks3wSJUwTIpPndFNy4r2Ad0xpJJM9PQZGZMMkKZ9h/c+CtuXCji1o07/8a0VqiiBy6cnHMvufd4MWdK2/aHlZmZnZtfyC7mlpZXVtfy6xt1FSWS0BqJeCSbHlaUs5DWNNOcNmNJsfA4bXi985HfuKVSsSi80v2YugIHIfMZwdpI7fxetTho7jsiGUAnoDew5ygWCLwLHW5eji8xSdEw7V2Xh+18wS7ZY8Apcmij0yME0UQpgAmq7fy704lIImioCcdKtZAdazfFUjPC6TDnJIrGmPRwQFuGhlhQ5abjm4Zwxygd6EfSVKjhWJ2eSLFQqi880ymw7qrf3kj8y2sl2j9xUxbGiaYh+frITzjUERwFBDtMUqJ53xBMJDO7QtLFJgdtYsyZEL4vhf+TermE7BK6PChUziZxZMEW2AZFgMAxqIALUAU1QMAdeABP4Nm6tx6tF+v1qzVjTWY2wQ9Yb5+g1p1k</latexit>

Chebyshev’s Inequality: An Example❖ Assume we have a distribution whose mean is 80 and

standard deviation is 10. What is a lower bound on the percentage of values that fall between 60 and 100 (exclusively) in this distribution?



p(|X(s)� E(X)| � r) V

r2



p(|X(s)� E(X)| � r) V

r2

E(X) = 80 V = 100 r = 20



p(|X(s)� E(X)| � r) V

r2

E(X) = 80 V = 100 r = 20

p(|X(s)� 80| � 20) 1

4



p(|X(s)� E(X)| � r) V

r2

E(X) = 80 V = 100 r = 20

p(|X(s)� 80| � 20) 1

4

) lower bound is 75%

Illustration: Estimating π Using the Monte Carlo Method

❖ Here’s a simple algorithm for estimating π:

❖ Throw darts at a square whose area is 1, inside which there’s a circle whose radius is 1/2.

❖ The probability that it lands inside the circle equals the ratio of the circle area to the square area (π/4). Therefore, calculate the proportion of times that the dart landed inside the circle and multiply it by 4.

r=1/2

(0.5,0.5)


r=1/2

(0.5,0.5)

COMP 182: Algorithmic Thinking

Monte Carlo Estimation of ⇡

Luay Nakhleh

April 16, 2018

Algorithm 1: MonteCarlo ⇡Estimation.

Input: n 2 N.

Output: Estimate ⇡̂ of ⇡.

for i = 1 to n do

a random(0, 1); // random number in [0, 1]b random(0, 1); // random number in [0, 1]Xi 0;

ifp

(a� 0.5)2 + (b� 0.5)2 0.5 then

Xi 1; // the dart landed inside/on the circle

⇡̂ 4 · (Pn

i=1 Xi)/n;

return ⇡̂;

1


❖ Let Xi be the random variable that denotes whether the i-th dart landed inside the circle (1 if it did, and 0 otherwise).

❖ Then, ⇡̂(n) = 4

Pni=1 Xi

n



❖ Then, ⇡̂(n) = 4

Pni=1 Xi

nE(Xi) =

⇡

41 + (1� ⇡

4)0 =

⇡

4

V (Xi) =⇡

4(1� ⇡

4)



❖ Then, ⇡̂(n) = 4

Pni=1 Xi

nE(Xi) =

⇡

41 + (1� ⇡

4)0 =

⇡

4

V (Xi) =⇡

4(1� ⇡

4)



❖ Then, ⇡̂(n) = 4

Pni=1 Xi

nE(Xi) =

⇡

41 + (1� ⇡

4)0 =

⇡

4

V (Xi) =⇡

4(1� ⇡

4)

V (⇡̂) = V (4

n

nX

i=1

Xi) =16

n2

nX

i=1

V (Xi) =⇡(4� ⇡)

n


❖ The question of interest is: How big should n be for us to get a good estimate?


❖ In a probabilistic setting, the question can be asked as:

❖ What should the value of n be so that the estimation error of π is within δ with probability at least ε?

❖ (of course, we want δ to be very small and ε to be as close to 1 as possible. For example, δ=0.001 and ε=0.95)


❖ In other words, we are interested in the value of n that yields

p(|⇡̂(n)� ⇡| < �) > "


❖ For δ=0.001 and ε=0.95, we seek n such that



r V/r2Chebyshev’s inequality:







So, we would like n such that ⇡(4� ⇡)

n(0.001)2 0.05


⇡(4� ⇡)

n(0.001)2 0.05


⇡(4� ⇡)

n(0.001)2 0.05

⇡(4� ⇡) 4


⇡(4� ⇡)

n(0.001)2 0.05

⇡(4� ⇡) 4

) ⇡(4� ⇡)

n(0.001)2 4

n(0.001)2 0.05


⇡(4� ⇡)

n(0.001)2 0.05

⇡(4� ⇡) 4

) ⇡(4� ⇡)

n(0.001)2 4

n(0.001)2 0.05

) n � 80, 000, 000

A Corollary of Chebyshev’s Inequality

❖ Let X1,X2,…,Xn be independent random variables with

E(Xi) = µi<latexit sha1_base64="7/IG0VTfjvOMvDzaktxkxgmGyXo=">AAACAXicbVDLSsNAFJ3UV62vqBvBzWAR6qYkIuhGKIrgsoJ9QBPCZDpph85MwsxEKKFu/BU3LhRx61+482+ctFlo64ELh3Pu5d57woRRpR3n2yotLa+srpXXKxubW9s79u5eW8WpxKSFYxbLbogUYVSQlqaakW4iCeIhI51wdJ37nQciFY3FvR4nxOdoIGhEMdJGCuyDzONID8MQ3kxq3YCewEvo8TSggV116s4UcJG4BamCAs3A/vL6MU45ERozpFTPdRLtZ0hqihmZVLxUkQThERqQnqECcaL8bPrBBB4bpQ+jWJoSGk7V3xMZ4kqNeWg683PVvJeL/3m9VEcXfkZFkmoi8GxRlDKoY5jHAftUEqzZ2BCEJTW3QjxEEmFtQquYENz5lxdJ+7TuOnX37qzauCriKINDcARqwAXnoAFuQRO0AAaP4Bm8gjfryXqx3q2PWWvJKmb2wR9Ynz+mw5W2</latexit><latexit sha1_base64="7/IG0VTfjvOMvDzaktxkxgmGyXo=">AAACAXicbVDLSsNAFJ3UV62vqBvBzWAR6qYkIuhGKIrgsoJ9QBPCZDpph85MwsxEKKFu/BU3LhRx61+482+ctFlo64ELh3Pu5d57woRRpR3n2yotLa+srpXXKxubW9s79u5eW8WpxKSFYxbLbogUYVSQlqaakW4iCeIhI51wdJ37nQciFY3FvR4nxOdoIGhEMdJGCuyDzONID8MQ3kxq3YCewEvo8TSggV116s4UcJG4BamCAs3A/vL6MU45ERozpFTPdRLtZ0hqihmZVLxUkQThERqQnqECcaL8bPrBBB4bpQ+jWJoSGk7V3xMZ4kqNeWg683PVvJeL/3m9VEcXfkZFkmoi8GxRlDKoY5jHAftUEqzZ2BCEJTW3QjxEEmFtQquYENz5lxdJ+7TuOnX37qzauCriKINDcARqwAXnoAFuQRO0AAaP4Bm8gjfryXqx3q2PWWvJKmb2wR9Ynz+mw5W2</latexit><latexit sha1_base64="7/IG0VTfjvOMvDzaktxkxgmGyXo=">AAACAXicbVDLSsNAFJ3UV62vqBvBzWAR6qYkIuhGKIrgsoJ9QBPCZDpph85MwsxEKKFu/BU3LhRx61+482+ctFlo64ELh3Pu5d57woRRpR3n2yotLa+srpXXKxubW9s79u5eW8WpxKSFYxbLbogUYVSQlqaakW4iCeIhI51wdJ37nQciFY3FvR4nxOdoIGhEMdJGCuyDzONID8MQ3kxq3YCewEvo8TSggV116s4UcJG4BamCAs3A/vL6MU45ERozpFTPdRLtZ0hqihmZVLxUkQThERqQnqECcaL8bPrBBB4bpQ+jWJoSGk7V3xMZ4kqNeWg683PVvJeL/3m9VEcXfkZFkmoi8GxRlDKoY5jHAftUEqzZ2BCEJTW3QjxEEmFtQquYENz5lxdJ+7TuOnX37qzauCriKINDcARqwAXnoAFuQRO0AAaP4Bm8gjfryXqx3q2PWWvJKmb2wR9Ynz+mw5W2</latexit><latexit sha1_base64="7/IG0VTfjvOMvDzaktxkxgmGyXo=">AAACAXicbVDLSsNAFJ3UV62vqBvBzWAR6qYkIuhGKIrgsoJ9QBPCZDpph85MwsxEKKFu/BU3LhRx61+482+ctFlo64ELh3Pu5d57woRRpR3n2yotLa+srpXXKxubW9s79u5eW8WpxKSFYxbLbogUYVSQlqaakW4iCeIhI51wdJ37nQciFY3FvR4nxOdoIGhEMdJGCuyDzONID8MQ3kxq3YCewEvo8TSggV116s4UcJG4BamCAs3A/vL6MU45ERozpFTPdRLtZ0hqihmZVLxUkQThERqQnqECcaL8bPrBBB4bpQ+jWJoSGk7V3xMZ4kqNeWg683PVvJeL/3m9VEcXfkZFkmoi8GxRlDKoY5jHAftUEqzZ2BCEJTW3QjxEEmFtQquYENz5lxdJ+7TuOnX37qzauCriKINDcARqwAXnoAFuQRO0AAaP4Bm8gjfryXqx3q2PWWvJKmb2wR9Ynz+mw5W2</latexit>

Then, for any a>0:

and V (Xi) = �2i

<latexit sha1_base64="Ng+7wxss3urtRwLzwgf1CfQ2G6A=">AAAB/HicbVBNS8NAEJ3Ur1q/oj16WSxCvZSkCHoRil48VrAf0Maw2W7bpZtN2N0IIdS/4sWDIl79Id78N27bHLT1wcDjvRlm5gUxZ0o7zrdVWFvf2Nwqbpd2dvf2D+zDo7aKEkloi0Q8kt0AK8qZoC3NNKfdWFIcBpx2gsnNzO88UqlYJO51GlMvxCPBhoxgbSTfLrerXZ+doSvUV2wUYp891H274tScOdAqcXNSgRxN3/7qDyKShFRowrFSPdeJtZdhqRnhdFrqJ4rGmEzwiPYMFTikysvmx0/RqVEGaBhJU0Kjufp7IsOhUmkYmM4Q67Fa9mbif14v0cNLL2MiTjQVZLFomHCkIzRLAg2YpETz1BBMJDO3IjLGEhNt8iqZENzll1dJu15znZp7d15pXOdxFOEYTqAKLlxAA26hCS0gkMIzvMKb9WS9WO/Wx6K1YOUzZfgD6/MHmyKTcg==</latexit><latexit sha1_base64="Ng+7wxss3urtRwLzwgf1CfQ2G6A=">AAAB/HicbVBNS8NAEJ3Ur1q/oj16WSxCvZSkCHoRil48VrAf0Maw2W7bpZtN2N0IIdS/4sWDIl79Id78N27bHLT1wcDjvRlm5gUxZ0o7zrdVWFvf2Nwqbpd2dvf2D+zDo7aKEkloi0Q8kt0AK8qZoC3NNKfdWFIcBpx2gsnNzO88UqlYJO51GlMvxCPBhoxgbSTfLrerXZ+doSvUV2wUYp891H274tScOdAqcXNSgRxN3/7qDyKShFRowrFSPdeJtZdhqRnhdFrqJ4rGmEzwiPYMFTikysvmx0/RqVEGaBhJU0Kjufp7IsOhUmkYmM4Q67Fa9mbif14v0cNLL2MiTjQVZLFomHCkIzRLAg2YpETz1BBMJDO3IjLGEhNt8iqZENzll1dJu15znZp7d15pXOdxFOEYTqAKLlxAA26hCS0gkMIzvMKb9WS9WO/Wx6K1YOUzZfgD6/MHmyKTcg==</latexit><latexit sha1_base64="Ng+7wxss3urtRwLzwgf1CfQ2G6A=">AAAB/HicbVBNS8NAEJ3Ur1q/oj16WSxCvZSkCHoRil48VrAf0Maw2W7bpZtN2N0IIdS/4sWDIl79Id78N27bHLT1wcDjvRlm5gUxZ0o7zrdVWFvf2Nwqbpd2dvf2D+zDo7aKEkloi0Q8kt0AK8qZoC3NNKfdWFIcBpx2gsnNzO88UqlYJO51GlMvxCPBhoxgbSTfLrerXZ+doSvUV2wUYp891H274tScOdAqcXNSgRxN3/7qDyKShFRowrFSPdeJtZdhqRnhdFrqJ4rGmEzwiPYMFTikysvmx0/RqVEGaBhJU0Kjufp7IsOhUmkYmM4Q67Fa9mbif14v0cNLL2MiTjQVZLFomHCkIzRLAg2YpETz1BBMJDO3IjLGEhNt8iqZENzll1dJu15znZp7d15pXOdxFOEYTqAKLlxAA26hCS0gkMIzvMKb9WS9WO/Wx6K1YOUzZfgD6/MHmyKTcg==</latexit><latexit sha1_base64="Ng+7wxss3urtRwLzwgf1CfQ2G6A=">AAAB/HicbVBNS8NAEJ3Ur1q/oj16WSxCvZSkCHoRil48VrAf0Maw2W7bpZtN2N0IIdS/4sWDIl79Id78N27bHLT1wcDjvRlm5gUxZ0o7zrdVWFvf2Nwqbpd2dvf2D+zDo7aKEkloi0Q8kt0AK8qZoC3NNKfdWFIcBpx2gsnNzO88UqlYJO51GlMvxCPBhoxgbSTfLrerXZ+doSvUV2wUYp891H274tScOdAqcXNSgRxN3/7qDyKShFRowrFSPdeJtZdhqRnhdFrqJ4rGmEzwiPYMFTikysvmx0/RqVEGaBhJU0Kjufp7IsOhUmkYmM4Q67Fa9mbif14v0cNLL2MiTjQVZLFomHCkIzRLAg2YpETz1BBMJDO3IjLGEhNt8iqZENzll1dJu15znZp7d15pXOdxFOEYTqAKLlxAA26hCS0gkMIzvMKb9WS9WO/Wx6K1YOUzZfgD6/MHmyKTcg==</latexit>

P

��

nX

i=1

Xi �nX

i=1

µi

�� a

!Pn

i=1 �2i

a2<latexit sha1_base64="U6RSeLtoQEK/q0r1Qt3/RcXczuI=">AAACWHicbVFNaxsxENVukyZxv5z02IuoKaSHmt0QaC+BkFxydKFODJazaOXZ9RBJu5FmC2brP1nIofkrvVT+OOSjAxKP9+aNpKe81ugpSf5E8Yut7Zc7u3udV6/fvH3X3T+49FXjFAxVpSs3yqUHjRaGhKRhVDuQJtdwld+cL/Wrn+A8VvYHzWuYGFlaLFBJClTWrQZCQ0GH4gzL8pfwjclaPEkX13aUIf/CHzJcmCbDdScXJdxyyYXDckafeZhyy0XhpGofWzyWRmZ4fbRoZdiybi/pJ6viz0G6AT22qUHW/S2mlWoMWFJaej9Ok5omrXSESsOiIxoPtVQ3soRxgFYa8JN2FcyCfwrMlBeVC8sSX7EPHa003s9NHjqNpJl/qi3J/2njhopvkxZt3RBYtT6oaDSnii9T5lN0oEjPA5DKYbgrVzMZ0qHwF50QQvr0yc/B5VE/Tfrp9+Pe6dkmjl32gX1khyxlX9kpu2ADNmSK3bG/0Va0Hd3HLN6J99atcbTxvGePKj74B++is3U=</latexit><latexit sha1_base64="U6RSeLtoQEK/q0r1Qt3/RcXczuI=">AAACWHicbVFNaxsxENVukyZxv5z02IuoKaSHmt0QaC+BkFxydKFODJazaOXZ9RBJu5FmC2brP1nIofkrvVT+OOSjAxKP9+aNpKe81ugpSf5E8Yut7Zc7u3udV6/fvH3X3T+49FXjFAxVpSs3yqUHjRaGhKRhVDuQJtdwld+cL/Wrn+A8VvYHzWuYGFlaLFBJClTWrQZCQ0GH4gzL8pfwjclaPEkX13aUIf/CHzJcmCbDdScXJdxyyYXDckafeZhyy0XhpGofWzyWRmZ4fbRoZdiybi/pJ6viz0G6AT22qUHW/S2mlWoMWFJaej9Ok5omrXSESsOiIxoPtVQ3soRxgFYa8JN2FcyCfwrMlBeVC8sSX7EPHa003s9NHjqNpJl/qi3J/2njhopvkxZt3RBYtT6oaDSnii9T5lN0oEjPA5DKYbgrVzMZ0qHwF50QQvr0yc/B5VE/Tfrp9+Pe6dkmjl32gX1khyxlX9kpu2ADNmSK3bG/0Va0Hd3HLN6J99atcbTxvGePKj74B++is3U=</latexit><latexit sha1_base64="U6RSeLtoQEK/q0r1Qt3/RcXczuI=">AAACWHicbVFNaxsxENVukyZxv5z02IuoKaSHmt0QaC+BkFxydKFODJazaOXZ9RBJu5FmC2brP1nIofkrvVT+OOSjAxKP9+aNpKe81ugpSf5E8Yut7Zc7u3udV6/fvH3X3T+49FXjFAxVpSs3yqUHjRaGhKRhVDuQJtdwld+cL/Wrn+A8VvYHzWuYGFlaLFBJClTWrQZCQ0GH4gzL8pfwjclaPEkX13aUIf/CHzJcmCbDdScXJdxyyYXDckafeZhyy0XhpGofWzyWRmZ4fbRoZdiybi/pJ6viz0G6AT22qUHW/S2mlWoMWFJaej9Ok5omrXSESsOiIxoPtVQ3soRxgFYa8JN2FcyCfwrMlBeVC8sSX7EPHa003s9NHjqNpJl/qi3J/2njhopvkxZt3RBYtT6oaDSnii9T5lN0oEjPA5DKYbgrVzMZ0qHwF50QQvr0yc/B5VE/Tfrp9+Pe6dkmjl32gX1khyxlX9kpu2ADNmSK3bG/0Va0Hd3HLN6J99atcbTxvGePKj74B++is3U=</latexit><latexit sha1_base64="U6RSeLtoQEK/q0r1Qt3/RcXczuI=">AAACWHicbVFNaxsxENVukyZxv5z02IuoKaSHmt0QaC+BkFxydKFODJazaOXZ9RBJu5FmC2brP1nIofkrvVT+OOSjAxKP9+aNpKe81ugpSf5E8Yut7Zc7u3udV6/fvH3X3T+49FXjFAxVpSs3yqUHjRaGhKRhVDuQJtdwld+cL/Wrn+A8VvYHzWuYGFlaLFBJClTWrQZCQ0GH4gzL8pfwjclaPEkX13aUIf/CHzJcmCbDdScXJdxyyYXDckafeZhyy0XhpGofWzyWRmZ4fbRoZdiybi/pJ6viz0G6AT22qUHW/S2mlWoMWFJaej9Ok5omrXSESsOiIxoPtVQ3soRxgFYa8JN2FcyCfwrMlBeVC8sSX7EPHa003s9NHjqNpJl/qi3J/2njhopvkxZt3RBYtT6oaDSnii9T5lN0oEjPA5DKYbgrVzMZ0qHwF50QQvr0yc/B5VE/Tfrp9+Pe6dkmjl32gX1khyxlX9kpu2ADNmSK3bG/0Va0Hd3HLN6J99atcbTxvGePKj74B++is3U=</latexit>

The Law of Large Numbers

❖ Let X1,X2,…,Xn be independently and identically distributed (i.i.d.) random variables, where the (unknown) expected value μ is the same for all variables (that is, ) and their variance is finite. Then, for any ε>0, we have

E(Xi) = µ

P

��

1

n

nX

i=1

Xi

!� µ

�� "

!n!1��! 0

Questions?

COMP 182 Algorithmic Thinking Discrete Probability Luay ...nakhleh/COMP182/DiscreteProbability.pdfBernoulli Trials Suppose an experiment can have only two possible outcomes, e.g.,

Documents