Lecture XI
Jan 17, 2016
Lecture XI
The Bernoulli distribution characterizes the coin toss. Specifically, there are two events X=0,1 with X=1 occurring with probability p. The probability distribution function P[X] can be written as:
Fall 2005 Lecture X 2
1[ ] 1xxP X p p
Next, we need to develop the probability of X+Y where both X and Y are identically distributed. If the two events are independent, the probability becomes:
Fall 2005 Lecture X 3
1 1 2
,
1 1 1x y x yx y x y
P X Y P X P Y
p p p p p p
Now, this density function is only concerned with three outcomes Z=X+Y={0,1,2}. There is only one way each for Z=0 or Z=2. Specifically for Z=0, X=0 and Y=0. Similarly, for Z=2, X=1 and Y=1. However, for Z=1 either X=1 and Y=0 or X=0 or Y=1. Thus, we can derive:
Fall 2005 Lecture X 4
Fall 2005 Lecture X 5
2 00
2 1 0 2 0 11 0 0 1
11
02
[ 0] 1
1 1, 0 0, 1
1 1
2 1
2 1
P Z p p
P Z P X Y P X Y
p p p p
p p
P Z p p
Next we expand the distribution to three independent Bernoulli events where Z=W+X+Y={0,1,2,3}.
Fall 2005 Lecture X 6
zz
yxwyxw
yxwyxw
pp
pp
pppppp
YPXPWP
YXWPZP
3
3
111
1
1
111
,,
Again, there is only one way for Z=0 and Z=3. However, there are now three ways for Z=1 or Z=2. Specifically, Z=1 if W=1, X=1 or Y=1. In addition, Z=2 if W=1 and X=1, W=1 and Y=1, and X=1 and Y=1. Thus the general distribution function for Z can now be written as:
Fall 2005 Lecture X 7
Fall 2005 Lecture X 8
3 00
3 1 0 0 3 0 1 01 0 0 0 1 0
3 0 0 1 20 0 1 1
3 1 1 0 3 1 0 11 1 0 1 0 1
3 0 1 1 10 1 1 2
03
0 1
1 1 1
1 3 1
2 1 1
1 3 1
3 1
P Z p p
P Z p p p p
p p p p
P Z p p p p
p p p p
P Z p p
Based on this development, the binomial distribution can be generalized as the sum of n Bernoulli events. For the case above, n=3. The distribution
function (ignoring the constants) can be written as:
Fall 2005 Lecture X 9
Fall 2005 Lecture X 10
3333
3
23232
13131
03030
13
12
11
10
ppCZP
ppCZP
ppCZP
ppCZP
rnrnr ppCrZP 1
The next challenge is to explain the Crn
term. To develop this consider the polynomial expression:
Fall 2005 Lecture X 11
32233
222
1
33
2
babbaaba
bababa
baba
This sequence can be linked to our discussion of the Bernoulli system by letting a=p and b=(1-p). What is of primary interest is the sequence of constants. This sequence is usually referred to as Pascal’s Triangle:
Fall 2005 Lecture X 12
Fall 2005 Lecture X 13
1
1 1
1 2 1
1 3 3 1
1 4 6 4 1
This series of numbers can be written as the combinatorial of n and r, or
Thus, any quadratic can be written as:
Fall 2005 Lecture X 14
!
! !nr
nC
r n r
n
r
rnrnr
n baCba1
As an aside, the quadratic form (a-b)n can be written as:
Fall 2005 Lecture X 15
n
r
rnrnr
rn
n
r
rnrnrnr
nn
baC
baC
baba
1
1
1
Thus, the binomial distribution X~B(n,p) is then written as:
Fall 2005 Lecture X 16
knknk ppCkXP 1
Next recalling Theorem 4.1.6: E[aX+bY]=aE[X]+bE[Y], the expectation of the binomial distribution function can be recovered from the Bernoulli distributions:
Fall 2005 Lecture X 17
Fall 2005 Lecture X 18
npp
pppp
Xpp
kppCXE
n
i
n
i
n
iX
iXX
n
k
knknk
i
ii
1
1
1001
1
1
0
0111
1
1
In addition, by Theorem 4.3.3:
Thus, variance of the binomial is simply the sum of the variances of the Bernoulli distributions or n times the variance of a single Bernoulli distribution
Fall 2005 Lecture X 19
n
i i
n
i i XVXV11
Fall 2005 Lecture X 20
pppp
ppppp
XEXEXV
1
01112
2210201
22
pnpXVn
i i 1
1