BEEX – ECE 5605, Virginia Tech 19 Expected value of r.v.’s CDF or PDF are complete (probabilistic) descriptions of the behavior of a random variable. Sometimes we are interested in less information; in a partial characterization. 0 10 20 30 40 50 60 70 80 90 100 -4 -2 0 2 4 6 8 10 trial # i X i Y different center different spread
41
Embed
Expected value of r.v.’s - webmail.aast.eduwebmail.aast.edu/~khedr/Courses/Undergraduate/Communication system II...BEEX – ECE 5605, Virginia Tech 20 Expected value or MEAN E[]Xtf
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
BEEX – ECE 5605, Virginia Tech 19
Expected value of r.v.’s
CDF or PDF are complete (probabilistic) descriptions of the behavior of a random variable. Sometimes we are interested in less information; in a partial characterization.
0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 1 0 0-4
-2
0
2
4
6
8
1 0
t r ia l #
iX
iY
different centerdifferent spread
BEEX – ECE 5605, Virginia Tech 20
Expected value or MEAN
[ ] ( )XE X t f t dt∞
−∞∫ [ ] ( )k X k
k
E X x p x∑defined if integral or sum converges absolutely
( )XE X t f t dt∞
−∞
⎡ ⎤ = < ∞⎣ ⎦ ∫ ( )k X kk
E X x p x⎡ ⎤ = < ∞⎣ ⎦ ∑
there are random variables for which the above do not convergewe then say “the mean does not exist”
[ ] represents the "center of mass"E X
arithmetic average of large # independent observations of a r.v.will tend to the mean; it’s like the “average of X”
BEEX – ECE 5605, Virginia Tech 21
Ex 3.29 mean of U[a,b]
[ ]
( )2 2
( )
1
2 2
X
b
a
E X t f t dt
t dtb a
b a b ab a
∞
−∞
=−
− += =
−
∫
∫
[ , ]U a b
1b a−
a b x→
( )Xf x ↑
2b a+
midpoint of interval [a,b]
BEEX – ECE 5605, Virginia Tech 23
Ex 3.31 mean of exponential r.v.
Time X between customer arrivals at a service station has an exponential PDF with parameter λ. Find the mean inter-arrival time.
[ ]0
0 0
0
1lim 0
t t t
tt
t
E X t e dt t e e dt
ete
λ λ λ
λλ
λ
λ λ
∞ ∞∞− − −
∞−−
→∞
= = +
= − + =−
∫ ∫udv uv vdu= −∫ ∫dvu
As λ is customer arrival rate in customers per second, themean inter-arrival time of λ-1 seconds/customer makes sense!
BEEX – ECE 5605, Virginia Tech 24
Ex 3.32 mean of geometric r.v.
N is the number of times a computer polls a terminal until the terminal has a message ready for transmission. Assuming the terminal produces messages according to a sequence of independent Bernoulli trials, N has a geometric distribution. Find the mean of N.
[ ]
( )
1
1
21 1
1
k
kE N k pq
ppq
∞−
=
=
= =−
∑
( )1
20 0
1 11 1
ddx
k k
k k
x kxx x
∞ ∞−
= =
= → =− −
∑ ∑
if probability of “success” is p, then it makes sensethat on average it takes p-1 trials to hit “success”
BEEX – ECE 5605, Virginia Tech 25
Expected value of a function of a r.v.
[ ] ( )YE Y t f t dt∞
−∞∫
ky h
kx x→
( )y g x= ↑
kh
[ ]
( )
( )
( )
( )
( )
k Y kk
k X k kk
X
E Y y f y h
g x f x h
g x f x dx∞
−∞
=
=
∑
∑
∫
alternative in terms of X
equivalent events
limit h→0
BEEX – ECE 5605, Virginia Tech 26
Ex 3.33
Sampling a sinusoid with random phase. Find the expected value of Y and of Y2,the power of Y.
( )cosY a tω= +Θ
( ]~ 0, 2U πΘ
[ ] ( )
( )
( )
( ) ( )
2
0
2
0
cos
1cos2
sin2
sin 2 sin 02
E Y E a t
a t d
a t
a t t
π
π
ω
ω θ θπ
ω θπ
ω π ωπ
= +Θ⎡ ⎤⎣ ⎦
= +
−= +
−= + − =⎡ ⎤⎣ ⎦
∫
( )
( )
( )
2 2 2
2
22
0
2
cos
1 cos 2 22
11 cos 2 22 2
2
E Y E a t
a E t
a t d
a
π
ω
ω
ω θ θπ
⎡ ⎤ ⎡ ⎤= +Θ⎣ ⎦ ⎣ ⎦
= + + Θ⎡ ⎤⎣ ⎦
⎡ ⎤= + +⎢ ⎥
⎣ ⎦
=
∫
agreement with time-averages: “DC” value of 0, power a2/2
BEEX – ECE 5605, Virginia Tech 28
Variance of XProvides information about a random variable – in addition to its mean value – regarding its deviations from the mean
[ ]D X E X−
[ ] [ ]( )2VAR X E X E X⎡ ⎤−
⎣ ⎦
deviation from the meanmeasures of “width/spread”
[ ] [ ] [ ][ ] [ ] [ ][ ]
2 2
2 2
2 2
2
2
VAR X E X E X X E X
E X E X E X E X
E X E X
⎡ ⎤= − +⎣ ⎦⎡ ⎤= − +⎣ ⎦⎡ ⎤= −⎣ ⎦
[ ] [ ]STD X VAR X standard deviation of r.v.
variance of r.v.
BEEX – ECE 5605, Virginia Tech 29
Ex 3.36 variance of X~U[a,b]
[ ]2
b aE X +=
[ ]
( )
( )
( ) ( )
( )( )
2
22322
22
12
13 12
b
a
b ab a
b ab a
b aVAR X x dxb a
b ayy dyb a b a
−−
− −− −
+⎛ ⎞= −⎜ ⎟− ⎝ ⎠
−= = =
− −
∫
∫
[ ] [ ] ( ) [ ] ( )( )24 24 2
2, 4 1; 32 12
U E X VAR X− −+ −
− → = = = =
BEEX – ECE 5605, Virginia Tech 5
3.38 variance of Gaussian r.v.
( )( )2
2212
x m
Xf x e xσ
σ π
− −
= −∞ < < ∞
( )2222
x m
e dxσσ π∞ − −
−∞
= ∫d
dσ
[ ] ( )( )2
22 2212
x m
VAR X x m e dxσ σσ π
∞ − −
−∞
= − =∫
( ) ( )22
2
232
x mx me dxσπ
σ
∞ − −
−∞
−= ∫
∞
−∞∫
BEEX – ECE 5605, Virginia Tech 6
Effect of VAR on the PDF of a Gaussian r.v.
-10 -8 -6 -4 -2 0 2 4 6 8 100
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
x
f X
PDF for Gaussian r.v.
VAR = 1VAR = 4VAR = 16
BEEX – ECE 5605, Virginia Tech 7
Mean & variance
Are the two most important parameters for (partially) characterizing the PDF of a r.v.Others are sometimes used, e.g.
which measures the degree of asymmetry about the meanSkewness is zero for a symmetric PDF
Each involves nth moment of r.v. X:
Under certain conditions, a PDF is completely specified if all moments are known (more on that later)
[ ]( )3
3
E X E Xskewness
σ
⎡ ⎤−⎣ ⎦
( )n nXE X x f x dx
∞
−∞
⎡ ⎤ =⎣ ⎦ ∫
[ ] [ ] [ ] [ ] [ ]20VAR c VAR X c VAR X VAR cX c VAR X= + = =
[ ] [ ]2 2VAR X E X E X⎡ ⎤= −⎣ ⎦
BEEX – ECE 5605, Virginia Tech 2
Multiple r.v.’s
Several r.v.’s at a timeMeasuring different quantities simultaneously
Engine oil pressure, RPM, generator voltageRepeated measurement of the same quantity
Sampling a waveform, such as EEG, speech
Joint behavior of two or more r.v.’sIndependence of sets of r.v.’sCorrelation if not independent
BEEX – ECE 5605, Virginia Tech 4
Ex 4.1
Let a random experiment be the selection of a student’s name from an urn: outcome ζ.
Define the following three functions:H(ζ) = student’s height, in inchesW(ζ) = student’s weight, in poundsA(ζ) = student’s age, in years
(H(ζ), W(ζ), A(ζ) ) is a vector r.v.
BEEX – ECE 5605, Virginia Tech 6
Events & probabilities
Each event involving an n-dimensional r.v. X=(X1, X2,…, Xn) has a corresponding region in an n-dimensional real space
E.g. 2-D r.v. X=(X,Y)
x→
y ↑
10
10
{ }10A X Y= + ≤ { }min( , ) 5B X Y= ≤
x→
y ↑
5
5
x→
y ↑
5
5
{ }2 2 25C X Y= + ≤
BEEX – ECE 5605, Virginia Tech 11
Independence
Intuitively, if r.v.’s X and Y are “independent,” then events that involve only X should be independent of events that involve only Y. In other words, if A1is any event that involves X only and A2 is anyevent that involves Y only, then
In general, n r.v.’s are independent if
[ ] [ ] [ ]1 2 1 2,P X A Y A P X A P Y A∈ ∈ = ∈ ∈
[ ] [ ] [ ]1 1 1 1, , n n n nP X A X A P X A P X A∈ ∈ = ∈ ∈where is an event that involves onlyk kA X
if r.v.’s are independent, knowing the probabilities of the r.v.’s in isolationsuffices to specify probabilities of joint events
BEEX – ECE 5605, Virginia Tech 12
Pairs of discrete r.v.’s ( )vector r.v. ,X Y=X
( ){ }, , 1, 2, , 1, 2,j kS x y j k= = =
joint probability mass function:
( ) { } { }, ,
, 1, 2, 1, 2,
X Y j k j k
j k
p x y P X x Y y
P X x Y y j k
⎡ ⎤= = =⎣ ⎦⎡ ⎤= = = =⎣ ⎦
∩
gives probability of occurrence of pairs (xj,yk)
For any event A: [ ] ( )( )
,,
,j k
X Y j kAx y
P A p x y∈
∈ = ∑ ∑X
( ) [ ],1 1
, 1X Y j kj k
p x y P S∞ ∞
= =
= =∑∑
BEEX – ECE 5605, Virginia Tech 13
Marginal PMF’s
( ) ( ){ } { }{ } { } { }{ }
( )
( ) ( )
,
1 2
,1
,1
, any
,
,
X j X Y j k
j j
X Y j kk
Y k X Y j kj
p x p x y
P X x Y y X x Y y
p x y
p y p x y
∞
=
∞
=
=
⎡ ⎤= = = = =⎣ ⎦
=
=
∑
∑
∩ ∪ ∩ ∪
.2
.1.1
.2
.2
.1
.1
.2
.2
.3
.2
.1
Y
.1 .2 .2 .2 .2.1X
marginal PMF’s areinsufficient – in general -for specifying joint PMF
( ) ( ) ( ), , ,X Y j k X j Y k j kp x y p x p y x y≠ ∀not independent
BEEX – ECE 5605, Virginia Tech 14
Independence of two r.v.’s in general
( ) ( ) ( ), , ,X Y j k X j Y k j kp x y p x p y x y= ∀
( ) ( ) ( ), , ,X Y X YF x y F x F y x y= ∀
( ) ( ) ( ), , ,X Y X Yf x y f x f y x y= ∀
IFF
discrete
jointly continuous
BEEX – ECE 5605, Virginia Tech 15
Ex 4.17 back to Ex 4.11
( ),2 0
,0 . .
x y
X Ye e y x
f x yo w
− −⎧ ≤ ≤ < ∞= ⎨⎩
( ) ( )0
2 2 1 0x x y x x
Xf x e e dy e e x− − − −= = − ≤ < ∞∫
( ) ( ) 22 2 2 0x y y y yY y
f y e e dx e e e y∞ − − − − −= = = ≤ < ∞∫
( ) 22 2 1 2x y x x ye e e e e− − − − −≠ −
joint
marginal PDFs
product?
x→
y ↑
0
y x=
X and Y are not independent
BEEX – ECE 5605, Virginia Tech 16
Ex 4.18 back to Ex 4.13
( )( )
( )2 2
2
2
2 1, 2
1,2 1
x xy y
X Yf x y eρ
ρ
π ρ
− − +
−=
−
( )2
2
2
x
Xef x
π
−
=
( ) ( )( )2 222
2 2 2
22 2
x yyx
X Ye e ef x f y
ππ π
− +−−
= =
( )2
2
2
y
Yef y
π
−
= iff 0ρ≡ =
BEEX – ECE 5605, Virginia Tech 18
Conditional probability
Many r.v.’s are not independent, such as when measuring one variable to learn about another one, or when sampling slowly relative to the rate of change of a signal.
We’re then interested in the probability of an event related to Y, given the knowledge of X=x (a measurement)
We’ll see that E[Y|X=x] is of significance also
BEEX – ECE 5605, Virginia Tech 19
[ ] [ ][ ]
,|
P Y A X xP Y A X x
P X x∈ =
∈ = ==
X discrete
( ) [ ][ ] [ ],
| for 0kY k k
k
P Y y X xF y x P X x
P X x≤ =
= = >=
( ) ( )| |Y k Y kdf y x F y xdy
=
if derivative exists
[ ] ( )| |k Y ky A
P Y A X x f y x dy∈
∈ = = ∫
independence
( ) ( )|Y YF y x F y=
( ) ( )|Y Yf y x f y=
Conditional probability
BEEX – ECE 5605, Virginia Tech 20
Conditional probability
[ ] [ ][ ]
,|
P Y A X xP Y A X x
P X x∈ =
∈ = ==
X and Y discrete
( ) [ ]( )( ) [ ] ( )
[ ] ( )
,
,| |
,for 0
0 for 0
k jY j k j k
k
X Y k jk X k
X k
k X k
P X x Y yp y x P Y y X x
P X x
p x yP X x p x
p xP X x p x
⎡ ⎤= =⎣ ⎦⎡ ⎤= = = =⎣ ⎦ =
⎧⎪ = = >
= ⎨⎪ = = =⎩
δ functions~ PMF weights
[ ] ( ) ( )| |j j
k Y j k Y jy A y A
P Y A X x p y x p y∈ ∈
∈ = = =∑ ∑if also independent
in general
BEEX – ECE 5605, Virginia Tech 21
Ex 4.20 back to Ex 4.14
X is the input and Y the output of a communication channel. P[Y<0|X=+1]?