Chapter 5. Operations on Multiple R. V.'s 1 Chapter 5. Operations on Multiple Random Variables 0. Introduction 1. Expected Value of a Function of Random.
Post on 21-Dec-2015
217 Views
Preview:
Transcript
Chapter 5. Operations on Multiple R. V.'s1
Chapter 5. Operations on Multiple Random Variables
0. Introduction
1. Expected Value of a Function of Random Variables
2. Joint Characteristic Functions
3. Jointly Gaussian Random Variables
4. Transformations of Multiple Random Variables
5. Linear Transformations of Gaussian Random Variables
6. Computer Generation of Multiple Random Variables
7. Sampling and Some Limit Theorems
8. Complex Random Variables
Chapter 5. Operations on Multiple R. V.'s2
5.1 Expected Value of a Function of Random Variables
,[ ( , )] ( , ) ( , )X YE g X Y g x y f x y dxdy
11 1 , , 1 1[ ( , , )] ( , , ) ( , , )NN N X X N NE g X X g x x f x x dx dx
, ,[ ( )] ( ) ( , ) ( ) ( , )X Y X YE g X g x f x y dxdy g x f x y dydx
( ) ( )Xg x f x dx
Chapter 5. Operations on Multiple R. V.'s3
5.1 Expected Value of a Function of Random Variables
11 1 1
[ ( , , )] [ ] [ ] [ ]N N N
N i i i i i ii i i
E g X X E X E X E X
Ex 5.1-1:
11
( , , )N
N i ii
g X X X
Joint moments
,[ ] ( , )n k n knk X Ym E X Y x y f x y dxdy
10 [ ]m E X 01 [ ]m E Y 11 [ ]m E XY
Correlation of & :X Y 11[ ]XYR E XY m
Chapter 5. Operations on Multiple R. V.'s4
5.1 Expected Value of a Function of Random Variables
Ex 5.1-2:
& -- uncorrelated [ ] [ ] [ ]X Y E XY E X E Y
& -- orthogonal [ ] 0X Y E XY
independent uncorrelated
,[ ] ( , ) ( ) ( )X Y X YE XY xyf x y dxdy xyf x f y dxdy
( ) ( ) [ ] [ ]X Yxf x dx yf y dy E X E Y
[ ] 3E X 2[ ] 11E X 6 22Y X
[ ] 6 [ ] 22 4E Y E X 2[ ] [ ( 6 22)] 6 [ ] 22 [ ] 0XYR E XY E X X E X E X orthogonal
Chapter 5. Operations on Multiple R. V.'s5
5.1 Expected Value of a Function of Random Variables
[ ] [ ] [ ] 12XYR E XY E X E Y NOT uncorrelated
Y aX b
2[ ] [ ( )] [ ] [ ]E XY E X aX b aE X bE X 2[ ] [ ] [ ] [ ] ( [ ]) [ ]E X E Y E X E aX b a E X bE X
0 [ ] [ ] [ ] uncorrelateda E XY E X E Y
2[ ]orthogonal
[ ]
b E X
a E X
3 2 53,2,5 1 2 3[ ]m E X X X
Chapter 5. Operations on Multiple R. V.'s6
5.1 Expected Value of a Function of Random Variables
Joint central moments
[( ) ( ) ]n knk E X X Y Y
10 [ ] 0E X X
Covariance of & :X Y
01 [ ] 0E Y Y
2 220 [( ) ] XE X X
2 202 [( ) ] YE Y Y
11 [( )( )]XYC E X X Y Y
[( )( )] [ ]XYC E X X Y Y E XY XY YX XY
[ ] [ ] [ ] [ ] [ ]XYE XY XE Y YE X XY R E X E Y
uncorrelated 0XYC
Chapter 5. Operations on Multiple R. V.'s7
5.1 Expected Value of a Function of Random Variables
Correlation coefficient of & :X Y
11
20 02
( ) ( )XY
X Y X Y
C X X Y YE
orthogonal [ ] [ ]XYC E X E Y
XY X YC
Schwarz's inequality 2 2 2( [ ]) [ ] [ ]E XY E X E YPF:
2 2 2 2 2 2 20 [( ) ] [ 2 ] [ ] 2 [ ] [ ]E aX Y E a X aXY Y a E X aE XY E Y 2 2 2' ( [ ]) [ ] [ ] 0D E XY E X E Y
a
2[ ] 0E X
2[ ] 0E X 2[ ] 0 & [ ] 0E XY E Y
Chapter 5. Operations on Multiple R. V.'s8
5.1 Expected Value of a Function of Random Variables
11
20 02
( ) ( )XY
X Y X Y
C X X Y YE
by Schwarz's inequalityPF:
1 1
22 2
22 2
( ) ( ) ( ) ( )1
X Y X Y
X X Y Y X X Y YE E E
uncorrelated 0
1Y X 2 1Y X
Chapter 5. Operations on Multiple R. V.'s9
5.1 Expected Value of a Function of Random Variables
Ex 5.1-3:
1
N
i ii
X X
1 1
[ ] [ ] [ ]N N
i i i ii i
E X E X E X
1
1N
ii
1
[ ] ( )N
i i ii
X E X X X
2 2
1 1
[( ) ] ( ) ( )N N
X i i i j j ji j
E X X E X X X X
,1 1 1 1
[( )( )]i j
N N N N
i j i i j j i j X Xi j i j
E X X X X C
2
,
,
0, i
i j
XX X
i jC
i j
2 2 2
1i
N
X i Xi
's uncorrelatediX
Chapter 5. Operations on Multiple R. V.'s10
5.2 Joint Characteristic Functions
1 2 1 2, 1 2 ,( , ) [ ] ( , )j X j Y j x j y
X Y X YE e f x y e dxdy
2-dim Fourier transform
1 2, , 1 2 1 22
1( , ) ( , )
(2 )j x j y
X Y X Yf x y e d d
1 , 1( ) ( ,0)X X Y
2 , 2( ) (0, )Y X Y marginal characteristic function
1 2
, 1 2
1 2 0
( , )( )
n kX Yn k
nk n km j
Chapter 5. Operations on Multiple R. V.'s11
5.2 Joint Characteristic Functions
Ex 5.2-1: 2 21 22 8
, 1 2( , )X Y e
2 21 2
1 21 2
, 1 2 2 810 1
01 0
( , )[ ] 4 0X YE X m j j e
2 21 2
1 21 2
, 1 2 2 801 2
02 0
( , )[ ] 16 0X YE Y m j j e
2 21 2
1 2
1 2
2, 1 2 2 82
11 1 20
1 2 0
( , )( ) 64 0X Y
XYR m j e
[ ] [ ] 0 uncorrelatedXY XYC R E X E Y
Chapter 5. Operations on Multiple R. V.'s12
5.2 Joint Characteristic Functions
Ex 5.2-2:3
1 1 2 2 3 3 1
1 2 3
3
, , 1 2 3 1 2 31
( , , ) [ ] ( )i i
i
i
j xj X j X j X
X X X X ii
E e f x e dx dx dx
1 2 3Y X X X 's indep.iX
3 3
1 1
( ) ( )i i
i i
j xX i i X i
i i
f x e dx
1 2 3
1 2 3
3
, ,1
( ) [ ] [ ] ( , , ) ( )i
j X j X j Xj YY X X X X
i
E e E e
3
1
1( ) ( )
2 i
j yY X
i
f y e d
31( ) ( )
2j y
Y Xf y e d
's iiid X
Chapter 5. Operations on Multiple R. V.'s13
5.3 Jointly Gaussian Random Variables
2 2
2 2 2
1 ( ) 2 ( )( ) ( )
2(1 ), 2
1( , )
2 1
X YX Y
x X x X y Y y Y
X Y
X Y
f x y e
, ,
2
( , ) ( , )
1
2 1
X Y X Y
X Y
f x y f X Y
[ ]E X X [ ]E Y Y2 2[( ) ] XE X X
[( )( )] X YE X X Y Y
Chapter 5. Operations on Multiple R. V.'s14
5.3 Jointly Gaussian Random Variables
2
2
( )
2,
1( ) ( , )
2X
x X
X X Y
X
f x f x y dy e
2
2
( )
2,
1( ) ( , )
2Y
y Y
Y X Y
Y
f y f x y dx e
,0 ( , ) ( ) ( )X Y X Yf x y f x f y
jointly gaussian & uncorr. indep.
Ex 5.3-1:
1 cos sinY X Y 2 sin cosY X Y
Chapter 5. Operations on Multiple R. V.'s15
5.3 Jointly Gaussian Random Variables
1 2, 1 1 2 2[( )( )]Y YC E Y Y Y Y
[{( )cos ( )sin }{ ( )sin ( )cos }]E X X Y Y X X Y Y
2 2 2 2( )sin cos [cos sin ]Y X XYC
2 2 2 21 1( )sin 2 cos 2 ( )sin 2 cos2
2 2Y X XY Y X X YC
1 2
1, 2 2
210 tan
2X Y
Y YX Y
C
Chapter 5. Operations on Multiple R. V.'s16
5.3 Jointly Gaussian Random Variables
1 1
2 2
3 3
x X
x X x X
x X
11 12 13
21 22 23
31 32 33
X
C C C
C C C C
C C C
1
1 2 3
1( ) ( )
2, , 1 2 3 1/ 2/ 2
1( , , )
(2 )
TXx X C x X
X X X NX
f x x x eC
1 1 21 1 2
1 2 2
1 2 2
22
122
2
1
1
11
X X XX X X
X X
X X X
X X X
C C
1 2
2 2 2(1 )X X XC
Chapter 5. Operations on Multiple R. V.'s17
5.3 Jointly Gaussian Random Variables
1 2 3Properties of jointly gaussian r.v.'s , , & :X X X
1. 1st & 2nd moments p.d.f
2. uncorr. indep.
3. linear transforms of gaussian r.v.'s is also jointly gaussian.
1 2, 1 24. marginal density ( , ) is also jointly gaussian.X Xf x x
1 2 3 1 2 3,5. conditional density ( , ) is also jointly gaussian.X X Xf x x x
Chapter 5. Operations on Multiple R. V.'s18
5.4 Transformations of Multiple Random Variables
One function 1 2( , )Y g X X1 2, 1 2( , )X Xf x x
1 2
1 2
1 2 , 1 2 1 2
( , )
( ) [ ( , ) ] ( , )Y X X
g x x y
F y P g X X y f x x dx dx
( )( ) Y
Y
dF yf y
dy
Ex 5.4-1: 1 2positive r.v.'s & X X 1
2
XY
X
2
1 2
1, 1 2 1 20 0
2
( ) [ ] ( , )yx
Y X X
XF y P y f x x dx dx
X
0y
Chapter 5. Operations on Multiple R. V.'s19
5.4 Transformations of Multiple Random Variables
1 22 , 2 2 20
( )( ) ( , )Y
Y X X
dF yf y x f yx x dx
dy
0y
Chapter 5. Operations on Multiple R. V.'s20
5.4 Transformations of Multiple Random Variables
1 2, 1 2( , )X Xf x x
2 22 2
( ) ( , ) ( , )( ) ( , ) ( , )
y yY
Y y y
dF y I y x I y xf y I y y I y y dx dx
dy y y
Ex 5.4-2: 2 21 2Y X X
2 22
2 2 1 22
2 21 2 , 1 2 1 2( ) [ ] ( , )
y y x
Y X Xy y xF y P X X y f x x dx dx
2 2( , )
y
yI y x dx
2 22
2 2 1 22
2 , 1 2 1( , ) ( , )y x
X Xy xI y x f x x dx
1 2 1 2
2 2 2 22, 2 2 , 2 22 2 2 2
2 2
( , )( , ) ( , )X X X X
I y x y yf y x x f y x x
y y x y x
Chapter 5. Operations on Multiple R. V.'s21
5.4 Transformations of Multiple Random Variables
22
( , )( )
y
Y y
I y xf y dx
y
1 2 1 2
2 2 2 2, 2 2 , 2 2 22 2
2
{ ( , ) ( , )}y
X X X Xy
yf y x x f y x x dx
y x
HW: Solve Problem 5.4-3.2 21 1 2 2
2 2
1 2
2
2 (1 ), 1 2 2 2
1( , )
2 1X
x x x x
X X
X
f x x e
2 21 2Y X X
( ) ?Yf y
0
Chapter 5. Operations on Multiple R. V.'s22
5.4 Transformations of Multiple Random Variables
Multiple functions 1 1 1 21 2
2 2 1 2
( , )( , )
( , )
Y T X XT X X
Y T X X
1 2, 1 2( , )X Xf x x
1 2 1 2, 1 2 , 1 2( , ) ( , )Y Y X Xf y y f x x J
11 1 1 21
1 2 12 2 1 2
( , )( , )
( , )
x T y yT y y
x T y y
1 11 1
1 2
1 12 2
1 2
T T
y yJ
T T
y y
jacobian
Chapter 5. Operations on Multiple R. V.'s23
5.4 Transformations of Multiple Random Variables
1 1 1 2 1 2 11 2
2 2 1 2 1 2 2
( , )( , )
( , )
Y T X X aX bX Xa bT X X
Y T X X cX dX c d X
Ex 5.4-3:
1 2
1 2
1 2 1 2,
, 1 2
( , )( , )
X X
Y Y
dy by cy ayf
ad bc ad bcf y yad bc
11 1 1 2 11
1 2 12 22 1 2
( , ) 1( , )
( , )
x T y y yd bT y y
x c a yad bcT y y
1 11 1
1 2
1 12 2
1 2
1
T T
y yJ
ad bcT T
y y
Chapter 5. Operations on Multiple R. V.'s24
5.5 Linear Transformations of Gaussian Random Variables
1 1 11 12
21 222 2
Y X a aY A AX X
a aY X
[ ] [ ] [ ]E Y E AX AE X
[( )( ) ] [ ( )( ) ]T T TYC E Y Y Y Y E A X X X X A
[( )( ) ]T T TXAE X X X X A AC A 1 1 1T
Y XC A C A
1
1 2
1( ) ( )
2, 1 2 1/ 2/ 2
1( , )
(2 )
TXx X C x X
X X NX
f x x eC
1 2, 1 2( , ) ?Y Yf y y
2det( ) det( ) det( )Y XC A C
Chapter 5. Operations on Multiple R. V.'s25
5.5 Linear Transformations of Gaussian Random Variables
1 1 11( ) ( )
21/ 2/ 2
1
(2 ) det( )
TXA y X C A y X
NX
eC A
1 2 1 2
1, 1 2 ,
1( , ) ( )
det( )Y Y X Xf y y f A yA
1 1 1 1 1( ) ( ) ( ) ( )T T TX XA y X C A y X y Y A C A y Y
1( ) ( )TYy Y C y Y
1
1 2
1( ) ( )
2, 1 2 1/ 2/ 2
1( , )
(2 )
TYy Y C y Y
Y Y NY
f y y eC
1/ 2 1/ 2det( )X YC A C
(gaussian)
Chapter 5. Operations on Multiple R. V.'s26
5.5 Linear Transformations of Gaussian Random Variables
1
1 2
1( ) ( )
2, 1 2 1/ 2
1( , )
(2 )
TYy Y C y Y
Y Y
Y
f y y eC
Ex 5.5-1: 1 1 1
2 2 2
1 2
3 4
Y X XA
Y X X
1
2
[ ] 0
[ ] 0
E X
E X
4 3
3 9XC
1 1
2 2
[ ] [ ] 0
[ ] [ ] 0
E Y E XA
E Y E X
1 2 4 3 1 3 28 66
3 4 3 9 2 4 66 252T
Y XC AC A
1 2
1 2
660.786
28 252
YY
Y Y
C
Chapter 5. Operations on Multiple R. V.'s27
5.6 Computer Generation of Multiple Random Variables
"OMITTED"
Chapter 5. Operations on Multiple R. V.'s28
5.7 Sampling and Some Limit Theorems
sampling and estimation
samples estimate
1
1 N
N nn
x xN
-- average of samplesN
1
1ˆN
N nn
x xN
-- estimate of average of samplesN
estimator of mean of r.v. X
1
1ˆN
N nn
X XN
(sample mean)r.v.
Chapter 5. Operations on Multiple R. V.'s29
5.7 Sampling and Some Limit Theorems
estimator of power of r.v. X
2 2
1
1 N
N nn
X XN
estimator of variance of r.v. X
2 2
1
1 ˆ( )1
N
X n Nn
X XN
1 1
1 1ˆ[ ] [ ] [ ]N N
N n nn n
E X E X E X XN N
(sample mean estimator is unbiased.)
22
1 1 1 1
1 1 1ˆ[ ] [ ] [ ]N N N N
N n n n mn n n m
E X E X X E X XN N N
2[ ] [ ]n mn m E X X E X 2[ ]n mn m E X X X
indep.2 2 2 2 2
2
1 1{ [ ] ( ) } { ( 1) }NE X N N X X N X
N N
Chapter 5. Operations on Multiple R. V.'s30
5.7 Sampling and Some Limit Theorems
2 2 2 2 2 21 1 1{ [ ] ( 1) } { [ ] } XE X N X X E X X
N N N
2
2ˆ
2 2ˆ 1 1NX XNP X X
N
(Chebychev's ineq.)
1N
ˆN NX X w.p. 1 (with probability 1)
Ex 5.7-1: 0.05X 50N
ˆ 0.95NP X X 2
21 0.95 160
50(0.05 )X
XXX
2 2 2 2 2 2ˆ
ˆ ˆ ˆ ˆ[( ) ] [ 2 ] [ ]N
N N N NXE X X E X XX X E X X
Chapter 5. Operations on Multiple R. V.'s31
5.7 Sampling and Some Limit Theorems
2 2
1
1 N
N nn
X XN
2 2
1
1 ˆ( )1
N
X n Nn
X XN
2 2 2 2 2
1 1
1 1[ ] [ ] [ ] [ ]
N N
N nn n
E X E X E X E X XN N
(unbiased)
2 2 2 2
1 1
1 1ˆ ˆ ˆ[ ] [( ) ] [ 2 ]1 1
N N
X n N n n N Nn n
E E X X E X X X XN N
2 2
1 1 1 1 1
1 1ˆ[ ] [ ] [ ] ( 1)N N N N N
n N n m n mn n m n m
E X X E X X E X X X N XN N
2 2 2 2 22 2 2 22{ ( 1) } { ( 1) }
[ ]1X X
N X X N X X N XE X X
N
(unbiased)
Chapter 5. Operations on Multiple R. V.'s32
5.7 Sampling and Some Limit Theorems
2 2 2 2
1
1 1ˆ( ) [(0.1 3.973) (12.0 3.973) ]1 10
N
X n Nn
X XN
1
1 1ˆ (0.1 0.4 0.9 12.0) 3.97311
N
N nn
X X VN
Ex 5.7-2:
11 samples (0.1, 0.4, 0.9,1.4, 2.0, 2.8, 3.7, 4.8, 6.4, 9.2,12.0 )V
sample mean
sample variance
214.75V
[ ] 4E X V 2 16X
Chapter 5. Operations on Multiple R. V.'s33
5.7 Sampling and Some Limit Theorems
ˆlim [ [ ] ] 1, 0NN
P X E X
Ex 5.7-3:
Weak Law of Large Numbers
Strong Law of Large Numbers
ˆ{lim [ ]} 1NN
P X E X
4
2 2 3
4 ( )( )
( )Y
a yu yf y
a y
[ ]
4
aE Y
2 22 (16 )
16Y
a
Chapter 5. Operations on Multiple R. V.'s34
5.8 Complex Random Variables
Z X jY
[ ] [ ] [ ] [ ]Z E Z E X jY E X jE Y X jY 22 *[( ) ( )] [ ]Z E Z Z Z Z E Z Z
mean
variance
1 1 1Z X jY 2 2 2Z X jY
1 1 2 2 1 1 2 21 2 , , , 1 1 2 2 , 1 1 , 2 2 & indep. ( , , , ) ( , ) ( , )X Y X Y X Y X YZ Z f x y x y f x y f x y
1 2
*, 1 2[ ]Z ZR E Z Z
1 2 1 2
* *, 1 1 2 2 , 1 2[( ) ( )]Z Z Z ZC E Z Z Z Z R Z Z
1 2 1 2
*1 2 1 2 & uncorr. C 0 RZ Z Z ZZ Z Z Z
1 21 2 & orthogonal R 0Z ZZ Z
[ ] ( ) ( , )XYE X jY x jy f x y dxdy
top related