Chapter 5. Operations on Multiple R. V.'s 1 Chapter 5. Operations on Multiple Random Variables 0. Introduction 1. Expected Value of a Function of Random.

Chapter 5. Operations on Multiple R. V.'s1

Chapter 5. Operations on Multiple Random Variables

0. Introduction

1. Expected Value of a Function of Random Variables

2. Joint Characteristic Functions

3. Jointly Gaussian Random Variables

4. Transformations of Multiple Random Variables

5. Linear Transformations of Gaussian Random Variables

6. Computer Generation of Multiple Random Variables

7. Sampling and Some Limit Theorems

8. Complex Random Variables

Chapter 5. Operations on Multiple R. V.'s2

5.1 Expected Value of a Function of Random Variables

,[ ( , )] ( , ) ( , )X YE g X Y g x y f x y dxdy

11 1 , , 1 1[ ( , , )] ( , , ) ( , , )NN N X X N NE g X X g x x f x x dx dx

, ,[ ( )] ( ) ( , ) ( ) ( , )X Y X YE g X g x f x y dxdy g x f x y dydx

( ) ( )Xg x f x dx

Chapter 5. Operations on Multiple R. V.'s3

5.1 Expected Value of a Function of Random Variables

11 1 1

[ ( , , )] [ ] [ ] [ ]N N N

N i i i i i ii i i

E g X X E X E X E X

Ex 5.1-1:

11

( , , )N

N i ii

g X X X

Joint moments

,[ ] ( , )n k n knk X Ym E X Y x y f x y dxdy

10 [ ]m E X 01 [ ]m E Y 11 [ ]m E XY

Correlation of & :X Y 11[ ]XYR E XY m

Chapter 5. Operations on Multiple R. V.'s4

5.1 Expected Value of a Function of Random Variables

Ex 5.1-2:

& -- uncorrelated [ ] [ ] [ ]X Y E XY E X E Y

& -- orthogonal [ ] 0X Y E XY

independent uncorrelated

,[ ] ( , ) ( ) ( )X Y X YE XY xyf x y dxdy xyf x f y dxdy

( ) ( ) [ ] [ ]X Yxf x dx yf y dy E X E Y

[ ] 3E X 2[ ] 11E X 6 22Y X

[ ] 6 [ ] 22 4E Y E X 2[ ] [ ( 6 22)] 6 [ ] 22 [ ] 0XYR E XY E X X E X E X orthogonal

Chapter 5. Operations on Multiple R. V.'s5

5.1 Expected Value of a Function of Random Variables

[ ] [ ] [ ] 12XYR E XY E X E Y NOT uncorrelated

Y aX b

2[ ] [ ( )] [ ] [ ]E XY E X aX b aE X bE X 2[ ] [ ] [ ] [ ] ( [ ]) [ ]E X E Y E X E aX b a E X bE X

0 [ ] [ ] [ ] uncorrelateda E XY E X E Y

2[ ]orthogonal

[ ]

b E X

a E X

3 2 53,2,5 1 2 3[ ]m E X X X

Chapter 5. Operations on Multiple R. V.'s6

5.1 Expected Value of a Function of Random Variables

Joint central moments

[( ) ( ) ]n knk E X X Y Y

10 [ ] 0E X X

Covariance of & :X Y

01 [ ] 0E Y Y

2 220 [( ) ] XE X X

2 202 [( ) ] YE Y Y

11 [( )( )]XYC E X X Y Y

[( )( )] [ ]XYC E X X Y Y E XY XY YX XY

[ ] [ ] [ ] [ ] [ ]XYE XY XE Y YE X XY R E X E Y

uncorrelated 0XYC

Chapter 5. Operations on Multiple R. V.'s7

5.1 Expected Value of a Function of Random Variables

Correlation coefficient of & :X Y

11

20 02

( ) ( )XY

X Y X Y

C X X Y YE

orthogonal [ ] [ ]XYC E X E Y

XY X YC

Schwarz's inequality 2 2 2( [ ]) [ ] [ ]E XY E X E YPF:

2 2 2 2 2 2 20 [( ) ] [ 2 ] [ ] 2 [ ] [ ]E aX Y E a X aXY Y a E X aE XY E Y 2 2 2' ( [ ]) [ ] [ ] 0D E XY E X E Y

a

2[ ] 0E X

2[ ] 0E X 2[ ] 0 & [ ] 0E XY E Y

Chapter 5. Operations on Multiple R. V.'s8

5.1 Expected Value of a Function of Random Variables

11

20 02

( ) ( )XY

X Y X Y

C X X Y YE

by Schwarz's inequalityPF:

1 1

22 2

22 2

( ) ( ) ( ) ( )1

X Y X Y

X X Y Y X X Y YE E E

uncorrelated 0

1Y X 2 1Y X

Chapter 5. Operations on Multiple R. V.'s9

5.1 Expected Value of a Function of Random Variables

Ex 5.1-3:

1

N

i ii

X X

1 1

[ ] [ ] [ ]N N

i i i ii i

E X E X E X

1

1N

ii

1

[ ] ( )N

i i ii

X E X X X

2 2

1 1

[( ) ] ( ) ( )N N

X i i i j j ji j

E X X E X X X X

,1 1 1 1

[( )( )]i j

N N N N

i j i i j j i j X Xi j i j

E X X X X C

2

,

,

0, i

i j

XX X

i jC

i j

2 2 2

1i

N

X i Xi

's uncorrelatediX

Chapter 5. Operations on Multiple R. V.'s10

5.2 Joint Characteristic Functions

1 2 1 2, 1 2 ,( , ) [ ] ( , )j X j Y j x j y

X Y X YE e f x y e dxdy

2-dim Fourier transform

1 2, , 1 2 1 22

1( , ) ( , )

(2 )j x j y

X Y X Yf x y e d d

1 , 1( ) ( ,0)X X Y

2 , 2( ) (0, )Y X Y marginal characteristic function

1 2

, 1 2

1 2 0

( , )( )

n kX Yn k

nk n km j

Chapter 5. Operations on Multiple R. V.'s11

5.2 Joint Characteristic Functions

Ex 5.2-1: 2 21 22 8

, 1 2( , )X Y e

2 21 2

1 21 2

, 1 2 2 810 1

01 0

( , )[ ] 4 0X YE X m j j e

2 21 2

1 21 2

, 1 2 2 801 2

02 0

( , )[ ] 16 0X YE Y m j j e

2 21 2

1 2

1 2

2, 1 2 2 82

11 1 20

1 2 0

( , )( ) 64 0X Y

XYR m j e

[ ] [ ] 0 uncorrelatedXY XYC R E X E Y

Chapter 5. Operations on Multiple R. V.'s12

5.2 Joint Characteristic Functions

Ex 5.2-2:3

1 1 2 2 3 3 1

1 2 3

3

, , 1 2 3 1 2 31

( , , ) [ ] ( )i i

i

i

j xj X j X j X

X X X X ii

E e f x e dx dx dx

1 2 3Y X X X 's indep.iX

3 3

1 1

( ) ( )i i

i i

j xX i i X i

i i

f x e dx

1 2 3

1 2 3

3

, ,1

( ) [ ] [ ] ( , , ) ( )i

j X j X j Xj YY X X X X

i

E e E e

3

1

1( ) ( )

2 i

j yY X

i

f y e d

31( ) ( )

2j y

Y Xf y e d

's iiid X

Chapter 5. Operations on Multiple R. V.'s13

5.3 Jointly Gaussian Random Variables

2 2

2 2 2

1 ( ) 2 ( )( ) ( )

2(1 ), 2

1( , )

2 1

X YX Y

x X x X y Y y Y

X Y

X Y

f x y e

, ,

2

( , ) ( , )

1

2 1

X Y X Y

X Y

f x y f X Y

[ ]E X X [ ]E Y Y2 2[( ) ] XE X X

[( )( )] X YE X X Y Y

Chapter 5. Operations on Multiple R. V.'s14

5.3 Jointly Gaussian Random Variables

2

2

( )

2,

1( ) ( , )

2X

x X

X X Y

X

f x f x y dy e

2

2

( )

2,

1( ) ( , )

2Y

y Y

Y X Y

Y

f y f x y dx e

,0 ( , ) ( ) ( )X Y X Yf x y f x f y

jointly gaussian & uncorr. indep.

Ex 5.3-1:

1 cos sinY X Y 2 sin cosY X Y

Chapter 5. Operations on Multiple R. V.'s15

5.3 Jointly Gaussian Random Variables

1 2, 1 1 2 2[( )( )]Y YC E Y Y Y Y

[{( )cos ( )sin }{ ( )sin ( )cos }]E X X Y Y X X Y Y

2 2 2 2( )sin cos [cos sin ]Y X XYC

2 2 2 21 1( )sin 2 cos 2 ( )sin 2 cos2

2 2Y X XY Y X X YC

1 2

1, 2 2

210 tan

2X Y

Y YX Y

C

Chapter 5. Operations on Multiple R. V.'s16

5.3 Jointly Gaussian Random Variables

1 1

2 2

3 3

x X

x X x X

x X

11 12 13

21 22 23

31 32 33

X

C C C

C C C C

C C C

1

1 2 3

1( ) ( )

2, , 1 2 3 1/ 2/ 2

1( , , )

(2 )

TXx X C x X

X X X NX

f x x x eC

1 1 21 1 2

1 2 2

1 2 2

22

122

2

1

1

11

X X XX X X

X X

X X X

X X X

C C

1 2

2 2 2(1 )X X XC

Chapter 5. Operations on Multiple R. V.'s17

5.3 Jointly Gaussian Random Variables

1 2 3Properties of jointly gaussian r.v.'s , , & :X X X

1. 1st & 2nd moments p.d.f

2. uncorr. indep.

3. linear transforms of gaussian r.v.'s is also jointly gaussian.

1 2, 1 24. marginal density ( , ) is also jointly gaussian.X Xf x x

1 2 3 1 2 3,5. conditional density ( , ) is also jointly gaussian.X X Xf x x x

Chapter 5. Operations on Multiple R. V.'s18

5.4 Transformations of Multiple Random Variables

One function 1 2( , )Y g X X1 2, 1 2( , )X Xf x x

1 2

1 2

1 2 , 1 2 1 2

( , )

( ) [ ( , ) ] ( , )Y X X

g x x y

F y P g X X y f x x dx dx

( )( ) Y

Y

dF yf y

dy

Ex 5.4-1: 1 2positive r.v.'s & X X 1

2

XY

X

2

1 2

1, 1 2 1 20 0

2

( ) [ ] ( , )yx

Y X X

XF y P y f x x dx dx

X

0y

Chapter 5. Operations on Multiple R. V.'s19

5.4 Transformations of Multiple Random Variables

1 22 , 2 2 20

( )( ) ( , )Y

Y X X

dF yf y x f yx x dx

dy

0y

Chapter 5. Operations on Multiple R. V.'s20

5.4 Transformations of Multiple Random Variables

1 2, 1 2( , )X Xf x x

2 22 2

( ) ( , ) ( , )( ) ( , ) ( , )

y yY

Y y y

dF y I y x I y xf y I y y I y y dx dx

dy y y

Ex 5.4-2: 2 21 2Y X X

2 22

2 2 1 22

2 21 2 , 1 2 1 2( ) [ ] ( , )

y y x

Y X Xy y xF y P X X y f x x dx dx

2 2( , )

y

yI y x dx

2 22

2 2 1 22

2 , 1 2 1( , ) ( , )y x

X Xy xI y x f x x dx

1 2 1 2

2 2 2 22, 2 2 , 2 22 2 2 2

2 2

( , )( , ) ( , )X X X X

I y x y yf y x x f y x x

y y x y x

Chapter 5. Operations on Multiple R. V.'s21

5.4 Transformations of Multiple Random Variables

22

( , )( )

y

Y y

I y xf y dx

y

1 2 1 2

2 2 2 2, 2 2 , 2 2 22 2

2

{ ( , ) ( , )}y

X X X Xy

yf y x x f y x x dx

y x

HW: Solve Problem 5.4-3.2 21 1 2 2

2 2

1 2

2

2 (1 ), 1 2 2 2

1( , )

2 1X

x x x x

X X

X

f x x e

2 21 2Y X X

( ) ?Yf y

0

Chapter 5. Operations on Multiple R. V.'s22

5.4 Transformations of Multiple Random Variables

Multiple functions 1 1 1 21 2

2 2 1 2

( , )( , )

( , )

Y T X XT X X

Y T X X

1 2, 1 2( , )X Xf x x

1 2 1 2, 1 2 , 1 2( , ) ( , )Y Y X Xf y y f x x J

11 1 1 21

1 2 12 2 1 2

( , )( , )

( , )

x T y yT y y

x T y y

1 11 1

1 2

1 12 2

1 2

T T

y yJ

T T

y y

jacobian

Chapter 5. Operations on Multiple R. V.'s23

5.4 Transformations of Multiple Random Variables

1 1 1 2 1 2 11 2

2 2 1 2 1 2 2

( , )( , )

( , )

Y T X X aX bX Xa bT X X

Y T X X cX dX c d X

Ex 5.4-3:

1 2

1 2

1 2 1 2,

, 1 2

( , )( , )

X X

Y Y

dy by cy ayf

ad bc ad bcf y yad bc

11 1 1 2 11

1 2 12 22 1 2

( , ) 1( , )

( , )

x T y y yd bT y y

x c a yad bcT y y

1 11 1

1 2

1 12 2

1 2

1

T T

y yJ

ad bcT T

y y

Chapter 5. Operations on Multiple R. V.'s24

5.5 Linear Transformations of Gaussian Random Variables

1 1 11 12

21 222 2

Y X a aY A AX X

a aY X

[ ] [ ] [ ]E Y E AX AE X

[( )( ) ] [ ( )( ) ]T T TYC E Y Y Y Y E A X X X X A

[( )( ) ]T T TXAE X X X X A AC A 1 1 1T

Y XC A C A

1

1 2

1( ) ( )

2, 1 2 1/ 2/ 2

1( , )

(2 )

TXx X C x X

X X NX

f x x eC

1 2, 1 2( , ) ?Y Yf y y

2det( ) det( ) det( )Y XC A C

Chapter 5. Operations on Multiple R. V.'s25

5.5 Linear Transformations of Gaussian Random Variables

1 1 11( ) ( )

21/ 2/ 2

1

(2 ) det( )

TXA y X C A y X

NX

eC A

1 2 1 2

1, 1 2 ,

1( , ) ( )

det( )Y Y X Xf y y f A yA

1 1 1 1 1( ) ( ) ( ) ( )T T TX XA y X C A y X y Y A C A y Y

1( ) ( )TYy Y C y Y

1

1 2

1( ) ( )

2, 1 2 1/ 2/ 2

1( , )

(2 )

TYy Y C y Y

Y Y NY

f y y eC

1/ 2 1/ 2det( )X YC A C

(gaussian)

Chapter 5. Operations on Multiple R. V.'s26

5.5 Linear Transformations of Gaussian Random Variables

1

1 2

1( ) ( )

2, 1 2 1/ 2

1( , )

(2 )

TYy Y C y Y

Y Y

Y

f y y eC

Ex 5.5-1: 1 1 1

2 2 2

1 2

3 4

Y X XA

Y X X

1

2

[ ] 0

[ ] 0

E X

E X

4 3

3 9XC

1 1

2 2

[ ] [ ] 0

[ ] [ ] 0

E Y E XA

E Y E X

1 2 4 3 1 3 28 66

3 4 3 9 2 4 66 252T

Y XC AC A

1 2

1 2

660.786

28 252

YY

Y Y

C

Chapter 5. Operations on Multiple R. V.'s27

5.6 Computer Generation of Multiple Random Variables

"OMITTED"

Chapter 5. Operations on Multiple R. V.'s28

5.7 Sampling and Some Limit Theorems

sampling and estimation

samples estimate

1

1 N

N nn

x xN

-- average of samplesN

1

1ˆN

N nn

x xN

-- estimate of average of samplesN

estimator of mean of r.v. X

1

1ˆN

N nn

X XN

(sample mean)r.v.

Chapter 5. Operations on Multiple R. V.'s29

5.7 Sampling and Some Limit Theorems

estimator of power of r.v. X

2 2

1

1 N

N nn

X XN

estimator of variance of r.v. X

2 2

1

1 ˆ( )1

N

X n Nn

X XN

1 1

1 1ˆ[ ] [ ] [ ]N N

N n nn n

E X E X E X XN N

(sample mean estimator is unbiased.)

22

1 1 1 1

1 1 1ˆ[ ] [ ] [ ]N N N N

N n n n mn n n m

E X E X X E X XN N N

2[ ] [ ]n mn m E X X E X 2[ ]n mn m E X X X

indep.2 2 2 2 2

2

1 1{ [ ] ( ) } { ( 1) }NE X N N X X N X

N N

Chapter 5. Operations on Multiple R. V.'s30

5.7 Sampling and Some Limit Theorems

2 2 2 2 2 21 1 1{ [ ] ( 1) } { [ ] } XE X N X X E X X

N N N

2

2ˆ

2 2ˆ 1 1NX XNP X X

N

(Chebychev's ineq.)

1N

ˆN NX X w.p. 1 (with probability 1)

Ex 5.7-1: 0.05X 50N

ˆ 0.95NP X X 2

21 0.95 160

50(0.05 )X

XXX

2 2 2 2 2 2ˆ

ˆ ˆ ˆ ˆ[( ) ] [ 2 ] [ ]N

N N N NXE X X E X XX X E X X

Chapter 5. Operations on Multiple R. V.'s31

5.7 Sampling and Some Limit Theorems

2 2

1

1 N

N nn

X XN

2 2

1

1 ˆ( )1

N

X n Nn

X XN

2 2 2 2 2

1 1

1 1[ ] [ ] [ ] [ ]

N N

N nn n

E X E X E X E X XN N

(unbiased)

2 2 2 2

1 1

1 1ˆ ˆ ˆ[ ] [( ) ] [ 2 ]1 1

N N

X n N n n N Nn n

E E X X E X X X XN N

2 2

1 1 1 1 1

1 1ˆ[ ] [ ] [ ] ( 1)N N N N N

n N n m n mn n m n m

E X X E X X E X X X N XN N

2 2 2 2 22 2 2 22{ ( 1) } { ( 1) }

[ ]1X X

N X X N X X N XE X X

N

(unbiased)

Chapter 5. Operations on Multiple R. V.'s32

5.7 Sampling and Some Limit Theorems

2 2 2 2

1

1 1ˆ( ) [(0.1 3.973) (12.0 3.973) ]1 10

N

X n Nn

X XN

1

1 1ˆ (0.1 0.4 0.9 12.0) 3.97311

N

N nn

X X VN

Ex 5.7-2:

11 samples (0.1, 0.4, 0.9,1.4, 2.0, 2.8, 3.7, 4.8, 6.4, 9.2,12.0 )V

sample mean

sample variance

214.75V

[ ] 4E X V 2 16X

Chapter 5. Operations on Multiple R. V.'s33

5.7 Sampling and Some Limit Theorems

ˆlim [ [ ] ] 1, 0NN

P X E X

Ex 5.7-3:

Weak Law of Large Numbers

Strong Law of Large Numbers

ˆ{lim [ ]} 1NN

P X E X

4

2 2 3

4 ( )( )

( )Y

a yu yf y

a y

[ ]

4

aE Y

2 22 (16 )

16Y

a

Chapter 5. Operations on Multiple R. V.'s34

5.8 Complex Random Variables

Z X jY

[ ] [ ] [ ] [ ]Z E Z E X jY E X jE Y X jY 22 *[( ) ( )] [ ]Z E Z Z Z Z E Z Z

mean

variance

1 1 1Z X jY 2 2 2Z X jY

1 1 2 2 1 1 2 21 2 , , , 1 1 2 2 , 1 1 , 2 2 & indep. ( , , , ) ( , ) ( , )X Y X Y X Y X YZ Z f x y x y f x y f x y

1 2

*, 1 2[ ]Z ZR E Z Z

1 2 1 2

* *, 1 1 2 2 , 1 2[( ) ( )]Z Z Z ZC E Z Z Z Z R Z Z

1 2 1 2

*1 2 1 2 & uncorr. C 0 RZ Z Z ZZ Z Z Z

1 21 2 & orthogonal R 0Z ZZ Z

[ ] ( ) ( , )XYE X jY x jy f x y dxdy