f*cxl{Y=y3) - Purdue University College of Engineeringmrb/resources/LecturesF/Session_2… · 419 Session 24 ~:Joint GaussianExamples 420 Recall that we have seen that f× (x/{Y=y3)

4 1 9

S e s s i o n 2 4

J o i n t G a u s s i a n Examples 4 2 0~ :

Recal l t h a t w e h a v e s e e n t h a t

f × ( x / { Y= y 3 ) = f x . u a #f i e r y )

a n d w e a l so know t h a t

a s

f.ua/1=ff*.wcx,yIdx.- a s

a n d£ , ex ,y ) = f l y I { * = t 3 ) -5*1×1

⇒ f*lxl{Y=y3) =f.ua/lEX=.t3)f*c#§fµcyl{*= D3 )faco ldd

(Bayes Theorem)

Also r e c a l l t h a t w e investigated 4 2 1

t w o es t ima to r s f o r estimating t h e

v a l u e o f X given t h a t w e observe

{ Y= y 3 :Immsly) = E [ X / {Y=y3]

andImaply) = argmgr f*cxl{Y=y3)

B o t h o f t h e s e require t h a t w e f i n d

f×cxlEY=y3).

Example 4 2 2-

L e t X a n d 1N be. t w o j -d ist , independentG a u s s i a n R V s w i t h v a r i a n c e s § and

8 2 , respectively. N o w c o n s i d e r

a n e w R V Y :

X - O t - Y = X t N

Isuppose I observe Y- y ? Wh a t

i s t h e M i n imum Mean-square e r r o r

e s t i m a t o r o f X given {Y=y3?

Emms (Y) = E [ * / {Yay} ]4 2 3

W e n e e d t o f i n d f*lxIEY=y3)= 5#µC×¥

f i r l y )w e n e e d t o f i n d t h e joint Gauss ianp. d . f . f * , i x . y ) .

- Easy t o s h o w t h a t 0,2=0×2+0,5.A lso, r × , = E l i x i r ] - ftp.oec.ws exercise

# = . . . = §- 0 × 0 -

E [ * Y ] = E [ * (H tm l ]=E [×2# IN ]=E [×2 ]=q '94%2

A l so, i t c a n b e shown (exercise) 4 2 4

t h a t

f cxl{My 3) = fa i r any)* -

(exercise)f i r l y )

= . . .=⇐÷.it#eHEEiEi.?iI.:Immscy1=EfXlEY=y5f=rxy#y

(exercise)= o a o =

0×2¥ 8 5 Y

Gampter: Assume X a n d H a v e 4 2 5

j - d i s t G a u s s i a n w i t h p-d.f

fam".'ll-⇐¥iEe×p{z¥l¥2¥¥t¥]}F ind Immsey).=E[* l i l y ] a n d Imapey)

.

( i ) f i n d f*lxlEY=y3).( I i ) F i n d E[XlEY=y3] ( read t h e m e a n off.)

= r a y←answers#cnet.my#z=expfEEIEI-g}

Exam3off

i f

ConvergenceofsequeucesofRt 4 2 6

µ , ,# z , . . . ,

# n , o . .

E t i suppose w e m a k e - a sequence o f

measurements

X i , = a t 1WK ,K = 1 , 2 , . . .

a = parameter o f i n te re s t

* k=

k - t h measurement

W e =experimental e r r o r o f k-themeasurement.

Assume E [ w e ] - o(unbiasedmeasurements)

I f w e cons i d e r t h e R V s 4 2 7

*i s . . . ,

*n , . . . t o b e measurements,

w e typically es t ima te t h e parametera by

% = I [X , t A z t . . - t # n ]

(sample average o f A , , . . . , An)

I s t h i s a good estimate o f a ?n o -Hopefully Y, → o r

I s t h i s t r u e ? I s i t always t r u e ?

I s i t e v e r t r u e ? w h a t d o w e mean?

N o t e t h a t 4 2 8

i n - I n [A, t . . - t # n ]i s i t s e l f a r a ndom variable, s o

Yi, % , . . . , Yin, . . .i s a l s o a r a n d om sequence.

§ ! w h a t d o e s i t m e a n t o a s k

i fY , → a a s n → c o ?

Defy: A random sequence o r a 4 2 9#

discrete-timestochasticprocessisa sequence o f r a n d o m var iables

* i s . . . ,Am , . . . d e f i n e d o n CS,F , P).

( I f X i i s a R - V. f o r each

i c - I N , t h e n measurabilityi s

n o t a n issue.)

• We o f t e n w r i t e t h i s a s

{ X n 3 o r § X n 3 n e w o r { * n3n=,• F o r a n y specific w o e s

o f C S , F , P )

X . ( w o ) , . . . , # n Cwo ) , . . . i s a sequence o fr e a l n umb e r s .

Defy: A sequence o f rea l numbers 4 3 0

X , , . . . , X n , . . . i s s a i d t o conveyt o a l i m i t × i f , f o r E > o ,t h e r e e x i s t s a n u m b e r n e e 1 N

s u c h t h a t

I X , - X I < E, A n t h e .

--

" X n→ x a s n > a s "

G i ve n a r a n d o m sequence 4 3 1

X , c o ) ,N z ( o ) , . . .

,# n l o )

, . . .

f o r any particular w e b, w e

h a v e

IX, Iw ) , Nz (w), . . . ,#n'w) , . . .

i s a sequence o f r e a l n u m b e r s .

- I t may converge t o a number * (w).

- o r i t mayn o t converge.

n.be N e w ) T h a t t h e r a ndom sequence

converges t o i s i t s e l f a f t n .

o f w ( K e w ) i s a R i v. )

G i v e n a random sequence4 3 2

H, I w ) ,. . .

,A n 1W) , . . .

m o s t likely i t w i l l converge f o r

s o m e w e b,a n d n o t converge f o r

o t h e r w e b .

When w e study convergence o f

random sequences (stochasticconvergence)

w e study t h e s e t A c t f o r which

*,(W ) , . . . ,

Xu lw l , . . .i s a convergent sequence f o r a l l

w e A .

Defy: W e say t h a ta sequence o f 4 3 3

R V s converges everywhere-

( e )

i f t h e sequences

X , Iw ) , Hzlw),..., #n'w),.. .e a c h converge t o a number

* c u ) f o r e ach w e b .

n d • T h e numbe r Kew) t h a t { K n w }

Converges t o i s a function o f W ,

• Convergence everywhere i s t o o

strong (restrictive) t ob e useful .

Defy: A random sequence { * n } 4 3 4

converges a lmos t everywhere#

care.)

i f t h e s e t o f outcomes A C S

s u c h t h a t # n e w ) I f # I w ) , W E Ae x i s t s and h a s probability 1 :

P I A ) = I

O t h e r n a m e s f o r convergence ( a . e . I a r e

- almost s u r e convergence ( a . s . )

- Convergence w i t hprobability o n e .

W e w r i t e t h i s a s

" X , I * "

" P l { * n - * 3 ) = L . "

Deff: w e s a y t h a t a r a n d om 4 3 5

sequence{[email protected].) t o a R V *

i fE [ I X , -XI?]→o a s n o w .

n I convergence ( m . s . ) i s a lso c a l l e d

" L im i t i n t h e y e a r convergence"a n d

i s w r i t t e n"

t . im . X , → X " ( B a d )

" X , ↳ X " (Better)

Defy: T h e random sequence { A n } 4 3 6

converges i n probability t ot h e R V * i f , H e > 0

P l a i n - * I > e 3 ) → 0

a s n - c o .