Today: First step analysis Next: PK 3ynemish/180b/180b... · Firststepanalysis-Thesimplest-settings.rsstates Suppose we have Markov chain on {ostie} with transition probabilitz matrix

MATH180B: Introduction to Stochastic Processes I

www.math.ucsd.edu/~ynemish/180b

This week:

HW3 due Friday, January 32, 23:59 pm

Regrades for HW2: Tuesday, Feb 4, 8am-11pm

Today: First step analysisNext: PK 3.5

→NO Home Wark

Firststepanalysis-Thesimplest-settings.rs states

Suppose we have Markov chain on { ostie } with

transition probabilitz matrix

Rap -1¥ Lins

* !ËÊ " l' E '

traçâto o I Start at X. = I

If the process baves 1, it is trappes in

Questions : where ( in which state ) will the process be trappe%?

Hon long will Au process story at I Can average)?

Both questions lead to simple linear équations with

coefficients x. pa z .

FirstslepanalysisfirstsolutionslJenote :

F- min {nzo : Xn⇒ or Xn = 23 - absorption time

a =P( Xt -- o IX. = , ) (probability to end up at o

starting from I )what trappeurs at n=t ? Condition on possible outâmesP (Xt = 01×0--1 ,X ,

= a) =L P (Xt = 0 IX. =L ,X , =2) = O

P ( Xt = 01 X. =L ,X,= 1) = le =P (XT = 01×0=1 )

u =P (Xt --01×0--1 ) = Ê P ( Xt⇒ IX. =L , Xi - i ) - P(X , -- il X.=/ )

= P(Xt⇒ 1×0=1 ,X ,- o) - PCX ,⇒ 1×0=1) t - - -

= t.RO t U . Pi , + 0 . Piz = l - L + u.pt O -j

⇒ a- a + up ⇒a = # = ¥

ExampleL

¥3 h ,r bi

o o'¥

.

"

Questions :

(a) Starting from O,what is the probability of hitting 6 ? Yy

O L R

P= ! ( È È È )"=P ( hitting 6) = Plgettiug trapped in r)

"ce = x + pu = f- + tu -- É-tH+Éu#

ÊIÈËÊ ) a-- Es -- t -¥4

First step unalysis .

First solutions

Dénote1-= min {neo : Xn⇒ or Xn = 23 - absorption time Casa bove)] = E (T IX. = I ) (time (expected ) speut af 1)

Try to get an équation for V :

✓ = E (T IX. = 1) = ÊECTIX - t,Xii ) P (X , -- il X. = 1)

= ÊECT IX.⇒ , X. = i ) . pi JEAN.⇒ Mio)- t

ECTIXOY ,XFD⇒= t - x + ( Itv) -

p + t -t = tt PV ECTIX.MX#)--tt✓ = #

Double check : p (T=k IX. = 1) = pk! ( t - p) for K- ci? --

ECTIX .- 1) = Être p" f- p ) = ¥

Example ( t more than one step )-

R213 5

"

20 8 03 • D qO

it :o o'è" R

Questions :O

F- min { n : Xn = l or Xn = a }E (Tt X.→ ) =Ë = É(c) Starting from I , howmany steps (on average) to hit 3 ? 3

Le t F- min { n : Xu =3 } ,start from I f- ECT IX. = 1)

f- ECT IX. =L , Xi - 2) = Pas " ⇒⑤+ECT IX.H , X , -2,4=1) . R, = §. Et ( ztp) fr-2 xp -t

firststepanalysis.Nstates-XDnz.isa Markov Chain on {ah 2 . . _ , N } with

transition pro babitity matrix ,first r states toi , _ _ , r- is

are transi ent , { r , - - en } are absor

bing.nl: :p:-.::::ui = Vik =P ( absorption ink / Xo - i )

N

= ZP ( absorption in k / X.= i. Xi -j ) - Pijf-oui

,

= t - Pik + oË Ë'

ai - Pis.