Flows and Networks Plan for today (lecture 2): • Questions? • Continuous time Markov chain • Birth-death process • Example: pure birth process • Example: pure death process • Simple queue • General birth-death process: equilibrium • Reversibility, stationarity • Truncation • Kolmogorov’s criteria • Summary / Next • Exercises
Flows and Networks Plan for today (lecture 2):. Questions? Continuous time Markov chain Birth-death process Example: pure birth process Example: pure death process Simple queue General birth-death process: equilibrium Reversibility, stationarity Truncation Kolmogorov’s criteria - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Flows and Networks
Plan for today (lecture 2):
• Questions?• Continuous time Markov chain• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
Reversibility; stationarity• Stationary process: A stochastic process is
stationary if for all t1,…,tn,
• Theorem: If the initial distribution is a stationary distribution, then the process is stationary
• Reversible process: A stochastic process is reversible if for all t1,…,tn,
NOTE: labelling of states only gives suggestion of one dimensional state space; this is not required
))(),...,(),((~))(),...,(),(( 2121 nn tXtXtXtXtXtX
1)(
jSj
))(),...,(),((~))(),...,(),(( 2121 nn tXtXtXtXtXtX
Reversibility; stationarity• Lemma: A reversible process is stationary.
• Theorem: A stationary Markov chain is reversible if and only if there exists a collection of positive numbers π(j), jS, summing to unity that satisfy the detailed balance equations
When there exists such a collection π(j), jS, it is the equilibrium distribution
• Proof
Skjjkqkkjqj ,),,()(),()(
Flows and Networks
Plan for today (lecture 2):
• Questions?• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process:
If the transition rates of a reversible Markov process with
state space S and equilibrium distribution are
altered by changing q(j,k) to cq(j,k) for
where c>0 then the resulting Markov process is
reversible in equilibrium and has equilibrium distribution
where B is the normalizing constant.
If c=0 then the reversible Markov process
is truncated to A and the resulting Markov
process is reversible with equilibrium distribution
Truncation of reversible processes
Sjj ),(
10
ASkAj \,
ASjjBc
AjjB
\)(
)(
Ajk
j
Ak
)(
)(
A
S\A
Time reversed processX(t) reversible Markov process X(-t) also, butLemma 1.11: tijdshomogeneity not inherited for
non-stationary process
Theorem 1.12 : If X(t) is a stationary Markov process with transition rates q(j,k), and equilibrium distribution π(j), jS, then the reversed processX(-t) is a stationary Markov process with transition rates
and the same equilibrium distribution
Theorem 1.13: Kelly’s lemmaLet X(t) be a stationary Markov processwith transition rates q(j,k). If we can find a collection of numbers q’(j,k) such that q’(j)=q(j), jS, and a collection of positive numbers (j), jS, summing to unity, such that
then q’(j,k) are the transition rates of the time-reversed process, and (j), jS, is the equilibrium distribution of both processes.
)(
),()(),('
j
jkqkkjq
Skj ,
),(')(),()( jkqkkjqj Skj ,
Flows and Networks
Plan for today (lecture 2):
• Questions?• Birth-death process• Example: pure birth process• Example: pure death process• Simple queue• General birth-death process: