This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transition ProbabilitiesTransition Probabilities Steady-State ProbabilitiesSteady-State Probabilities Absorbing StatesAbsorbing States Transition Matrix with Transition Matrix with
SubmatricesSubmatrices Fundamental MatrixFundamental Matrix
Markov process modelsMarkov process models are useful in are useful in studying the evolution of systems over studying the evolution of systems over repeated trials or sequential time periods repeated trials or sequential time periods or stages.or stages.
They have been used to describe the They have been used to describe the probability that:probability that:
•a machine that is functioning in one a machine that is functioning in one period will continue to function or break period will continue to function or break down in the next period.down in the next period.
•A consumer purchasing brand A in one A consumer purchasing brand A in one period will purchase brand B in the next period will purchase brand B in the next period.period.
Transition probabilitiesTransition probabilities govern the govern the manner in which the state of the manner in which the state of the system changes from one stage to system changes from one stage to the next. These are often the next. These are often represented in a represented in a transition matrixtransition matrix. .
A system has a A system has a finite Markov chainfinite Markov chain with with stationary transition probabilitiesstationary transition probabilities if: if:
•there are a finite number of states,there are a finite number of states,
•the transition probabilities remain the transition probabilities remain constant from stage to stage, andconstant from stage to stage, and
•the probability of the process being in the probability of the process being in a particular state at stage a particular state at stage n+n+1 is 1 is completely determined by the state completely determined by the state of the process at stage of the process at stage nn (and not the (and not the state at stage state at stage n-n-1). This is referred to 1). This is referred to as the as the memory-less propertymemory-less property..
The The state probabilitiesstate probabilities at any stage at any stage of the process can be recursively of the process can be recursively calculated by multiplying the initial calculated by multiplying the initial state probabilities by the state of state probabilities by the state of the process at stage the process at stage nn..
The probability of the system being The probability of the system being in a particular state after a large in a particular state after a large number of stages is called a number of stages is called a steady-state probabilitysteady-state probability. .
Steady state probabilitiesSteady state probabilities can be can be found by solving the system of found by solving the system of equations equations PP = = together with together with the condition for probabilities that the condition for probabilities that ii = 1. = 1.
•Matrix Matrix PP is the transition is the transition probability matrixprobability matrix
•Vector Vector is the vector of steady is the vector of steady state probabilities.state probabilities.
An An absorbing stateabsorbing state is one in which is one in which the probability that the process the probability that the process remains in that state once it enters remains in that state once it enters the state is 1.the state is 1.
If there is more than one absorbing If there is more than one absorbing state, then a steady-state state, then a steady-state condition independent of initial condition independent of initial state conditions does not exist.state conditions does not exist.
Transition Matrix with SubmatricesTransition Matrix with Submatrices
If a Markov chain has both absorbing and If a Markov chain has both absorbing and nonabsorbing states, the states may be nonabsorbing states, the states may be rearranged so that the transition matrix can be rearranged so that the transition matrix can be written as the following composition of four written as the following composition of four submatrices: submatrices: II,, 0 0, , RR, and , and QQ: :
Transition Matrix with SubmatricesTransition Matrix with Submatrices
II = an identity matrix indicating one always = an identity matrix indicating one always remains in an absorbing state once it is remains in an absorbing state once it is
reachedreached
00 = a zero matrix representing 0 probability of = a zero matrix representing 0 probability of
transitioning from the absorbing states to transitioning from the absorbing states to the the
nonabsorbing statesnonabsorbing states
RR = the transition probabilities from the = the transition probabilities from the nonabsorbing states to the nonabsorbing states to the
absorbing statesabsorbing states
QQ = the transition probabilities between the = the transition probabilities between the nonabsorbing states nonabsorbing states
The The fundamental matrixfundamental matrix, , NN, is the inverse of , is the inverse of the difference between the identity matrix the difference between the identity matrix and the and the QQ matrix. matrix.
The The NRNR matrix matrix is the product of the is the product of the fundamental (fundamental (NN) matrix and the ) matrix and the R R matrix. matrix.
It gives the probabilities of eventually moving It gives the probabilities of eventually moving from each nonabsorbing state to each from each nonabsorbing state to each absorbing state. absorbing state.
Multiplying any vector of initial nonabsorbing Multiplying any vector of initial nonabsorbing state probabilities by state probabilities by NRNR gives the vector of gives the vector of probabilities for the process eventually probabilities for the process eventually reaching each of the absorbing states. Such reaching each of the absorbing states. Such computations enable economic analyses of computations enable economic analyses of systems and policies.systems and policies.
How many times per year can Henry How many times per year can Henry expect to talk to Shirley?expect to talk to Shirley?
AnswerAnswer
To find the expected number of accepted To find the expected number of accepted calls per year, find the long-run proportion calls per year, find the long-run proportion (probability) of a call being accepted and (probability) of a call being accepted and multiply it by 52 weeks.multiply it by 52 weeks.
Solving using equations (2) and (3). Solving using equations (2) and (3). (Equation 1 is redundant.) Substitute (Equation 1 is redundant.) Substitute = 1 - = 1 - into (2) to give:into (2) to give:
.65(1 - .65(1 - 22) + ) + = = 22
This gives This gives = .76471. Substituting back = .76471. Substituting back into equation (3) gives into equation (3) gives = .23529. = .23529.
Thus the expected number of accepted Thus the expected number of accepted calls per year is:calls per year is:
(.76471)(52) = 39.76 or about 40(.76471)(52) = 39.76 or about 40
What is the probability Shirley will What is the probability Shirley will accept Henry's next two calls if she does not accept Henry's next two calls if she does not accept his call this week?accept his call this week?
What is the probability of Shirley What is the probability of Shirley accepting exactly one of Henry's next two calls accepting exactly one of Henry's next two calls if she accepts his call if she accepts his call this week?this week?
The probability of exactly one of the next The probability of exactly one of the next two calls being accepted if this week's call is two calls being accepted if this week's call is accepted can be found by adding the accepted can be found by adding the probabilities of (accept next week and refuse probabilities of (accept next week and refuse the following week) and (refuse next week and the following week) and (refuse next week and accept the following week) = accept the following week) =
The probabilities of eventually moving to The probabilities of eventually moving to the absorbing states from the nonabsorbing the absorbing states from the nonabsorbing states are given by:states are given by:
What is the probability of someone who What is the probability of someone who was just was just promoted eventually retiring? . . . promoted eventually retiring? . . . quitting? . . . quitting? . . . being fired? being fired?
Absorbing States (continued)Absorbing States (continued)
AnswerAnswer
The answers are given by the bottom The answers are given by the bottom row of the row of the NRNR matrix. The answers are matrix. The answers are therefore:therefore:
Eventually Retiring = .12Eventually Retiring = .12
Eventually Quitting = .64Eventually Quitting = .64
Eventually Being Fired = .24Eventually Being Fired = .24