Markov Analysis

MARKOV ANALYSISMARKOV ANALYSISSession 11

IntroductionIntroduction

The term Markov Analysis refers to a quantitative technique that involves the analysis of the current behavior of some variable in order to predict the future behavior of that same variable.

Markov analysis is applicable to systems in which we can determine probability of movement from one state to another over time.

A Markov process is a special type of stochastic process in which the current state of a system depends only upon the immediately preceding state of the system.

An example of the use of Markov Analysis is the determination of the probability that a customer will change brands of substitute product from one month to the next month, also it has been used to describe the probability that a machine is functioning in one period will continue to function in the next period.

An Illustration : “The Accounts An Illustration : “The Accounts Receivable Control Problem”Receivable Control Problem”

A wholesale distributor is attempting to determine a more effective set of credit-control policies. The company would classify all of its accounts receivable into several categories.

First, at any point in time in which an accounts receivable balance is paid in full, it is immediately placed in full category, and second if the payment exceeds 90 days, that portion is placed in the defaulted category.

Third, the payment category according to the oldest unpaid bill for each customer. For example, the total accounts receivable balance is $6000. A payment on November 30, then it would be classified as the delinquent (31-90 days) category since the delivery placed on October 10 (41 days old). And on December 5, the customer paid the October 10 bill of $1000. The remaining total balance of $5000, corresponding to the November 20 purchase, is classified as the delinquent (0-30 days) category since it less than 31 days old.

……An Illustration : “The Accounts …An Illustration : “The Accounts …

Use of this method of accounts receivable aging means that an accounts receivable classified in the delinquent (31-90 days) category at one point in time may appear in the delinquent (0-30 days) category at a later point.

The various accounts receivable categories are the states of the system and describe the status of the system at a point in time.

Markov analysis assumes that those states are collectively exhaustive (list all the possible states of the system) and mutually exclusive (a property that the system can only be in one state at any point in time).

In this discussion, it is assumed that Markov analysis has a finite number of states for the system being analyzed.

Once the states have been identified, the next step in Markov analysis process is to determine the probabilities associated with the system being in a particular state.


For this case, assume that there was $100,000 accounts receivable position and it is distributed (probability) as follows.State 1 – Paid in full $45,000/$100,000 = .45State 2 – Defaulted $15,000/$100,000 = .15State 3 – Delinquent (0-30 days) $25,000/$100,000 = .25State 4 – Delinquent (31-90 days) $15,000/$100,000 = .15

Written in V(1) = ( .45 .15 .25 .15 )General form V(i) = (v1, v2, v3, …,vn)

V sounds to the vector of states probabilities.

......The next step is to specify the matrix of transition

probabilities. A transition probability (pij) is defined as conditional probability of the system being in state j one period, or step, in the future, given that system is in state i currently.

S1 S2 S3 S4

S1 1 0 0 0 P = S2 0 1 0 0

S3 .5 .2 .1 .2S4 .4 .4 .1 .1

P11 P12 P13 P14

= P21 P22 P23 P24

P31 P32 P33 P34

P41 P42 P43 P44


Property 1 : the transition probabilities for a given beginning state of the system must sum to 1.0

Property 2 : the transition probabilities encompass all possibilities for the system.

Property 3 : the transition probabilities are constant over time.

Property 4 : the states of the transition probability matrix are independent over time.

......Now, if there is any period n, the state

probabilities for period n+1 can be computed as follows

V(n+1) = V(n) . P.1 .1 .4 .2 .1 .2 0 0 1 0 0 0

4.5.01

4

3

2

1

SSSS

P


For the case…V(2) = V(1) . PV(2) = (.45 .15 .25 .15)

= [ (.45)(1) + (.15)(0) + (.25)(.5) + (.15)(.4) ]= [ (.45)(0) + (.15)(1) + (.25)(.2) + (.15)(.4) ] = [ (.45)(0) + (.15)(0) + (.25)(.1) + (.15)(.1) ] = [ (.45)(0) + (.15)(0) + (.25)(.2) + (.15)(.1) ] = [ .635 .26 .04 .065]

S1 S2 S3 S4

Paid in full Defaulted Delinquent Delinquent (0-30 days) (31-90 days)

1 0 0 0

0 1 0 0

.5 .2 .1 .2

.4 .4 .1 .1

Computing Steady State Probabilities Computing Steady State Probabilities

Continuing the computation process of the state probabilities: Period 3 =V(3) = V(2) . P

V(3) = (.681 .26 .04 .065)

V(3) = [ .681 .26 .04 .065 ] Period 4 =V(4) = V(3) . P V(4) = [ .6921 .3019 .0025 .0036 ] Period 5 = V(5) = V(4) . P V(5) = [ .6947 .3038 .0006 .0009 ] Period 6 = V(6) = V(5) . P V(6) = [ .6954 .3043 .0001 .0002 ]

1 0 0 00 1 0 0.5 .2 .1 .2.4 .4 .1 .1

Computing Steady State Probabilities Computing Steady State Probabilities

Period 7 =V(7) = V(6) . P V(7) = [ .69552 .30439 .00004 .000051 ] Period 8 = V(8) = V(7) . P V(8) = [ .6956 .3044 .0000 .0000 ] Note that as it proceeded from period 2 to period

8, the changes in the state probabilities become smaller and smaller. The virtually constant state probabilities that occur after a certain number of periods are referred to as steady-state probabilities.

Therefore, for accounts receivable problem, the steady state are the same transition probabilities at period 8, means that each probability being in its category after a number of periods in the future is independent toward its disposition in period 1.

•Computing Steady State Probabilities Computing Steady State Probabilities

Category Of Accounts

Receivable

Steady-State Probab

ility

Number of Accounts

Receivable Dollar Amount of

Accounts Receivable

Paid in fullDefaultedDelinquent, 0-30 daysDelinquent, 31-90 days

.6956

.3044

.0000

.0000

1000 x .6956 = 695.61000 x .3044 = 304.41000 x .0000 = 01000 x .0000 = 0

$1,000,000 x .6956 = $695,600$1,000,000 x .3044 = $304.400$1,000,000 x .0000 = $0$1,000,000 x .6956 = $0

• For other variation state probabilities

State Period 1

Period 2

Period 3

Period 4

Period 5

Period 6(steady-state prob.)

S1

S2

S3

.3150

.3000

.3850

.3930

.3000

.3070

.4086

.3000

.2914

.4117

.3000

.2883

.4123

.3000

.2877

.4125

.3000

.2875

S1 S2 S3

S1 .5 .3 .2

P =

S2 .4 .3 .3

S3 .3 .3 .4

Beginning Condition V(1) = (.05 .05 .90)

AnAn Algebraic approach to compute steady-state Algebraic approach to compute steady-state probabilitiesprobabilities

If the steady-state probabilities is designated as 1, 2 , and 3 , for both period n and period n+1:

V(n) = V(n+1) = [1 2 3 ] Consequently, for the previous example, the computation given

by [1 2 3 ]

The results is 1 = .51 + .42 + .33

2 = .31 + .32 + .33

3 = .21 + .32 + .43

Since the steady-state probabilities must sum to 1.0, then1 = 1 + 2 + 3

.5 .3 .2

.4 .3 .3

.3 .3 .4


The above equations can be rewritten as follows:

Arbitrarily it can be solved that:Step 1 equate eq. 1 and eq. 2 -.51 + .42 + .33 = .31 - .72 + .33

-.51 + .42 = .31 - .72

- 81 = -1.1 2

1 = 1.1/.8 2

Step 2 add 3 times eq.1 to 5 times eq. 2

0 = -1.51 + 1.22 + .93

0 = 1.51 – 3.52 + 1.53

0 = - 2.3 2 + 2.43 3 = 2.3/2.4 2

0 = -.51 + .42 + .33 eq. 1 0 = .31 - .72 + .33 　　　　 eq. 2 0 = .21 + .32 - .63 eq. 3 1 = 1 + 2 + 3 eq. 4


Step 3 solve for 1 2 3 1 = 1 + 2 + 3

1 = 1.1/.8 2 + 2 + 2.4/2.4 2

1 = 3.333 2 2 = 1/3.333 = .3000 1 = 1.1/.8 2 = .4125 3 = 1 – (.3000 + .4125) = .2875

It is necessary to emphasize that this method for solving the steady-state probabilities will work only for non-absorbing Markov processes.

Analysis of Absorbing Markov Processes Analysis of Absorbing Markov Processes

A special case of a Markov process involves an absorbing or trapping state.

An absorbing state is a state that has a zero probability of being left once it is entered. Once the absorbing state is entered, the process either stops completely or stops then is reinitiated from some other state.

A Markov process can be shown to be an absorbing Markov process if

1. it has at least one absorbing state. 2. It is possible to move from every non-absorbing state in a

finite number of steps. For the case accounts receivable, it has two absorbing states

1. S1 : paid in full2. S1 : defaulted

The probability of being in both categories above, in a future time period given for the current period is 1.0 or 100 percent.


Observe also that both of other states, S3 and S4, are non-absorbing but it is possible to move from any of these non-absorbing states to the absorbing states, S1 and S2. Or as analyzed, after period 6, the steady-state probabilities of both states had been reduced to zero.

In particular, the probability of absorption by any absorbing state; and the expected number of steps before the process is absorbed can be determined.

To begin this process, initially the matrix of transition possibilities is transform into the following general form.

where I = an r-by-r identity matrix defining the probabilities of staying within an absorbing state once it reachedO = an r-by-s null matrix indicating the probabilities of going from an absorbing state to a non-absorbing state.

P = I O

A N


where A = an s-by-r matrix containing the probabilities of going from a non-absorbing state to an absorbing state.N = an s-by-s matrix showing the probabilities of going from one non-absorbing state to another non-absorbing state.

For accounts receivable case :

The fundamental matrix F, can be calculated asF = (1 – N)-1

P = I O

A N

P =

1 00 1

0 00 0

.5 .2

.4 .4.1 .2.1 .1


First, determining (1 - N)

Next, the above matrix is inverted to obtain F = (1-N)-1

To interpret the results given by the fundamental matrix (F) matrix, recall that the non-absorbing states are 3 and 4. The expected number of steps before absorption is the sum of the times the process in each absorbing state.

1 0-

.1 .2

0 1 .1 .1

=.9 -.2

=9/10 -2/10

-.1 .9 -1/10 9/10

=90/79 20/79

=1.14 .25

10/79 90/79 .13 1.14


Beginning State Expected steps before absorptionS3

S4

90/79 + 20/79 = 110/79 =1.3910/79 + 90/79 = 100/79 = .27

=1.14 .25 .5 .2

=.67 .33

.13 1.14 .4 .4 .52 .48

• To compute the probability of absorption = FA = (1-N)-1 . A

The interpretation is that (first row) there is .67 probability that a delinquent account in state 3 will end up in state 1 (paid in full), and .33 probability a delinquent account beginning in state 3 will end up in state 2 (bad debt).The second row indicates that a delinquent account in state 4 will end up in state 1 at .52 probability and state 2 at .48 probability.


Finally, assume that the company has found that it has $25,000 accounts receivable in state 3 (delinquent,0-30 days) and $15,000 in state 4 (delinquent, 31-90 days), the absorption probability matrix that how many of these dollars will eventually end up in state 1 or 2 is determined (denoted by T) by

T` = T . Probability of absorption

Therefore, if this company currently has $25,000 (state S3) + $15,000 (state S4) = $40,000 of delinquent accounts receivable, it can expect that eventually $24,450 will be collected in, while $15,450 will become bad debts.

T = S3 S4

($25,000) ($15,000)S1 S2

= S3 S4 S3 .67 .33

($25,000) ($15,000) S4 .52 .48

= S3 S4

($24,550) ($15,450)

Markov Analysis

Documents

days category

accounts receivable

accounts receivable

markov analysis process

use of markov analysis

markov process

current state

days old