Markov Processes Markov Processes System Change Over Time System Change Over Time Data Mining and Forecast Management Data Mining and Forecast Management MGMT E-5070 MGMT E-5070 ne from the television series “Time Tunnel” (1970s)
Jan 20, 2016
Markov ProcessesMarkov Processes
System Change Over TimeSystem Change Over Time
Data Mining and Forecast ManagementData Mining and Forecast ManagementMGMT E-5070MGMT E-5070
scene from the television series “Time Tunnel” (1970s)
Markov Process ModelsMarkov Process Models
Also known as Markov Chains.
Analyze how systems change over time.
Common applications include: consumer brand loyalty tendencies consumer brand-switching tendencies reliability analysis for equipment the aging / writeoff of accounts receivable the spoilage tendencies of perishable items over time
Andrei Andreyevich Markov( 1856 – 1922 )
Андрєй Aндрєєвич Марков
Ph.D , St. Petersburg University (1884)
Studied under Pafnuty Chebyshev.
Professor, St. Petersburg University (1886-1905) , but taught informally until 1922.
Early work in number theory, algebraic continual fractions, limits of integrals, and least squares method.
Launched the theory of stochastic processes ( Markov Chains ) : an all new branch of probability theory.
Markov Process ModelsMarkov Process Models
We will consider only We will consider only the simplest types the simplest types which are:which are:
discretediscrete finitefinite stationarystationary first-orderfirst-order
THE SCOPE OF STUDYTHE SCOPE OF STUDY
Markov Process ModelsMarkov Process Models
The states and transitionsThe states and transitions
are “discrete”are “discrete”
This means, for example,that market share among different stores can only change once per week or once per month, but not
second - to - second.
DISCRETE
Markov Process ModelsMarkov Process Models
The number of states is The number of states is “finite”“finite” Means, for example, that
an accounts receivablecan only age 3 months
before it is written off asa bad debt.
Once a bottle of winespoils, there are no
additional aging periods for it.
FINITE
Markov Process ModelsMarkov Process Models
Transitions dependTransitions dependonly on the only on the currentcurrentstate……….not on state……….not on
prior prior statesstatesThe chances of an
accounts receivablebeing paid off arehigher if it is one
month old as opposedto two months old.
STATIONARY
Markov Process ModelsMarkov Process Models
Transition probabilitiesTransition probabilitiesremain constant overremain constant over
timetimeFor example, thedefection rates ofcustomers from alocal supermarketto its competitorsmonth-by-monthwill always stay
the same.
FIRST-ORDER PROCESSES
Markov Process ModelsMarkov Process ModelsBASIC EXAMPLEBASIC EXAMPLE
TWO BARBERS IN A SMALL TOWN EACH HAVE A 50% MARKETSHARE. THEREFORE, THE VECTOR OF STATE PROBABILITIES
AT THIS TIME IS:
π ( 1 ) = ( 0.50.5 , 0.50.5 )
PERIOD NUMBER
ONE50% MARKET SHARE
FOR BARBER “AA”50% MARKET SHARE
FOR BARBER “BB”
Markov Process ModelsMarkov Process ModelsBASIC EXAMPLEBASIC EXAMPLE
P = P = [ ].90.90
.25.25
.10.10
.75.75
LOYALTYLOYALTYPROBABILITYPROBABILITYFOR BARBERFOR BARBER
““A” IN ANYA” IN ANYGIVEN PERIODGIVEN PERIOD
DEFECTIONDEFECTIONPROBABILITYPROBABILITYFOR BARBER FOR BARBER
““B” IN ANYB” IN ANYGIVEN PERIODGIVEN PERIOD
DEFECTIONDEFECTIONPROBABILITYPROBABILITYFOR BARBERFOR BARBER
““A” IN ANYA” IN ANYGIVEN PERIODGIVEN PERIOD
LOYALTYLOYALTYPROBABILITYPROBABILITYFOR BARBERFOR BARBER
““B” IN ANYB” IN ANYGIVEN PERIODGIVEN PERIOD
THE MATRIX OF TRANSITION PROBABILITIES PER MONTH ARE:
Markov Process ModelsMarkov Process ModelsBASIC EXAMPLEBASIC EXAMPLE
THE MARKET SHARE EACH BARBER HAS IN THE 2THE MARKET SHARE EACH BARBER HAS IN THE 2ndnd MONTH MONTHIS EQUAL TO THE PRODUCT OF THE IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATEVECTOR OF STATE
PROBABILITIESPROBABILITIES IN PERIOD ( MONTH ) 1 AND THE MONTHLY IN PERIOD ( MONTH ) 1 AND THE MONTHLY MATRIX OF TRANSITION PROBABILITIESMATRIX OF TRANSITION PROBABILITIES::
ππ ( 2 ) = ( 2 ) = ππ ( 1 ) x P ( 1 ) x P
.90.90
.25.25
.10.10
.75.75
( ( 0.50.5 , , 0.50.5 ) ) = ( = ( 0.5750.575 , , 0.4250.425 ) )[ ].575.575 .425.425
BARBER ‘A’BARBER ‘A’ BARBER ‘B’BARBER ‘B’.45.45
.125.125
.05.05
.375.375BARBER ‘A’BARBER ‘A’ BARBER ‘B’BARBER ‘B’
Markov Process ModelsMarkov Process ModelsBASIC EXAMPLEBASIC EXAMPLE
THE MARKET SHARE EACH BARBER HAS IN THE 3THE MARKET SHARE EACH BARBER HAS IN THE 3rdrd MONTH MONTHIS EQUAL TO THE PRODUCT OF THE IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATEVECTOR OF STATE
PROBABILITIESPROBABILITIES IN PERIOD (MONTH) 2 AND THE MONTHLY IN PERIOD (MONTH) 2 AND THE MONTHLY MATRIX OF TRANSITION PROBABILITIESMATRIX OF TRANSITION PROBABILITIES::
ππ ( 3 ) = ( 3 ) = ππ ( 2 ) x P ( 2 ) x P
.90.90
.25.25
.10.10
.75.75
( ( 0.5750.575 , , 0.4250.425 ) ) = ( = ( 0.620.62 , , 0.380.38 ) )[ ].62375.62375 .37625.37625
BARBER ‘A’BARBER ‘A’ BARBER ‘B’BARBER ‘B’.5175.5175
.10625.10625
.0575.0575
.31875.31875BARBER ‘A’BARBER ‘A’ BARBER ‘B’BARBER ‘B’
Markov Process ModelsMarkov Process ModelsBASIC EXAMPLEBASIC EXAMPLE
THE MARKET SHARE EACH BARBER HAS IN THE 4th MONTHTHE MARKET SHARE EACH BARBER HAS IN THE 4th MONTHIS EQUAL TO THE PRODUCT OF THE IS EQUAL TO THE PRODUCT OF THE VECTOR OF STATEVECTOR OF STATE
PROBABILITIESPROBABILITIES IN PERIOD (MONTH) 3 AND THE MONTHLY IN PERIOD (MONTH) 3 AND THE MONTHLY MATRIX OF TRANSITION PROBABILITIESMATRIX OF TRANSITION PROBABILITIES::
ππ ( 4 ) = ( 4 ) = ππ ( 3 ) x P ( 3 ) x P
.90.90
.25.25
.10.10
.75.75
( ( 0.620.62 , , 0.38 0.38 )) = ( = ( 0.653 0.653 , , 0.347 0.347 ))[ ].653.653 .347.347
BARBER ‘A’BARBER ‘A’ BARBER ‘B’BARBER ‘B’
.095.095
.062.062
.285.285
.558.558
BARBER ‘A’BARBER ‘A’ BARBER ‘B’BARBER ‘B’
Steady – State ProbabilitiesSteady – State Probabilities
Reached when the before and after state probabilities stay the same
forever, assuming no changes in thematrix of transition probabilities
Here, the eventual market share in the “barber” problem.
ALSO KNOWN AS THEALSO KNOWN AS THEEQUILIBRIUM OREQUILIBRIUM ORSTEADY-STATESTEADY-STATE
SOLUTIONSOLUTION
Equilibrium ConditionEquilibrium ConditionBARBER EXAMPLEBARBER EXAMPLE
The eventual market shares of the two barbers can be calculated directly from the matrix of transition probabilities.
Prior period vectors of state probabilities are not required.
The equations: .90 π1 + .25 π2 = π1
.10 π1 + .75 π2 = π2
.90 .10
.25 .75[ ]MATRIX OF TRANSITION
Equilibrium ConditionEquilibrium ConditionBARBER EXAMPLEBARBER EXAMPLE
Since π1 + π2 = 1.0 , π1 = 1.0 – π2
Therefore, “ 1.0 – π2 “ may be substituted for π1 in eitherof the two equations below:
.90 ( 1 – π2 ) + .25 π2 = 1 – π2
.10 ( 1 – π2 ) + .75 π2 = π2
or
Equilibrium ConditionEquilibrium ConditionBARBER EXAMPLEBARBER EXAMPLE
.90 ( 1 – π2 ) + .25 π2 = 1 – π2
.90 - .90 π2 + .25 π2 = 1 – π2
- .10 = - .35 π2
π2 = .2857
and π1 = ( 1 - .2857 ) = .7143
SUBSTITUTING IN THE 1st EQUATION, WE GET:
EVENTUALLY BARBER ‘A’ WILL HAVE 71% OF THE MARKET WHILE BARBER ‘B’ WILL HAVE THE REMAINING 29%
Equilibrium ConditionEquilibrium ConditionBARBER EXAMPLEBARBER EXAMPLE
.10 ( 1 – π2 ) + .75 π2 = π2
.10 - .10 π2 + .75 π2 = π2
.10 = .10 π2 - .75 π2 + 1.0 π2
.10 = .35 π2
π2 = .2857
π1 = ( 1 - .2857 ) = .7143
SUBSTITUTING IN THE 2nd EQUATION, WE GET:
Markov Processes with QM for WINDOWSMarkov Processes with QM for WINDOWS
We scroll to
“ Markov Analysis ”
We click on
“ New ”to solve a new problem
We specify thenumber of states.
Here, it is the market share
for the two barbers
The “ initial “ market shares(loyalty rates) are 50% / 50%
respectively.( Barber “A” is “1” )( Barber “B” is “2”)
The Matrix of Transitionis inserted to the right.
We desire to findthe market sharesover “4” periods
( months )
The “ End of Period 1 “ is actually the end of period “2”
The “ End of Period 2 “Is actually the end of period “3” , etc.
The market share ( loyalty rates ) after 4 months:
Barber A ( 1 ) - 66%Barber B ( 2 ) - 34%
The ‘Steady-State’or
‘Equilibrium’market shares
( loyalty rates ) are:
Barber ‘A’ ( 71% )Barber ‘B’ ( 29% )
Markov Processes UsingMarkov Processes Using
Template
Insert theMatrix of Transition
here
Matrix of Transition for the
next three periods
To obtain the Steady-State market shares,
( loyalty rates )
go to Tools, Solver
Steady State orEquilibirum
market shares( loyalty rates )
Gas Station ExampleGas Station ExampleMARKOV PROCESSESMARKOV PROCESSES
A town has three gas stations: A,B,C. The onlyfactor influencing the choice of station for the
next purchase is the prior purchase.
Each station is concerned about brand share.The town’s weekly gas sales are $10,000.00,and we assume each driver buys gas once
per week.
Gas Station ExampleGas Station ExampleMARKOV PROCESSESMARKOV PROCESSES
THE MARKET SHARES AT THIS PARTICULAR TIME AREAS FOLLOWS:
STATION ‘A’ - 30%STATION ‘B’ - 40%STATION ‘C’ - 30%
THEREFORE, THE VECTOR OF STATE PROBABILITIES IS:
π (1) = ( 0.3 , 0.4 , 0.3 )PERIOD NUMBER ONE
Gas Station ExampleGas Station Example
THE THE MATRIX OF TRANSITIONMATRIX OF TRANSITION PROBABILITIES PER WEEK ARE:PROBABILITIES PER WEEK ARE:
.90 .05 .05
.10 .80 .10
.20 .10 .70
P =
LOYALTY RATELOYALTY RATESTATION ‘A’STATION ‘A’
LOYALTY RATELOYALTY RATESTATION ‘B’STATION ‘B’
LOYALTY RATELOYALTY RATESTATION ‘C’STATION ‘C’
ALL OTHERS ARE DEFECTION RATESALL OTHERS ARE DEFECTION RATES
MARKOV PROCESSESMARKOV PROCESSES
Gas Station ExampleGas Station ExampleTHE MARKET SHARE THAT EACH GAS STATION HAS IN THETHE MARKET SHARE THAT EACH GAS STATION HAS IN THE
NEXT WEEK IS EQUAL TO THE PRODUCT OF THE VECTOR OFNEXT WEEK IS EQUAL TO THE PRODUCT OF THE VECTOR OFSTATESTATE PROBABILITIESPROBABILITIES IN PERIOD ( WEEK ) 1 AND THE WEEKLY IN PERIOD ( WEEK ) 1 AND THE WEEKLY
MATRIX OF TRANSITION PROBABILITIESMATRIX OF TRANSITION PROBABILITIES::
ππ ( 2 ) = ( 2 ) = ππ ( 1 ) x P ( 1 ) x P
P (A) = 0.30 A .90 .05 .05 .27 .015 .015P (A) = 0.30 A .90 .05 .05 .27 .015 .015 P (B) = 0.40 P (B) = 0.40 XX B .10 .80 .10 B .10 .80 .10 == .04 .32 .04 .04 .32 .04 P (C) = 0.30 C .20 .10 .70 .06 .03 .21P (C) = 0.30 C .20 .10 .70 .06 .03 .21 .37 .365 .265.37 .365 .265
A B CA B C
ππ ( 1 ) P ( 1 ) P
Gas Station ExampleGas Station Example THE MARKET SHARE THAT EACH GAS STATION HAS IN THETHE MARKET SHARE THAT EACH GAS STATION HAS IN THE 33rdrd WEEK IS EQUAL TO THE PRODUCT OF THE VECTOR OF WEEK IS EQUAL TO THE PRODUCT OF THE VECTOR OF
STATESTATE PROBABILITIESPROBABILITIES IN PERIOD ( WEEK ) 2 AND THE WEEKLY IN PERIOD ( WEEK ) 2 AND THE WEEKLY MATRIX OF TRANSITION PROBABILITIESMATRIX OF TRANSITION PROBABILITIES::
ππ ( 3 ) = ( 3 ) = ππ ( 2 ) x P ( 2 ) x P
P (A) = 0.37 A .90 .05 .05 .333 .018 .018P (A) = 0.37 A .90 .05 .05 .333 .018 .018 P (B) = 0.365 P (B) = 0.365 XX B .10 .80 .10 B .10 .80 .10 == .037 .292 .037 .037 .292 .037 P (C) = 0.265 C .20 .10 .70 .053 .027 .185P (C) = 0.265 C .20 .10 .70 .053 .027 .185 .423 .337 .24.423 .337 .24
A B CA B C
ππ ( 2 ) P ( 2 ) P
Gas Station ExampleGas Station ExampleTHE MARKET SHARE THAT EACH GAS STATION HAS IN THETHE MARKET SHARE THAT EACH GAS STATION HAS IN THE44thth WEEK IS EQUAL TO THE PRODUCT OF THE VECTOR OF WEEK IS EQUAL TO THE PRODUCT OF THE VECTOR OF
STATE PROBABILITIESSTATE PROBABILITIES IN PERIOD ( WEEK ) 3 AND THE WEEKLY IN PERIOD ( WEEK ) 3 AND THE WEEKLY MATRIX OF TRANSITION PROBABILITIESMATRIX OF TRANSITION PROBABILITIES::
ππ ( 4 ) = ( 4 ) = ππ ( 3 ) x P ( 3 ) x P
P (A) = 0.423 A .90 .05 .05 .381 .021 .021P (A) = 0.423 A .90 .05 .05 .381 .021 .021 P (B) = 0.337 P (B) = 0.337 XX B .10 .80 .10 B .10 .80 .10 = = .034 .269 .034 .034 .269 .034 P (C) = 0.240 C .20 .10 .70 .048 .024 .168P (C) = 0.240 C .20 .10 .70 .048 .024 .168 .463 .314 .223.463 .314 .223
A B CA B C
ππ ( 3 ) P ( 3 ) P
Steady–State ProbabilitiesSteady–State ProbabilitiesWE KNOW THAT WE HAVE ARRIVED AT THE EVENTUAL MARKETWE KNOW THAT WE HAVE ARRIVED AT THE EVENTUAL MARKETSHARES WHEN THE SHARES WHEN THE STARTING PROBABILITIESSTARTING PROBABILITIES ARE EQUAL TO ARE EQUAL TO
THE THE NEXT STATE PROBABILITIESNEXT STATE PROBABILITIES::
ππ ( Final State ) = ( Final State ) = ππ ( Starting ) x P ( Starting ) x P
P (A) = 0.P (A) = 0.589589 A .90 .05 .05 .530 .029 .029 A .90 .05 .05 .530 .029 .029 P (B) = 0.P (B) = 0.235235 XX B .10 .80 .10 B .10 .80 .10 == .024 .188 .024 .024 .188 .024 P (C) = 0.P (C) = 0.176176 C .20 .10 .70 .035 .018 .123 C .20 .10 .70 .035 .018 .123 ..589589 . .235235 . .176176
A B CA B Cππ ( starting ) P ( starting ) P
Gas Station ExampleGas Station ExampleCONCLUSIONCONCLUSION
Given that the total weekly gas sales are $10,000.00,Given that the total weekly gas sales are $10,000.00,the average weekly sales per station are:the average weekly sales per station are:
A : ( 0.589 ) ( 10,000 ) = $5,890.00A : ( 0.589 ) ( 10,000 ) = $5,890.00B : ( 0.235 ) ( 10,000 ) = $2,350.00B : ( 0.235 ) ( 10,000 ) = $2,350.00C : ( 0.176 ) ( 10,000 ) = $1,760.00C : ( 0.176 ) ( 10,000 ) = $1,760.00
Unless there is some change, the share of marketwill stay approximately 58.9% for station ‘A’,
23.5% for station ‘B’, and 17.6% for station ‘C’.
Calculation of Steady-State ProbabilitiesCalculation of Steady-State Probabilities
GAS STATION EXAMPLEGAS STATION EXAMPLE
INITIALINITIALSTATESTATE
PROBABILITIESPROBABILITIESTRANSITIONTRANSITION
PROBABILITIESPROBABILITIESNEW STATENEW STATE
PROBABILITIESPROBABILITIES
X1X1X2X2X3X3
AABBCC
A B CA B C
.90 .05 .05.90 .05 .05
.10 .80 .10.10 .80 .10
.20 .10 .70.20 .10 .70
A B CA B C
.90X1 .05X1 .05X1.90X1 .05X1 .05X1
.10X2 .80X2 .10X2.10X2 .80X2 .10X2
.20X3 .10X3 .70X3.20X3 .10X3 .70X3
P(A) = X1 P(B) = X2 P(C) = X3P(A) = X1 P(B) = X2 P(C) = X3
XX ==
Calculation of Steady-State Calculation of Steady-State ProbabilitiesProbabilities
GAS STATION EXAMPLEGAS STATION EXAMPLE
P(A) = .90XP(A) = .90X11 + .10X + .10X22 + .20X + .20X33 = 1X = 1X11
P(B) = .05XP(B) = .05X11 + .80X + .80X22 + .10X + .10X33 = 1X = 1X22
P(C) = .05XP(C) = .05X11 + .10X + .10X22 + .70X + .70X33 = 1X = 1X33
1X1X11 + 1X + 1X22 + 1X + 1X33 = 1 = 1
DEPENDENT EQUATIONDEPENDENT EQUATION
INDEPENDENT EQUATIONINDEPENDENT EQUATION
DEPENDENT EQUATIONDEPENDENT EQUATION
DEPENDENT EQUATIONDEPENDENT EQUATION
Equation ConversionEquation ConversionSET ALL 3 DEPENDENT EQUATIONS EQUAL TO ZERO:SET ALL 3 DEPENDENT EQUATIONS EQUAL TO ZERO:
P(A) = .9XP(A) = .9X11 + .1X + .1X22 + .2X + .2X33 = 1X = 1X11
.9X.9X11 – 1.0X – 1.0X11 + .1X + .1X22 + .2X + .2X33 = 0 = 0
-.1X-.1X11 + .1X + .1X22 + .2X + .2X33 = 0 = 0
P(B) = .05XP(B) = .05X11 + .8X + .8X22 + .1X + .1X33 = 1X = 1X22
.05X.05X11 + .8X + .8X22 – 1.0X – 1.0X22 + .1X + .1X33 = 0 = 0
.05X.05X11 -.2X -.2X22 + .1X + .1X33 = 0 = 0
P(C) = .05XP(C) = .05X11 + .1X + .1X22 + .7X + .7X33 = 1X = 1X33
.05X.05X11 + .1X + .1X22 + .7X + .7X33 – 1.0X – 1.0X33 = 0 = 0
.05X.05X11 + .1X + .1X22 -.3X -.3X33 = 0 = 0
Summary EquationsSummary Equations-.1X-.1X11 + .1X + .1X22 + .2X + .2X33 = 0 = 0
.05X.05X11 - .2X - .2X22 + .1X + .1X33 = 0 = 0
.05X.05X11 + .1X + .1X22 - .3X - .3X33 = 0 = 0
1X1X11 + 1X + 1X22 + 1X + 1X33 = 1 = 1
DEPENDENT EQUATIONSDEPENDENT EQUATIONS
INDEPENDENT EQUATIONINDEPENDENT EQUATION
The 3 dependent equations will not solve for theThe 3 dependent equations will not solve for thevalues of Xvalues of X11, X, X22, and X, and X33..
Therefore, we add the independent equation!Therefore, we add the independent equation!
The SolutionThe SolutionTO ELIMINATE X1 AMONG THE DEPENDENT EQUATIONS:
.05 X.05 X11 - .2X - .2X22 + .1X + .1X33 = 0 = 0 .05 X.05 X11 + .1X + .1X22 - .3X - .3X33 = 0 = 0
- .3X- .3X22 + .4X + .4X33 = 0 = 0
.05 X.05 X11 - .2X - .2X22 + .1X + .1X33 = 0 = 0
““.05”.05” ( 1.0 X ( 1.0 X11 + 1.0 X + 1.0 X22 + 1.0 X + 1.0 X33 = 1 ) = 1 )
.05 X05 X11 + .05 X + .05 X22 + .05 X + .05 X33 = .05 = .05
- .25 X.25 X22 + .05 X + .05 X33 = - .05 = - .05
The SolutionThe Solution
WITH JUST X2 AND X3 LEFT TO SOLVE, WE WILL SOLVE FORVARIABLE X2 FIRST:
- .3X- .3X22 + .4X + .4X33 = 0 = 0
““8”8” ( - .25X ( - .25X22 + .05X + .05X33 = - .05 ) = - .05 )
-2.0X2.0X22 + .4X + .4X33 = - .40 = - .40
1.7 X1.7 X22 = .40 = .40
XX22 = = .2352.2352
The SolutionThe SolutionWE SOLVE FOR XWE SOLVE FOR X33 BY SUBSTITUTING BY SUBSTITUTING “ X“ X22 = .2352 ” = .2352 ” INTO INTO
EITHER OF THE TWO DEPENDENT EQUATIONS:EITHER OF THE TWO DEPENDENT EQUATIONS:
-.3X.3X22 + .4X + .4X33 = 0 = 0-.3 (.2352) + .4X.3 (.2352) + .4X33 = 0 = 0
-.07056 + .4X.07056 + .4X33 = 0 = 0-.07056 = - .4X-.07056 = - .4X33
.1764.1764 = X = X33
WITH “XWITH “X22” and “X” and “X33” NOW KNOWN , WE EASILY SOLVE FOR” NOW KNOWN , WE EASILY SOLVE FOR““XX11” USING THE INDEPENDENT EQUATION:” USING THE INDEPENDENT EQUATION:
SINCE 1XSINCE 1X11 + 1X + 1X22 + 1X + 1X33 = 1 = 1
1X1X11 = 1 – 1X = 1 – 1X22 – 1X – 1X33
1X1X11 = 1 – ( = 1 – ( .2352.2352 ) – ( ) – ( .1764.1764 ) ) 1X1X11 = 1 – [ .4116 ] = 1 – [ .4116 ]
XX11 = = .5884.5884
Gas Station Market SharesGas Station Market SharesSTATE PROBABILITIESSTATE PROBABILITIES
Week P(A) P(B) P(C)
1 .30 .40 .30
2 .37 .365 .265
3 .423 .337 .240
4 .463 .314 .223
5 .492 .297 .211
6 .515 .283 .202
Gas Station MarketGas Station Market SharesSharesSTATE PROBABILITIESSTATE PROBABILITIES
Week P(A) P(B) P(C)
13 .577 .243 .180
14 .579 .242 .179
15 .581 .240 .179
16 .583 .239 .178
17 .584 .238 .178
18 .585 .237 .177
Gas Station Market SharesGas Station Market SharesSTATE PROBABILITIESSTATE PROBABILITIES
Week P(A) P(B) P(C)
24 .588 .236 .176
25 .588 .236 .176
26 .588 .236 .176
27 .588 .236 .176
28 .589 .235 .176
29 .589 .235 .176
Markov Processes with QM for WINDOWSMarkov Processes with QM for WINDOWS
We specify the number
of initial states.Here, it is themarket share
of the three ( 3 )gas stations
The “ Matrix of Transition “is inserted to the right of the
“ Initial States “
We arbitrarily selecttwelve ( 12 ) periods( weeks ) for analysis
The ‘ Steady - State ‘or
‘ Equilibrium ‘market shares ( loyalty rates ) are:
Station ‘A’ - 58.9 %Station ‘B’ - 23.5 %Station ‘C’ - 17.6 %
The market shares( loyalty rates )
for weeks2, 3, 4, and 5respectively
The market shares( loyalty rates )
for weeks6, 7, 8, and 9respectively
Themarket shares( loyalty rates )
for weeks 10, 11, 12, and 13
respectively
Markov Processes UsingMarkov Processes Using
Template
Changes to the transition matrixover 3 periods
Excel will not show the market shares for each gas station on a period-by-period basis !
By selecting Tools, Solver,the program will provide
the equilibrium (steady) statemarket shares
( loyalty probabilities )
Markov Process ModelsMarkov Process ModelsFOUR BASIC ASSUMPTIONS
I. The probability of an element changing from one state to another state remains constant from one period to another.
II. There are a limited number of states of nature.
Markov Process ModelsMarkov Process ModelsFOUR BASIC ASSUMPTIONS
III. If we know the present state and the matrix of transition, we can predict any future state.
IV. The parameters of the system do not change. That is, none of the states of nature are eliminated, and elements are not entering or leaving the system.
Absorbing StatesAbsorbing States
This is where the Markov model contains at least one absorbing or trapping state.
When the system reaches that absorbing state(s), it remains there forever !
Here, steady-state probabilities are no longer mean- ingful. Other measures of performance such as the mean number of transitions until absorption become important.
MARKOV PROCESSESMARKOV PROCESSES
Absorbing StatesAbsorbing StatesLIQUOR STORE EXAMPLELIQUOR STORE EXAMPLE
A liquor store owns 10,000 bottles of select wines. Each December, it permits its special customers to purchase wines from its collection. Some bottles that have not aged sufficiently are not offered and are kept for sale in future years. Currently, 3,000 bottles are not available for sale. Of the bottles available for sale in any year, some are not sold until future years. In addition, a bottle not sold may turn bad during the year and will not be available the next year.
REQUIREMENT:
How many of the 10,000 bottles will eventually be sold?
Absorbing StatesAbsorbing StatesLIQUOR STORE EXAMPLELIQUOR STORE EXAMPLE
THE FOLLOWING FOUR STATES ARE DEFINED:
1. AVAILABLE AND SOLD
2. TURNED BAD AND LOST
3. NOT SUFFICIENTLY AGED
4. AVAILABLE BUT NOT SOLD
Liquor Store ExampleLiquor Store ExampleTHE FOLLOWING THE FOLLOWING TRANSITION MATRIX TRANSITION MATRIX APPLIES:APPLIES:
FROMFROM
TOTO44332211
11
22
33
44
11 0 0 00 0 0
0 1 0 00 1 0 0
.1 .3 .4 .2.1 .3 .4 .2
.5 .2 0 .3.5 .2 0 .3
1. AVAILABLE AND SOLD , 2. TURNED BAD AND LOST , 3. NOT SUFFICIENTLY AGED , 4. AVAILABLE BUT NOT SOLD 1. AVAILABLE AND SOLD , 2. TURNED BAD AND LOST , 3. NOT SUFFICIENTLY AGED , 4. AVAILABLE BUT NOT SOLD
Absorption StateAbsorption State Problems ProblemsSUB - MATRIX SYMBOLSSUB - MATRIX SYMBOLS
• Transition Matrix “T” or “P”
• Fundamental Matrix “F” or “N”
• Transition Matrix for Absorption in the “R” or “A” Next Period •Transition Matrix for Movement Between “Q” or “B” Non-Absorption States • Identity Matrix “I”
The Identity MatrixThe Identity Matrix
State 1State 1Available Available
and Soldand Sold
State 2State 2Turned Bad Turned Bad
and Lostand Lost
State 1State 1Available Available
and Soldand Sold
State 2State 2Turned BadTurned Bad
and Lostand Lost
1 01 0
0 10 1““I”I”
( UPPER LEFT QUADRANT OF THE TRANSITION MATRIX )( UPPER LEFT QUADRANT OF THE TRANSITION MATRIX )
FROMFROM
TOTO
The “Q” or “B” MatrixThe “Q” or “B” Matrix
State 3State 3Not SufficientlyNot Sufficiently
AgedAged
State 4State 4Available but Available but
not Soldnot Sold
State 3State 3Not Sufficiently Not Sufficiently
AgedAged
State 4State 4Available but Available but
not Soldnot Sold
.4 .2.4 .2
0 .30 .3““Q”Q”
( BOTTOM RIGHT QUADRANT OF THE TRANSITION MATRIX )( BOTTOM RIGHT QUADRANT OF THE TRANSITION MATRIX )
FROMFROM
TOTO
The “R” or “A” MatrixThe “R” or “A” Matrix
State 1State 1Available Available
and Soldand Sold
State 2State 2Turned Bad Turned Bad
and Lostand Lost
State 3State 3Not Sufficiently Not Sufficiently
AgedAged
State 4State 4Available butAvailable but
not Soldnot Sold
.1 .3.1 .3
.5 .2.5 .2““R”R”
( BOTTOM LEFT QUADRANT OF THE TRANSITION MATRIX )( BOTTOM LEFT QUADRANT OF THE TRANSITION MATRIX )
FROMFROM
TOTO
Initial CalculationsInitial Calculations
1 0 .4 .2 1 0 .4 .2 .6 -.2 .6 -.2 0 1 0 .3 0 .70 1 0 .3 0 .7I - QI - Q
aa
bb
cc
dd
1 - .41 - .4 0 - .20 - .2
0 - 00 - 0 1 - .31 - .3
== ==--
LIQUOR STORE EXAMPLELIQUOR STORE EXAMPLE
Q or BMatrix
IdentityMatrix
The Fundamental MatrixThe Fundamental Matrix
F = [ I – Q ] =F = [ I – Q ] =-1 d/e – c/ed/e – c/e
-b/e a/e-b/e a/e==
..77/./.4242 . .22/./.4242
00 /. /.4242 . .66/./.4242
Where “Where “ee” = ( a x d ) – ( b x c ) = ( .6 x .7 ) – ( 0 x -.2 ) = [ .42 – 0] = ” = ( a x d ) – ( b x c ) = ( .6 x .7 ) – ( 0 x -.2 ) = [ .42 – 0] = .42.42
IdentityMatrix
“Q” or “B”
MatrixNegative Power
( matrix inversed )
The Fundamental MatrixThe Fundamental Matrix
F =F = 1.667 .4761.667 .476 0 1.4290 1.429
SUMSUM
2.1432.1431.4291.429
FOR “SFOR “S33” ” UNAVAILABLE UNAVAILABLE BOTTLES, THIS IS THE MEANBOTTLES, THIS IS THE MEANNUMBER OF YEARS UNTIL NUMBER OF YEARS UNTIL
DISPOSAL ( DISPOSAL ( 2.14 years2.14 years ) )
FOR “SFOR “S44” ” AVAILABLEAVAILABLEBOTTLES, THIS IS THE MEANBOTTLES, THIS IS THE MEAN
NUMBER OF YEARS UNTILNUMBER OF YEARS UNTILDISPOSAL ( DISPOSAL ( 1.429 years1.429 years ) )
THE NUMBER OF PERIODSTHE NUMBER OF PERIODS( ( HERE HERE YEARSYEARS ) THAT BOTTLES) THAT BOTTLES
WILL BE IN ANY OF THE WILL BE IN ANY OF THE NON - ABSORBING STATESNON - ABSORBING STATES
BEFORE ABSORPTION FINALLYBEFORE ABSORPTION FINALLYOCCURSOCCURS
The FR MatrixThe FR Matrix
FR = FR = 1.667 .4761.667 .476 0 1.4290 1.429 XX .1 .3.1 .3
.5 .2.5 .2
aa bb
cc dd
““FF” MATRIX” MATRIX ““RR” MATRIX” MATRIX
ee ff
gg hh
==.1667 + .238.1667 + .238 0 + .71450 + .7145
.5001 + .0952.5001 + .0952 0 + .28580 + .2858
ae + bgae + bg
ce + dgce + dg
af + bhaf + bh
cf + dhcf + dh
The FR MatrixThe FR Matrix
FR =FR = .4047 .5953.4047 .5953.7145 .2858.7145 .2858
SS33
SS44
SS11 S S22THE PROBABILITIES OFTHE PROBABILITIES OF
ABSORPTION GIVEN ANYABSORPTION GIVEN ANYSTARTING STATESTARTING STATE
40.47%40.47% chance that chance thatan inadequately-agedan inadequately-agedbottle will eventuallybottle will eventually
be soldbe sold
71.45%71.45% chance that chance thatan available, but notan available, but not
sold bottle will eventuallysold bottle will eventuallybe sold.be sold.
59.53%59.53% chance that chance thatan inadequately-agedan inadequately-agedbottle will eventuallybottle will eventuallyturn bad and be lostturn bad and be lost
28.58%28.58% chance that chance thatan available, but notan available, but not
sold bottle will eventuallysold bottle will eventuallyturn bad and be lost.turn bad and be lost.
The Final SolutionThe Final SolutionLIQUOR STORE EXAMPLELIQUOR STORE EXAMPLE
10,000 bottles of wine are currently owned by the liquor store
3,000 bottles of wine are not nownot now available for sale ( given )
7,000 bottles of wine are now availablenow available for sale ( inferred )
EVENTUAL ( EXPECTED ) SALES in bottles :
S3: .405.405 x 3,000 bottles = 1,215 bottles
S4: .714.714 x 7,000 bottles = 4,998 bottles
Σ = 6,2136,213 bottles
There are 4 states:
1. Available & sold2. Turned bad & lost3. Not sufficiently aged4. Available but not sold
Since we do not knowthe initial state
probabilities, we can arbitrarily
assign an equalprobability to each
event ( 25% )
The ‘ Matrix of Transition ‘is inserted to the right
of the initial stateprobabilities
The‘ FR ‘ Matrix
40.47% chance that an inadequately - aged bottle will eventually be sold
71.45% chance that an available but not yet sold bottle will eventually be sold
59.53% chance that an inadequately - aged bottle will turn bad and be lost
28.58% chance that an available but not yet sold bottle will eventually turn bad and be lost
Eventually,
52.98% of all bottles will be sold
&47.02% of all bottles
will be lost
The “ B “ or “ Q “ Matrix
The “ F “ or “ Fundamental Matrix “
The “ FA “ or “ FR “ Matrix
Markov Processes UsingMarkov Processes Using
Matrix of TransitionDevelopment
( period by period )
State 1Available + Sold
State 2Turned Bad + Lost
State 3Not Aged Enough
State 4Available, Not Sold
State 1Available + Sold
State 2Turned Bad + Lost
State 3Not Aged Enough
State 4Available, Not Sold
40.5% chance bottles not aged enough will be sold eventually
59.5% chance bottles not aged enough will be lost eventually
71.4% chance bottles available will be sold eventually
28.6% chance bottles available will be lost eventually
Markov ProcessesMarkov Processes
System Change Over TimeSystem Change Over Time