Concur 2001 August 21, 2001
Performance Evaluation := (Process Algebra + Model
Checking)x Markov Chains
Holger Hermanns and Joost-Pieter Katoen
with contributions ofChristel Baier, Ed Brinksma, Boudewijn Haverkort, Ulrich Herzog, Joachim Meyer-Kayser, Markus Siegle
22
A reactive, embedded system:The ‘Hubble Space Telescope’A reactive, embedded system:The ‘Hubble Space Telescope’
and its stabilising
unit
33
s
r
56 4 23 1 crash
f f f f f f
sleep sleep ff
r
s
A simple model of the Hubble
The base station prepares a shuttle mission to repair the telescope (r).
Each gyroscope may fail (f).
The telescope turns into sleep mode if less than 3 gyroscopes remain operational (s).
Without operational gyro the telescope eventually crashes.
44
What is this? What is it good for?
A model
A stochastic model
A continuous-time Markov model
Prediction of the system behaviour
Computer-assisted analysis of
CorrectnessPerformanceDependability
on the basis of a model, instead of the real system
s
r
56 4 23 1 crash
sleep sleepf
fr
s
f f f f f f
55
Quantitative Verification
Information technology is finally reaching a scale where
probabilistic methodsprobabilistic methods should play a larger role in system design.
D. Tennenhouse, director research Intel Corp.
Proactive Computing, Communications of the ACM, May 2000
66
Why probabilities?practically relevant for
deterministically unsolvable problems:randomised distributed algorithms.
unreliable and unpredictable system behaviour:fault tolerant systems, ...
performance and dependability analysis:‘quality of service’, ...
wheighting important (likely/frequent) and unimportant (unlikely/rare) aspects in the specification.
approximating large ‘populations’ of discrete structures
77
s
r
56 4 23 1 crash
6 f 5 f 4 f 3 f 2 f f
sleep sleep2 ff
r
s
A Markov model of the Hubble
The base station prepares a shuttle mission to repair the telescope (r).
Each gyroscope posesses a failure rate f.
To turn on sleep mode requires some time (s).
Without operational gyroscope the telescope eventually crashes.
88
Specification formalisms for CTMCs
stochastic Petri nets [Molloy]
Markovian queueing networks [Muppala & Trivedi]
stochastic automata networks [Plateau]
stochastic process algebra [Herzog et al]
probabilistic I/O automata [Stark et al]
and many variants/combinations thereof.
99
Continuous-time Markov chains (CTMCs)
(finite state) automata,
all times are exponentially distributed,
sojourn time in states are memory-less,
very well investigated class of stochastic processes,
widely used in practice,
best guess, if only mean values are known,
efficient and numerically stable algorithms for stationary and transient analysis are available.
00.10.2
0.30.40.50.60.7
0.80.9
1
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
h
PrPr(X (X >>t) = t) = ee--hhtt
1010
1
1
2
33
Transient and Stationary Behaviour of CTMCs
transient probability
stationary (‘steady state’) probability
)0( s, )10( s, )20( s, )30( s, )40( s, )60( s, )70( s, )80( s, )90( s, )100( s, )110( s, )120( s, )130( s, )140( s, )150( s, )160( s, )170( s, )180( s, )190( s, )200( s, )210( s, )220( s,
s
)( s,
1111
1212
Model Checking CTMCs
Continuous Stochastic Logic
Fixpoint Characterisations
Model Checking Algorithms
Extensions and Applications
1313
Model Checking
Automated verification technique
Checks whether a given finite-state model satisfies a given requirement, by
systematic state-space explorationeffective means to combat the state-space explosion
Some model checkers: Spin, SMV, Mur, Uppaal
Application areas:hardware verification (VHDL-code, ...)software validation (storm surge barrier, ...)software bug hunting (web server design, e-commerce, ...)
1414
CTL - Computation Tree Logic
state-formula :
true
a atomic proposition’
1 2 ‘and’
‘not’
‘for All paths’
‘there Exists a path’
path-formula :
X ‘neXt’
1 U 2 ‘Until’
‘eventually’
‘invariantly’
a branching-timetemporal logic
powerful specification language for requirements
widely used
true U =
=
[Clarke & Emerson 83]
1515Sat(6) Sat(6) Sat(sleep)
Model checking CTL by example
Given: a finite-state model and a CTL state-formula :
Strategy: calculate recursively the sets for all sub-formulas of
| sSsSat
= ( 6 U sleep)
56 4 23 1 crash
sleep sleep
initialisation first iterationsecond iterationthird iterationfourth iteration
Sat()
s satisfies
fifth iteration
fixed point!
1616
Basic idea specify a desired performance/reliability property using appropriate extension oftemporal logic, e.g.,
P<0.01(<10 error) , S<10-6(error) ,
or similar
probability that an error occurs within 10 years is less than 1 %probability that an error occurs in equilibrium is less than 10-6.
interpret and check these formulas on CTMCs
1717
state-formula :
true
a atomic proposition
1 2 and
not
for all paths
there is a path
CSL - Continuous Stochastic Logic
path-formula :
X neXt
1 U 2 Until
CTL plus probabilistic path-quantifier [Hansson and Jonsson]
probabilistic ‘time-bounded until’ [Aziz et al]
stationary probability quantifier
[Baier et al]
0
,
1,0
I
p
state-formula :
true
a atomic proposition
1 2 and
not
S~p() stationary probability
P~p() path probability
path-formula :
XI timed neXt
1 UI 2 timed Until
1818
A few requirements for the Hubble
availability? S>p( (sleep crash))
gyroscope failure between 1993 and 1997? P>q([3,7] 6)
sleep mode between 1997 and September 1999?
Pr( sleep U[7,9.8]sleep)
risk of a crash before 2010? P<10-2([0,20] crash)
56 4 23 1 crash
0.6 0.5 0.4 0.3 0.2 0.1
sleep sleep0.20.1
6
6
100 100
1990
1919
State formulas:
s a iff a L(s) s 1 2 iff s i , i=1,2
s iff s /
state in at time t
probability that “on the long run” the system is in a -state (when starting in s)
requires -algebra
and probability measure
Prob on paths of CTMC
ptst
~ @| lim
PathsProb s S~p() iff
ps ~ | PathsProb s P~p() iff
Formal semantics of CSL (1)
2020
Path formulas:interpretation over the paths (from state ) in a CTMC
state wins the race after time units, and so on
33
22
110
ts
ts
ts
ts o
0 s
01 ts
kk
k
ttytt
sy
010
with@ where
2
1
@
@ . ,0
.
x
yxy
Ix
1 UI 2 iff
Formal semantics of CSL (2)
XI iff s1 and It 0
2121
Model Checking CTMCs
Continuous Stochastic Logic
Fixpoint Characterisations
Model Checking Algorithms
Extensions and Applications
2222
For the non-probabilistic fragment: as for CTL
Model checking CSL Given: a CTMC and a CSL state-formula :
Strategy: recursively compute the sets for all sub-formulas of
| sSsSat
2323
Model checking CSL Given: a CTMC and a CSL state-formula :
Strategy: recursively compute the sets for all sub-formulas of
Steady-state operator requires slight adaptations of standard methods for steady-state probabilities
S~p() ps,s'ss
~ '
iff
steady state probability for s’ in the BSCC Bsystem of linear equations
graph algorithm
system of
linear equations
matrix-vector multiplication
Bs sBs
s
tstsss
B ' ' ,Pr
' 0
'@|',
BSCC
BSCC
PathsProb
if
if
for
where
| sSsSat
2424
BSCC B1
BSCC B2
{stable}{unstable}
{initial}{stable} 1
1
2
33
S 0.5 (P 0.98 ( 1.5 stable) )
s
5.03
15.015.0
5.0 ,Pr ,Pr 21 BsBs
3
1
1
2
1
B
B
An example
2525
Model checking CSL Given: a CTMC and a CSL state-formula :
Strategy: recursively compute the sets for all subformulas of
P~p()
,Pr ps s iff
Probabilistic state-formula with ‘neXt step’ X and ‘until’ U are treated as in the discrete-time case [Hansson & Jonsson]
vector U is the least fixed point in [0,1] of
if s 2 then
if s / 1 2 then
if s 1 2 then
ss F ,Pr 21 s's,s's
s
s
s'
FF
0F
1F
P
'
,Prs
s,s's P X matrix-vector multiplication
system of linear equations
iterative solution
| sSsSat
2626
dxxts't
es,s'ts
ts
ts
s'
xs ,F ,F
0,F
1,F
0
EQ
tss ,F,Pr 21 values Ut are the least solution in [0,1] of
if s 2 then
if s / 1 2 then
if s 1 2 then
Model checking ‘time-bounded until’
21 ,Pr s' U t-x
s’s
1 2 2
t
t0 x
t-x
system of integral equations
probability to move from s to s’ at time x
2727
Model Checking CTMCs
Continuous Stochastic Logic
Fixpoint Characterisations
Model Checking Algorithms
Extensions and Applications
2828
Model checking ‘time bounded until’ Pr(s, 1 UI 2) via transient
analysis
transient analysis determines a snapshot of the state probabilities at time t (if starting in state s at time 0)
state-of-the-art: uniformisation
numerically stable
(relatively) easy to implement: boils down to iterative matrix-vector multiplications
a priori calculation of number of iterations based on user-given accuracy
on-the-fly steady-state detection possible
)( s,t
2929
calculating transient probabilities:
Transient analysis of CTMCstransient probability distribution (s,t ):
the (snapshot)
probability at time t when starting
in state s at time 0
'@|)( ' stss,ts PathsProbin CSL expressed as:
P~p([t,t] ats’ )
and
S~p(ats’)
),(lim)( '' tss st
s
steady-state probability (s):
EQQ Diagˆ i.e.
CTMC, of matrix generator ˆ),()( Q tss
dt
d
Chapman-Kolmogorov equation
3030
Transient analysis of CTMCs
to rise gives ˆ),()( Q tssdt
d
Techniques: Runge-Kutta and (more efficient and accurate):
Uniformisation (“Jensen’s Method”)
Basic idea of uniformisation:
transform CTMC into a corresponding DTMC,
normalise transition rates w.r.t. shortest (average) residence time
!
)ˆ(
0
i
i
i
tQas compute
otherwise 0 and
i.e. ies,probabilit initial
,(s,t)πs 1
)0()( ˆ tes,s,t Q
ˆ~
*
QIP
ˆ iii* qmaxwith
3131
ˆ
~
*
QIP
Uniformisation
different outgoing rates per stateno self-loops*= +
same outgoing “rate” * per state branching probabilities self-loops (mimic delays)
10
2
CTMC*,ˆ Q
DTMC
P~
/ ( +)
/ ( +)
/ ( +)
/ ( +)
0 1 2 +
+
3232
(given stepping rate *)
Uniformisation
0 Pr)( s,n,tns,t in steps probability distribution
in DTMC after n steps,
starting from state s
P
~,1,
)0,,0,1,0,0(0,
nsπnsπ
sπ
matrix-vector
multiplication
Round-off error can be calculated a priori:
probability of n arrivals in [0,t]in a Poisson process with rate *
!
**
n
nte
compute
recursively
(Fox-Glynn)
k
n
t
n
ntetss,t
0
*
!1,
*crequired
accurac
y
number of steps in
DTMC
exact compute
d
3333
Reduction to transient analysis
Aim: Compute Pr(s, 1 UI 2) via (...,... )
1 2
1 2
1 2
s
1 2
3434
1 2
1 2
1 2
s
1 2
s’ (s,t)2 's
1 2
1 2
1 2
s
1 2
s’ (s,t)2 's
Lemma A
Pr(s, 1 U[0,t0,t] 2) =
1 2
1 2
1 2
s
1 2
Assume all 2-states are absorbing
3535
Pr(s, 1 U[0,t0,t] 2)
1 2
1 2
1 2
s
1 2
Pr(s, 1 U[0,t0,t] 2)
1 2
1 2
1 2
s
1 2
Pr(s, 1 U[0,t0,t] 2)
1 2
1 2
1 2
s
1 2
Theorem 1
Pr(s, 1 U[0,t0,t] 2) =
then apply Lemma A
1 2
1 2
1 2
s
1 2
= s’ (s,t )2 's
3636
Model checking CSL
‘Bottom-up’ strategy along the property of interest,
recursively collects states satisfying sub-formulae
Ingredients:
graph algorithms, and matrix-vector multiplication
solvers for linear equation systems
model transformations and uniformisation
Worst-case time complexity:
O(|formula| x (M.q.tmax + N2.81))
number of transitions Muniformisation rate qmaximal time-bound tmax
number of states N
3737
Lumping
Two CTMCsCTMCs are lumping equivalentlumping equivalent, if they can mimic their
cumulated ratescumulated rates stepwise, and stay bisimilar in doing so
22
if then ,
and vice versa, and so on
such that = ,
Lumping ensures that cumulated (transient/steady)-state probabilities of
equivalent states can be computed on the quotient CTMC
3838
Lumping and CSL
Two states in a CTMC are lumping equivalentlumping equivalent
if and only if
they satisfy the same CSL-formulas
(... if the bisimulation respects the state labelling)
3939
Model Checking CTMCs
Continuous Stochastic Logic
Fixpoint Characterisations
Model Checking Algorithms
Extensions and Applications
4040
The model checker
implemented in JAVA (version 1.2 with Swing)
about 8,000 lines of code, 15 man months
implements iterative numerical algorithms to solvelinear system of equations (standard)
uses backwards uniformisation for UI
uses dedicated algorithms for P=1() and P=0()
uses sparse data structures for matrices
www7.informatik.uni-erlangen.de/etmcc/TE MC2
4141
The model checker TE MC2
GUIGUI
VerificationparametersVerificationparameters
ModelinputModelinput
ResultoutputResultoutput
PropertyManagerPropertyManager
Tool DriverTool Driver CSL parserCSL parser
S~p() P~p() S~p() P~p()
State Space ManagerState Space Manager
SatSat States States TransitionsTransitions RatesRates
Analysis Engine
( 1 U 2) ( 1 U 2)
BSCC
Analysis Engine
( 1 U 2) ( 1 U 2)
BSCC
FilterFilter
Numerical Engine
Linear systems of equationsNumerical integration
Backwards uniformisation
Numerical Engine
Linear systems of equationsNumerical integration
Backwards uniformisation
4242
Current developments
Application/case studies:performance assessment of cyclic polling systemdependability analysis of a workstation clusterperformance and availability analysis of distributed database server
Extensions towards CTMCs with costs (rewards): “with probability at most 0.01 at most 10 jobs have been processed before the first error occurs”
extension of CSL has been definedmodel checking combined reward- and time-bounded formulas?
Using symbolic data structures (MTBDDs) in Prism
Extension of model checking algorithms for Markov decision processes
4343
Summary
CTMC algebra:
compositional and abstract specificationautomated generation of CTMCsreduction and comparison of performance models
CTMC model checking:
specification language for performance propertiesautomated verification technique with property-driven transformationallows model reduction cross-fertilisation of formal
specification and performance modeling techniques
cross-fertilisation of formalverification and performance
analysis techniques