Statistical Probabilistic Model Checking Håkan L. S. Younes Carnegie Mellon University
Jan 23, 2016
StatisticalProbabilistic Model Checking
Håkan L. S. YounesCarnegie Mellon University
2
Introduction
Model checking for stochastic processes Stochastic discrete event systems Probabilistic time-bounded properties
Model independent approach Discrete event simulation Statistical hypothesis testing
3
Example:Tandem Queuing Network
q1 q2
arrive route depart
q1 = 0q2 = 0q1 = 0q2 = 0
q1 = 1q2 = 0
q1 = 1q2 = 1
q1 = 2q2 = 0
q1 = 1q2 = 0
t = 0 t = 1.2 t = 3.7 t = 3.9 t = 5.5
With both queues empty, is the probability less than 0.5that both queues become full within 5 seconds?
q1 = 1q2 = 0
q1 = 2q2 = 0
q1 = 1q2 = 1
q1 = 1q2 = 0
4
Probabilistic Model Checking
Given a model M, a state s, and a property , does hold in s for M? Model: stochastic discrete event
system Property: probabilistic temporal logic
formula
5
Continuous Stochastic Logic (CSL)
State formulas Truth value is determined in a single
state Path formulas
Truth value is determined over a pathDiscrete-time analogue: PCTL
6
State Formulas
Standard logic operators: , 1 2, …
Probabilistic operator: P≥ () Holds in state s iff probability is at
least that holds over paths starting in s
P< () P≥1– ()
7
Path Formulas
Until: 1 U ≤T 2
Holds over path iff 2 becomes true in some state along before time T, and 1 is true in all prior states
8
CSL Example
With both queues empty, is the probability less than 0.5 that both queues become full within 5 seconds? State: q1 = 0 q2 = 0
Property: P<0.5(true U ≤5 q1 = 2 q2 = 2)
9
Model Checking Probabilistic Time-Bounded Properties
Numerical Methods Provide highly accurate results Expensive for systems with many states
Statistical Methods Low memory requirements Adapt to difficulty of problem
(sequential) Expensive if high accuracy is required
10
Statistical Solution Method [Younes & Simmons 2002]
Use discrete event simulation to generate sample paths
Use acceptance sampling to verify probabilistic properties Hypothesis: P≥ () Observation: verify over a sample
pathNot estimation!
11
Error Bounds
Probability of false negative: ≤ We say that is false when it is true
Probability of false positive: ≤ We say that is true when it is false
12
Performance of Test
Actual probability of holding
Pro
bab
ility
of
acc
ep
tin
gP
≥ (
) as
tru
e
1 –
13
Ideal Performance of Test
Actual probability of holding
Pro
bab
ility
of
acc
ep
tin
gP
≥ (
) as
tru
e
1 –
False negatives
False positives
Unrealistic!
14
Realistic Performance of Test
Actual probability of holding
Pro
bab
ility
of
acc
ep
tin
gP
≥ (
) as
tru
e
1 –
p1 p0
Indifference region
False negatives
False positives
2
15
SequentialAcceptance Sampling [Wald 1945]
True, false, or another observatio
n?
16
Graphical Representation of Sequential Test
Number of observations
Nu
mb
er
of
posi
tive
ob
serv
ati
on
s
17
Graphical Representation of Sequential Test
We can find an acceptance line and a rejection line given , , , and acceptance line
rejection line
reject
accept
continue
Number of observations
Nu
mb
er
of
posi
tive
ob
serv
ati
on
s
Start here
Verify oversample paths
Continue untilline is crossed
18
Special Case
p0 = 1 and p1 = 1 – 2 Reject at first negative observation Accept at stage m if p1
m ≤ Sample size at most dlog / log p1e
“Five nines”: p1 = 1 – 10–5
Maximum sample size
10–2 460,515
10–4 921,030
10–8 1,842,059
19
Case Study:Tandem Queuing Network
M/Cox2/1 queue sequentially composed with M/M/1 queue
Each queue has capacity n State space of size O(n2)
1 2
a……
1 − a
20
Tandem Queuing Network (results) [Younes et al. 2004]V
eri
fica
tion
tim
e (
seco
nds)
Size of state space101 102 103 104 105 106 107 108 109 1010 1011
10−2
10−1
100
101
102
103
104
105
106 T=500 (numerical)T=50 ( " )T=5 ( " )T=500 (statistical)T=50 ( " )T=5 ( " )
P≥0.5(true U≤T full)
= 10−6
= = 10−2
= 0.5·10−2
21
Tandem Queuing Network (results) [Younes et al. 2004]
n=255 (numerical)n=31 ( " )n=3 ( " )n=255 (statistical)n=31 ( " )n=3 ( " )
Veri
fica
tion
tim
e (
seco
nds)
T
10−2
10−1
100
101
102
103
104
105
106
101 102 103 104
= 10−6
= = 10−2
= 0.5·10−2
P≥0.5(true U≤T full)
22
Case Study:Symmetric Polling System
Single server, n polling stations Stations are attended in cyclic
order Each station can hold one message State space of size O(n·2n)
Server
…Polling stations
23
Symmetric Polling System (results) [Younes et al. 2004]
T=40 (numerical)T=20 ( " )T=10 ( " )T=40 (statistical)T=20 ( " )T=10 ( " )
Veri
fica
tion
tim
e (
seco
nds)
Size of state space
10−2
10−1
100
101
102
103
104
105
106
102 104 106 108 1010 1012 1014
serv1 P≥0.5(true U≤T poll1)
= 10−6
= = 10−2
= 0.5·10−2
24
Symmetric Polling System (results) [Younes et al. 2004]
n=15 (numerical)n=10 ( " )n=5 ( " )n=15 (statistical)n=10 ( " )n=5 ( " )
Veri
fica
tion
tim
e (
seco
nds)
T
10−2
10−1
100
101
102
103
104
105
106
101 102 103
= 10−6
= = 10−2
= 0.5·10−2
serv1 P≥0.5(true U≤T poll1)
25
Symmetric Polling System (results) [Younes et al. 2004]
numerical (=10−6)==10−2
==10−4
==10−6
==10−8
==10−10
Veri
fica
tion
tim
e (
seco
nds)
10−1
100
101
102
10−4 10−210−3
n = 10T = 40
serv1 P≥0.5(true U≤T poll1)
26
Tandem Queuing Network: Distributed Sampling Use multiple machines to generate samples
m1: Pentium IV 3GHz m2: Pentium III 733MHz m3: Pentium III 500MHz
% samples % samples m1 only
n m1 m2 m3 time m1 m2 time time
63 70 20 10 0.46 71 29 0.50 0.58
2047 60 26 14 1.28 70 30 1.46 1.93
65535 65 21 14 26.29 67 33 33.89 44.85
27
Summary
Acceptance sampling can be used to verify probabilistic properties of systems
Sequential acceptance sampling adapts to the difficulty of the problem
Statistical methods are easy to parallelize
28
Other Research
Failure trace analysis “failure scenario” [Younes & Simmons 2004a]
Planning/Controller synthesis CSL goals [Younes & Simmons 2004a]
Rewards (GSMDPs) [Younes & Simmons 2004b]
29
Tools
Ymer Statistical probabilistic model
checking Tempastic-DTP
Decision theoretic planning with asynchronous events
30
ReferencesWald, A. 1945. Sequential tests of statistical hypotheses. Ann.
Math. Statist. 16: 117-186.Younes, H. L. S., M. Kwiatkowska, G. Norman, and D. Parker.
2004. Numerical vs. statistical probabilistic model checking: An empirical study. In Proc. TACAS-2004.
Younes, H. L. S., R. G. Simmons. 2002. Probabilistic verification of discrete event systems using acceptance sampling. In Proc. CAV-2002.
Younes, H. L. S., R. G. Simmons. 2004a. Policy generation for continuous-time stochastic domains with concurrency. In Proc. ICAPS-2004.
Younes, H. L. S., R. G. Simmons. 2004b. Solving generalized semi-Markov decision processes using continuous phase-type distributions. In Proc. AAAI-2004.