Conditional Probabilities over Probabilistic and Nondeterministic Systems M. E. Andrés and P. van Rossum Radboud Universiteit Nijmegen, The Netherlands.

Conditional Probabilities over Conditional Probabilities over Probabilistic and Nondeterministic Probabilistic and Nondeterministic

SystemsSystems

M. E. Andrés and

P. van Rossum

Radboud Universiteit Nijmegen, The Netherlands.

2TACAS - April 1TACAS - April 1stst

Budapest, HungaryBudapest, HungaryMiguel E. AndresRadboud University

OverviewOverview

Motivation Background

Markov Decision Processes and Schedulers Conditional Probabilities pCTL

Our Logic (cpCTL) Model Checking issues

Fully probabilistic case Probabilistic and Nondeterministic case

Comparison (pCTL vs cpCTL) cpCTL Complications

Model Checker Counterexamples Future work



OverviewOverview


Markov Decision Processes and Schedulers Conditional Probabilities pCTL







MotivationMotivation

Model Checking

Modelj=

Temporal Logics

'

P[§ DeadL]§ DeadL

P+[§DeadL]P+[§ DeadL j¤ SingU]

· 0:1

· 0:1

· 0:1(+ cond prob) cpCTL

(+ nondet) pCTL

(+ prob) pCTL

LTL – CTL

NEW



MotivationMotivation

Conditional ProbabilitiesAnonymity

Strong AnonymityProbable innocence

What we doDefine cpCTLModel Checker for cpCTLPresent a Notion of Counterexamples

Deterministic CaseNondeterministic Case

Risk assessmentP[dyke breaks| it rains heavily]

Diagnosability P[A failed|error message E]



OverviewOverview


Markov Decision Processes and Schedulers pCTL Conditional Probabilities







Probabilistic and Nondeterministic

Example

Background – MDPsBackground – MDPs

The Model (MDP)

®0®®®

TEXPH3:1:2®£ ±2

² S is the¯nitestatespaceof thesystem² s0 2 S is the initial state² L : S ! }(P ) is a labeling function² ¿: S ! }(Distr(S))

MDP =(S,s0;L ;¿), where:

Finite Paths Paths

s0s2s0s2s3...

s0s2(s3)!

s0(s1)!...



Background – SchedulersBackground – Schedulers

Schedulers resolve the Nondeterminism!

Schedulers: FinitePath ! Distr(S)

²P [s0s2s5]= 18

²P [s0s2s6]= 0

S2 ! ¼2

S2 ! ¼3

²P [s0s2s5]= 0²P [s0s2s6]= 1

40

S214! ¼2

S234! ¼3



Background – pCTLBackground – pCTL

SyntaxisState

Pathª :=©U©j §©j ¤© Semantic

¾j= ÁUÃ , Á holds until at somepoint Ã holds¾j= §Á , ¾j= trueUÁ¾j= ¤Á , ¾j= : § : Á

©:= P j ©^©j : ©j 8ª j 9ª j P ./ a[ª ]

a2 [0;1]

./ 2 f<;· ;>;¸ g

s j= var , var 2 L(S)s j= Á^Ã , s j= Á and s j= Ãs j= : Á , s 6j= Ás j= 8Á , ¾j= Á for all f. paths¾starting fromss j= 9Á , ¾j= Á for any f. path ¾starting fromss j= P · a[Á] , max´ P s;´ [Á] , P+

s [Á] · a



Example

6j=

Background – computing satisfactionBackground – computing satisfaction

34+

140 =

0;775 34+14(12¡ ®) +

14®= 0;875



Background – Conditional ProbabilitiesBackground – Conditional Probabilities

Standard Conditional Probabilities

P (A j B) =P (A \ B)P (B)

Max and Min Conditional Probabilities

P +(¢ 1 j ¢ 2) = sup´2Sch> 0

¢ 2

P ´ (¢ 1 j ¢ 2) P ¡ (¢ 1 j ¢ 2) = inf´2Sch> 0

¢ 2

P ´ (¢ 1 j ¢ 2)

Conditional Probabilities over MDPs

P ´ (¢ 1j¢ 2) =P ´ (¢ 1 \ ¢ 2)P ´ (¢ 2)

² ( s;Bs;P ´ ) is theprobability space² ¢ 1;¢ 2 2 Bs are two sets of paths² P ´ (¢ 2) > 0

² ( ;F;P ) is a probability space² A;B 2 F are two events² P (B) > 0



OverviewOverview









s j= P · a[ÁjÃ]P s [Á^Ã]P s [Ã]

· a

Our Logic – cpCTLOur Logic – cpCTL

pCTL cpCTL

ª :=©U©j §©j ¤©

j

©:= P j ©^©j : ©j 8ª j 9ª j P ./ a[ª ]

Interpretation

P+s [ÁjÃ]

s j= P · a[ÁjÃ] max´2Sch> 0

P s;´ [Á^Ã]P s;´ [Ã]

· a

P [AjB]=P [A \ B]P [B]



max´P s0;´

[§ B ^¤ P ]

P s0;´[¤ P ]

· 0;99

cpCTL - ExamplecpCTL - Example

S0 j= P · 0;99[§ B j¤P ]

²P s0;´¼2[§ B j¤P ]= P [s0s1]+P [s0s2s3]

P [s0s1]+P [s0s2s3]+P [s0s2s4]= 1¡ 2®

7

max(1¡ 2®7; 3031) · 0;99

²P s0 ;´¼3[§ B j¤P ]= P [s0s1]

P [s0s1]+P [s0s2s6]= 30

31



OverviewOverview









Model Checking IssuesModel Checking Issues

Fully probabilistic case

Can be reduced to a pCTL* problem, using

P +s [ÁjÃ] 6=

P +s [Á^Ã]P +s [Ã]

Observation

Probabilistic and Nondeterministic case

pCTL cpCTLDeterministic Schedulers Deterministic Schedulers

History Independent Schedulers

Semi History Independent Schedulers

Bellman Equations NO Bellman Equations

P +s [ÁjÃ]=max

´

P s;´ [Á^Ã]

P s;´ [Ã]



Model Checking Issues – Model Checking Issues – Nondeterministic caseNondeterministic case

cpCTL case Deterministic Schedulers (Not trivial) Semi History Independent Schedulers No Bellman equations

Theorem: Deterministic Schedulers

P ´ [ÁjÃ]= P+[ÁjÃ] and P ´0[ÁjÃ]= P ¡ [ÁjÃ]

Thereexists Deterministic schedulers ´ and ´0 such that

Coming…




Semi History Independent Schedulers Why?If P +

s0[§ B j§ P ]= P s0 ;´

[§ B j§ P ]then ´ satis es

´(¾) =

8<

:

¼3 if ¾= s0¼5 if ¾= s0s3¼1 if ¾= s0s3s0

Definition´ is ' -semi History Independent if

² ´ takes always the samedecision before the system reaches '² ´ takes always the samedecision after the system reaches '

P ´ [ÁjÃ]= P+[ÁjÃ] and P ´0[ÁjÃ]= P ¡ [ÁjÃ]Thereexists deterministic and sHI schedulers ´ and ´0 such that

Theorem: sHI Schedulers

Stopping condition



Local Bellman equation


P +s2[§ P ]=

¼2

¼3

P +s [Á]= max

¼2¿(s)

0

@X

t2succ(s)

¼(t) ¢P +t [Á]

1

ABellman Equations

110

¢P+s6[§P ]+

910

¢P+s7[§P ]

(12¡ ®) ¢P +

s3[§ P ]+®¢P +

s4[§ P ]+

12¢P +

s5[§ P ]

P+s2[§P ]=max

8<

:

(12¡ ®) ¢P+

s3[§P ]+®¢P+

s4[§P ]+ 1

2 ¢P+s5[§P ]

110 ¢P

+s6[§P ]+ 9

10 ¢P+s7[§P ]

M aximum over all outgoingdistributions ¼of s

RecursiveComputation




Why Not Bellman equations?

Bellman equation on cpCTL case… P +s0[ÁjÃ]= max

¼2¿(s)

0

@X

t2 succ(s)

¼(t) ¢P +t [ÁjÃ]

1

A

P+s0 [§Bj¤P ] · 0;99

max(1¡ 2®7; 3031) · 0;99

P +s0[§Bj¤P ]= P s0;´¼3

[§Bj¤P ]

If ®¸ 762 then

…but P +s2[§Bj¤P ]= P s2;´¼2

[§Bj¤P ]= 1¡ 2¢®



OverviewOverview









Idea

Model Checker - IdeaModel Checker - Idea

P +s [ÁjÃ]=max

´

µP s;´ [Á^Ã]

P s;´ [Ã]

¶

{By deterministic and sHI Theorem}

P +s [ÁjÃ]=max

µP s;´1

[Á^Ã]P s;´1

[Ã];¢¢¢;

P s;´k[Á^Ã]

P s;´k[Ã]

¶

where f ´1;´2; : : : ;´kg is the set of all deterministic and sHI schedulers

What we actually computef (s;Á;Ã) =

©(P s;´1

[Á^Ã];P s;´1[Ã]);¢¢¢;(P s;´k

[Á^Ã];P s;´k[Ã])

ª

P +s [ÁjÃ]=max

³ nabj (a;b) 2 f (s;Á;Ã) ^b6= 0

o[ f0g

´



Model Checker - ExampleModel Checker - Example

Optimizations Reusing information

Ussing pCTL algorithms after reaching the stopping condition

Example¡Case P+

s [Á1UÁ2jÃ1UÃ2]¢

f (s;Á1UÁ2;Ã1UÃ2) = f (P +s [Ã1UÃ2];P

+s [Ã1UÃ2])g if s j= Á2

f (s;Á1UÁ2;Ã1UÃ2) = f (P +s [Á1UÁ2];1)g if s j= : Á2^Ã2

f (s;Á1UÁ2;Ã1UÃ2) = f (0;P ¡s [Ã1UÃ2])g if s j= : Á1^: Á2^: Ã2

f (s;Á1UÁ2;Ã1UÃ2) = f (0;0)g if s j= Á1^: Á2^: Ã1^: Ã2f (s;Á1UÁ2;Ã1UÃ2) =S¼2¿(s)

³ Lt2succ(s)¼(t) ¯ f (t;Á1UÁ2;Ã1UÃ2)

´if s j= Á1^: Á2^Ã1^: Ã2



OverviewOverview









Why?

CounterexamplesCounterexamples

Counterexamples

Model'j=

Counterexamples for cpCTLA counterexample for P · a[ÁjÃ] is a pair (¢ 1;¢ 2) of measurable sets

of paths satisfying ¢ 1 µ ¢ Á^Ã , ¢ 2 µ ¢ : Ã , and a<P ´ (¢ 1)

1¡ P ´ (¢ 2), for some

scheduler ´.

s j= P · a[ÁjÃ] , for all ´ P s ;´ [Á^Ã]P s ;´ [Ã]

· a

Lemma

where¢ 1 µ ¢ Á^Ã , f ! 2 j ! j= Á^Ãgand ¢ 2 µ ¢ : Ã , f ! 2 j ! j= : Ãg

P ´ [Á^Ã]P ´ [Ã]

> a P ´ (¢ 1)1¡ P ´ (¢ 2)

> a



OverviewOverview

Motivation Backgorund








Future WorkFuture Work

Implement our Algorithms in a probabilistic model checker.

Investigate features of cpCTL (expressivness –bisimulation issues).

Improve complexity.

Extend cpCTL to cpCTL*.

More research about counterexamples in cpCTL and cpCTL*.



Thanks for your attention!Thanks for your attention!



Why Deterministic Schedulers?Why Deterministic Schedulers?

Lema: Let v1;v2 2 [0;1 ) and w1;w2 2 (0;1 ). Then the functionf : R ! R de ned by f (®) , ®v1+(1¡ ®)v2

®w1+(1¡ ®)w2ismonotonous.

Á ÁÃ Ã

s0

s1 s21¡ ®®

P s0[ÁjÃ]=

®P s 1[Á^Ã]+(1¡ ®)P s 2

[Á^Ã]®P s1

[Ã]+(1¡ ®)P s 2[Ã]

Conditional Probabilities over Probabilistic and Nondeterministic Systems M. E. Andrés and P. van Rossum Radboud Universiteit Nijmegen, The Netherlands.

Documents

s j s j

b j p p s

max p s

d i s t r s p s

s b s p i s t h epro

cpctl example s

t woeven t s p b

d i s t r s mdp