Bisimulation and Metrics for Labelled MarkovProcesses
Prakash Panangaden1
1School of Computer ScienceMcGill University
13th June 2012, University of Aalborg
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 1 / 46
Outline
1 Introduction
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 2 / 46
Outline
1 Introduction
2 Labelled transition systems
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 2 / 46
Outline
1 Introduction
2 Labelled transition systems
3 Ordinary bisimulation
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 2 / 46
Outline
1 Introduction
2 Labelled transition systems
3 Ordinary bisimulation
4 Discrete probabilistic transition systems
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 2 / 46
Outline
1 Introduction
2 Labelled transition systems
3 Ordinary bisimulation
4 Discrete probabilistic transition systems
5 Labelled Markov processes
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 2 / 46
Outline
1 Introduction
2 Labelled transition systems
3 Ordinary bisimulation
4 Discrete probabilistic transition systems
5 Labelled Markov processes
6 Probabilistic bisimulation
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 2 / 46
Outline
1 Introduction
2 Labelled transition systems
3 Ordinary bisimulation
4 Discrete probabilistic transition systems
5 Labelled Markov processes
6 Probabilistic bisimulation
7 * Proof of the logical characterization theorem
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 2 / 46
Outline
1 Introduction
2 Labelled transition systems
3 Ordinary bisimulation
4 Discrete probabilistic transition systems
5 Labelled Markov processes
6 Probabilistic bisimulation
7 * Proof of the logical characterization theorem
8 Metrics
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 2 / 46
Outline
1 Introduction
2 Labelled transition systems
3 Ordinary bisimulation
4 Discrete probabilistic transition systems
5 Labelled Markov processes
6 Probabilistic bisimulation
7 * Proof of the logical characterization theorem
8 Metrics
9 Continuous-state systems
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 2 / 46
Outline
1 Introduction
2 Labelled transition systems
3 Ordinary bisimulation
4 Discrete probabilistic transition systems
5 Labelled Markov processes
6 Probabilistic bisimulation
7 * Proof of the logical characterization theorem
8 Metrics
9 Continuous-state systems
10 Conclusions
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 2 / 46
Introduction
Summary of Results
Probabilistic bisimulation can be defined for continuousstate-space systems. [LICS97]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 3 / 46
Introduction
Summary of Results
Probabilistic bisimulation can be defined for continuousstate-space systems. [LICS97]
Logical characterization. [LICS98,Info and Comp 2002]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 3 / 46
Introduction
Summary of Results
Probabilistic bisimulation can be defined for continuousstate-space systems. [LICS97]
Logical characterization. [LICS98,Info and Comp 2002]
Metric analogue of bisimulation. [CONCUR99, TCS2004]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 3 / 46
Introduction
Summary of Results
Probabilistic bisimulation can be defined for continuousstate-space systems. [LICS97]
Logical characterization. [LICS98,Info and Comp 2002]
Metric analogue of bisimulation. [CONCUR99, TCS2004]
Approximation of LMPs. [LICS00,Info and Comp 2003, CONCUR2005]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 3 / 46
Introduction
Summary of Results
Probabilistic bisimulation can be defined for continuousstate-space systems. [LICS97]
Logical characterization. [LICS98,Info and Comp 2002]
Metric analogue of bisimulation. [CONCUR99, TCS2004]
Approximation of LMPs. [LICS00,Info and Comp 2003, CONCUR2005]
Weak bisimulation. [LICS02,CONCUR02]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 3 / 46
Introduction
Summary of Results
Probabilistic bisimulation can be defined for continuousstate-space systems. [LICS97]
Logical characterization. [LICS98,Info and Comp 2002]
Metric analogue of bisimulation. [CONCUR99, TCS2004]
Approximation of LMPs. [LICS00,Info and Comp 2003, CONCUR2005]
Weak bisimulation. [LICS02,CONCUR02]
Real time. [QEST 2004, JLAP 2003,LMCS 2006]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 3 / 46
Introduction
Summary of Results
Probabilistic bisimulation can be defined for continuousstate-space systems. [LICS97]
Logical characterization. [LICS98,Info and Comp 2002]
Metric analogue of bisimulation. [CONCUR99, TCS2004]
Approximation of LMPs. [LICS00,Info and Comp 2003, CONCUR2005]
Weak bisimulation. [LICS02,CONCUR02]
Real time. [QEST 2004, JLAP 2003,LMCS 2006]
Event Bisimulation [I and C, 2006]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 3 / 46
Introduction
Summary of Results
Probabilistic bisimulation can be defined for continuousstate-space systems. [LICS97]
Logical characterization. [LICS98,Info and Comp 2002]
Metric analogue of bisimulation. [CONCUR99, TCS2004]
Approximation of LMPs. [LICS00,Info and Comp 2003, CONCUR2005]
Weak bisimulation. [LICS02,CONCUR02]
Real time. [QEST 2004, JLAP 2003,LMCS 2006]
Event Bisimulation [I and C, 2006]
Abstract MPs [ICALP 2009]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 3 / 46
Introduction
Collaborators
Josée Desharnais
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 4 / 46
Introduction
Collaborators
Josée DesharnaisGheorghe Comanici
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 4 / 46
Introduction
Collaborators
Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-Côté
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 4 / 46
Introduction
Collaborators
Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe Chaput
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 4 / 46
Introduction
Collaborators
Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent Danos
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 4 / 46
Introduction
Collaborators
Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas Edalat
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 4 / 46
Introduction
Collaborators
Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm Ferns
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 4 / 46
Introduction
Collaborators
Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet Gupta
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 4 / 46
Introduction
Collaborators
Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha Jagadeesan
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 4 / 46
Introduction
Collaborators
Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim Larsen
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 4 / 46
Introduction
Collaborators
Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim LarsenFrancois Laviolette
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 4 / 46
Introduction
Collaborators
Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim LarsenFrancois LavioletteRadu Mardare
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 4 / 46
Introduction
Collaborators
Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim LarsenFrancois LavioletteRadu MardareGordon Plotkin
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 4 / 46
Introduction
Collaborators
Josée DesharnaisGheorghe ComaniciAlexandre Bouchard-CôtéPhilippe ChaputVincent DanosAbbas EdalatNorm FernsVineet GuptaRadha JagadeesanKim LarsenFrancois LavioletteRadu MardareGordon PlotkinDoina Precup
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 4 / 46
Labelled transition systems
The definition
A set of states S,
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 5 / 46
Labelled transition systems
The definition
A set of states S,
a set of labels or actions, L or A and
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 5 / 46
Labelled transition systems
The definition
A set of states S,
a set of labels or actions, L or A and
a transition relation ⊆ S ×A× S, usually written
→a⊆ S × S.
The transitions could be indeterminate (nondeterministic).
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 5 / 46
Labelled transition systems
The definition
A set of states S,
a set of labels or actions, L or A and
a transition relation ⊆ S ×A× S, usually written
→a⊆ S × S.
The transitions could be indeterminate (nondeterministic).
We write s a−−→ s′ for (s, s′) ∈→a.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 5 / 46
Ordinary bisimulation
Bisimulation
s and t are states of a labelled transition system. We say s is bisimilarto t – written s ∼ t – if
s a−−→ s′ ⇒ ∃t ′ such that t a
−−→ t ′ and s′ ∼ t ′
andt a−−→ t ′ ⇒ ∃s′ such that s a
−−→ s′ and s′ ∼ t ′.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 6 / 46
Ordinary bisimulation
Bisimulation relations
Define a (note the indefinite article) bisimulation relation R to bean equivalence relation on S such that
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 7 / 46
Ordinary bisimulation
Bisimulation relations
Define a (note the indefinite article) bisimulation relation R to bean equivalence relation on S such that
sRt means ∀a, s a−−→ s′ ⇒ ∃t ′, t a
−−→ t ′ with s′Rt ′
and vice versa.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 7 / 46
Ordinary bisimulation
Bisimulation relations
Define a (note the indefinite article) bisimulation relation R to bean equivalence relation on S such that
sRt means ∀a, s a−−→ s′ ⇒ ∃t ′, t a
−−→ t ′ with s′Rt ′
and vice versa.
We define s ∼ t if there is some bisimulation relation R with sRt .
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 7 / 46
Ordinary bisimulation
Bisimulation relations
Define a (note the indefinite article) bisimulation relation R to bean equivalence relation on S such that
sRt means ∀a, s a−−→ s′ ⇒ ∃t ′, t a
−−→ t ′ with s′Rt ′
and vice versa.
We define s ∼ t if there is some bisimulation relation R with sRt .
This is the version that is used most often.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 7 / 46
Ordinary bisimulation
An example
s0
a
a
a
;;
;;;;
;
s1
b
s2
b
c
;;
;;;;
;s3
c
s4 s5
t0
a
t2b
c
88
8888
8
t4 t5P1 P2
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 8 / 46
Ordinary bisimulation
An example
s0
a
a
a
;;
;;;;
;
s1
b
s2
b
c
;;
;;;;
;s3
c
s4 s5
t0
a
t2b
c
88
8888
8
t4 t5P1 P2
Here s0 and t0 are not bisimilar.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 8 / 46
Ordinary bisimulation
An example
s0
a
a
a
;;
;;;;
;
s1
b
s2
b
c
;;
;;;;
;s3
c
s4 s5
t0
a
t2b
c
88
8888
8
t4 t5P1 P2
Here s0 and t0 are not bisimilar.
However s0 and t0 can simulate each other!
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 8 / 46
Ordinary bisimulation
How do we know that two processes are not bisimilar?
Define a logic as follows:
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 9 / 46
Ordinary bisimulation
How do we know that two processes are not bisimilar?
Define a logic as follows:
φ ::== T|¬φ|φ1 ∧ φ2|〈a〉φ
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 9 / 46
Ordinary bisimulation
How do we know that two processes are not bisimilar?
Define a logic as follows:
φ ::== T|¬φ|φ1 ∧ φ2|〈a〉φ
s |= 〈a〉φ means that s a−−→ s′ and t |= φ.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 9 / 46
Ordinary bisimulation
How do we know that two processes are not bisimilar?
Define a logic as follows:
φ ::== T|¬φ|φ1 ∧ φ2|〈a〉φ
s |= 〈a〉φ means that s a−−→ s′ and t |= φ.
We can define a dual to 〈〉 (written []) by using negation.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 9 / 46
Ordinary bisimulation
How do we know that two processes are not bisimilar?
Define a logic as follows:
φ ::== T|¬φ|φ1 ∧ φ2|〈a〉φ
s |= 〈a〉φ means that s a−−→ s′ and t |= φ.
We can define a dual to 〈〉 (written []) by using negation.
s |= [a]φ means that if s can do an a the resulting state mustsatisfy φ.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 9 / 46
Ordinary bisimulation
Examples of HM Logic
T is satisfied by any process, F is not satisfied by any process.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 10 / 46
Ordinary bisimulation
Examples of HM Logic
T is satisfied by any process, F is not satisfied by any process.
s |= 〈a〉T means s can do an a action.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 10 / 46
Ordinary bisimulation
Examples of HM Logic
T is satisfied by any process, F is not satisfied by any process.
s |= 〈a〉T means s can do an a action.
s |= ¬〈a〉T or s |= [a]F means s cannot do an a action.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 10 / 46
Ordinary bisimulation
Examples of HM Logic
T is satisfied by any process, F is not satisfied by any process.
s |= 〈a〉T means s can do an a action.
s |= ¬〈a〉T or s |= [a]F means s cannot do an a action.
s |= 〈a〉(〈b〉T ) means that s can do an a and then do a b.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 10 / 46
Ordinary bisimulation
The logical characterization theorem
Two processes are bisimilar if and only if they satisfy the sameformulas of HM logic.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 11 / 46
Ordinary bisimulation
The logical characterization theorem
Two processes are bisimilar if and only if they satisfy the sameformulas of HM logic.
Basic assumption: the processes are finitely-branching (otherwiseyou need infinitary conjunctions).
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 11 / 46
Ordinary bisimulation
The logical characterization theorem
Two processes are bisimilar if and only if they satisfy the sameformulas of HM logic.
Basic assumption: the processes are finitely-branching (otherwiseyou need infinitary conjunctions).
To show that two processes are not bisimilar find a formula onwhich they disagree.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 11 / 46
Ordinary bisimulation
The role of negation
Consider the processes below:
s0
a
a
;;
;;;;
;
s1 s2
b
s3
t0
a
t2
b
t3
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 12 / 46
Ordinary bisimulation
The role of negation
Consider the processes below:
s0
a
a
;;
;;;;
;
s1 s2
b
s3
t0
a
t2
b
t3
s0 |= 〈a〉¬〈b〉T but t0 does not.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 12 / 46
Ordinary bisimulation
The role of negation
Consider the processes below:
s0
a
a
;;
;;;;
;
s1 s2
b
s3
t0
a
t2
b
t3
s0 |= 〈a〉¬〈b〉T but t0 does not.
s0 and t0 agree on all formulas without negation.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 12 / 46
Ordinary bisimulation
The role of negation
Consider the processes below:
s0
a
a
;;
;;;;
;
s1 s2
b
s3
t0
a
t2
b
t3
s0 |= 〈a〉¬〈b〉T but t0 does not.
s0 and t0 agree on all formulas without negation.
Note that [a] has an implicit negation.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 12 / 46
Discrete probabilistic transition systems
Discrete probabilistic transition systems
Just like a labelled transition system with probabilities associatedwith the transitions.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 13 / 46
Discrete probabilistic transition systems
Discrete probabilistic transition systems
Just like a labelled transition system with probabilities associatedwith the transitions.
(S,L,∀a ∈ L Ta : S × S −→ [0,1])
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 13 / 46
Discrete probabilistic transition systems
Discrete probabilistic transition systems
Just like a labelled transition system with probabilities associatedwith the transitions.
(S,L,∀a ∈ L Ta : S × S −→ [0,1])
The model is reactive: All probabilistic data is internal - noprobabilities associated with environment behaviour.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 13 / 46
Discrete probabilistic transition systems
Bisimulation for PTS: Larsen and Skou
Consider
t0a[ 1
3 ]
a[ 2
3 ]
88
8888
8
t1 t2
b[1]
t3
s0a[ 1
3 ]
a[ 13 ]
a[ 13 ]
;;
;;;;
;
s1 s2
b[1]
s3
b[1]
s4
P1 P2
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 14 / 46
Discrete probabilistic transition systems
Bisimulation for PTS: Larsen and Skou
Consider
t0a[ 1
3 ]
a[ 2
3 ]
88
8888
8
t1 t2
b[1]
t3
s0a[ 1
3 ]
a[ 13 ]
a[ 13 ]
;;
;;;;
;
s1 s2
b[1]
s3
b[1]
s4
P1 P2
Should s0 and t0 be bisimilar?
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 14 / 46
Discrete probabilistic transition systems
Bisimulation for PTS: Larsen and Skou
Consider
t0a[ 1
3 ]
a[ 2
3 ]
88
8888
8
t1 t2
b[1]
t3
s0a[ 1
3 ]
a[ 13 ]
a[ 13 ]
;;
;;;;
;
s1 s2
b[1]
s3
b[1]
s4
P1 P2
Should s0 and t0 be bisimilar?
Yes, but we need to add the probabilities.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 14 / 46
Discrete probabilistic transition systems
The Official Definition
Let S = (S,L,Ta) be a PTS. An equivalence relation R on S is abisimulation if whenever sRs′, with s, s′ ∈ S, we have that for alla ∈ A and every R-equivalence class, A, Ta(s,A) = Ta(s′,A).
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 15 / 46
Discrete probabilistic transition systems
The Official Definition
Let S = (S,L,Ta) be a PTS. An equivalence relation R on S is abisimulation if whenever sRs′, with s, s′ ∈ S, we have that for alla ∈ A and every R-equivalence class, A, Ta(s,A) = Ta(s′,A).
The notation Ta(s,A) means “the probability of starting from s andjumping to a state in the set A.”
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 15 / 46
Discrete probabilistic transition systems
The Official Definition
Let S = (S,L,Ta) be a PTS. An equivalence relation R on S is abisimulation if whenever sRs′, with s, s′ ∈ S, we have that for alla ∈ A and every R-equivalence class, A, Ta(s,A) = Ta(s′,A).
The notation Ta(s,A) means “the probability of starting from s andjumping to a state in the set A.”
Two states are bisimilar if there is some bisimulation relation Rrelating them.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 15 / 46
Labelled Markov processes
What are labelled Markov processes?
Reactive systems: Larsen and Skou
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 16 / 46
Labelled Markov processes
What are labelled Markov processes?
Reactive systems: Larsen and Skou
Labelled Markov processes are probabilistic versions of labelledtransition systems. Labelled transition systems where the finalstate is governed by a probability distribution - no otherindeterminacy.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 16 / 46
Labelled Markov processes
What are labelled Markov processes?
Reactive systems: Larsen and Skou
Labelled Markov processes are probabilistic versions of labelledtransition systems. Labelled transition systems where the finalstate is governed by a probability distribution - no otherindeterminacy.
All probabilistic data is internal - no probabilities associated withenvironment behaviour.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 16 / 46
Labelled Markov processes
What are labelled Markov processes?
Reactive systems: Larsen and Skou
Labelled Markov processes are probabilistic versions of labelledtransition systems. Labelled transition systems where the finalstate is governed by a probability distribution - no otherindeterminacy.
All probabilistic data is internal - no probabilities associated withenvironment behaviour.
We observe the interactions - not the internal states.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 16 / 46
Labelled Markov processes
What are labelled Markov processes?
Reactive systems: Larsen and Skou
Labelled Markov processes are probabilistic versions of labelledtransition systems. Labelled transition systems where the finalstate is governed by a probability distribution - no otherindeterminacy.
All probabilistic data is internal - no probabilities associated withenvironment behaviour.
We observe the interactions - not the internal states.
In general, the state space of a labelled Markov process maybe a continuum.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 16 / 46
Labelled Markov processes
Motivation
Model and reason about systems with continuous state spaces orcontinuous time evolution or both.
hybrid control systems; e.g. flight management systems.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 17 / 46
Labelled Markov processes
Motivation
Model and reason about systems with continuous state spaces orcontinuous time evolution or both.
hybrid control systems; e.g. flight management systems.
telecommunication systems with spatial variation; e.g. cell phones
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 17 / 46
Labelled Markov processes
Motivation
Model and reason about systems with continuous state spaces orcontinuous time evolution or both.
hybrid control systems; e.g. flight management systems.
telecommunication systems with spatial variation; e.g. cell phones
performance modelling,
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 17 / 46
Labelled Markov processes
Motivation
Model and reason about systems with continuous state spaces orcontinuous time evolution or both.
hybrid control systems; e.g. flight management systems.
telecommunication systems with spatial variation; e.g. cell phones
performance modelling,
continuous time systems,
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 17 / 46
Labelled Markov processes
Motivation
Model and reason about systems with continuous state spaces orcontinuous time evolution or both.
hybrid control systems; e.g. flight management systems.
telecommunication systems with spatial variation; e.g. cell phones
performance modelling,
continuous time systems,
probabilistic process algebra with recursion.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 17 / 46
Labelled Markov processes
The Need for Measure Theory
Basic fact: There are subsets of R for which no sensible notion ofsize can be defined.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 18 / 46
Labelled Markov processes
The Need for Measure Theory
Basic fact: There are subsets of R for which no sensible notion ofsize can be defined.
More precisely, there is no translation-invariant measure definedon all the subsets of the reals.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 18 / 46
Labelled Markov processes
The Need for Measure Theory
Basic fact: There are subsets of R for which no sensible notion ofsize can be defined.
More precisely, there is no translation-invariant measure definedon all the subsets of the reals.
Actually there is if you only require finite additivity.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 18 / 46
Labelled Markov processes
Stochastic Kernels
A stochastic kernel (Markov kernel) is a function h : S × Σ−→ [0,1] with (a) h(s, ·) : Σ −→ [0,1] a (sub)probability measureand (b) h(·,A) : X −→ [0,1] a measurable function.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 19 / 46
Labelled Markov processes
Stochastic Kernels
A stochastic kernel (Markov kernel) is a function h : S × Σ−→ [0,1] with (a) h(s, ·) : Σ −→ [0,1] a (sub)probability measureand (b) h(·,A) : X −→ [0,1] a measurable function.
Though apparantly asymmetric, these are the stochasticanalogues of binary relations
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 19 / 46
Labelled Markov processes
Stochastic Kernels
A stochastic kernel (Markov kernel) is a function h : S × Σ−→ [0,1] with (a) h(s, ·) : Σ −→ [0,1] a (sub)probability measureand (b) h(·,A) : X −→ [0,1] a measurable function.
Though apparantly asymmetric, these are the stochasticanalogues of binary relations
and the uncountable generalization of a matrix.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 19 / 46
Labelled Markov processes
Formal Definition of LMPs
An LMP is a tuple (S,Σ,L,∀α ∈ L.τα) where τα : S × Σ −→ [0,1] isa transition probability function such that
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 20 / 46
Labelled Markov processes
Formal Definition of LMPs
An LMP is a tuple (S,Σ,L,∀α ∈ L.τα) where τα : S × Σ −→ [0,1] isa transition probability function such that
∀s : S.λA : Σ.τα(s,A) is a subprobability measureand∀A : Σ.λs : S.τα(s,A) is a measurable function.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 20 / 46
Probabilistic bisimulation
Larsen-Skou Bisimulation
Let S = (S, i ,Σ, τ) be a labelled Markov process. An equivalencerelation R on S is a bisimulation if whenever sRs′, with s, s′ ∈ S,we have that for all a ∈ A and every R-closed measurable setA ∈ Σ, τa(s,A) = τa(s′,A).Two states are bisimilar if they are related by a bisimulationrelation.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 21 / 46
Probabilistic bisimulation
Larsen-Skou Bisimulation
Let S = (S, i ,Σ, τ) be a labelled Markov process. An equivalencerelation R on S is a bisimulation if whenever sRs′, with s, s′ ∈ S,we have that for all a ∈ A and every R-closed measurable setA ∈ Σ, τa(s,A) = τa(s′,A).Two states are bisimilar if they are related by a bisimulationrelation.
Can be extended to bisimulation between two different LMPs.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 21 / 46
Probabilistic bisimulation
Logical Characterization
L ::== T|φ1 ∧ φ2|〈a〉qφ
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 22 / 46
Probabilistic bisimulation
Logical Characterization
L ::== T|φ1 ∧ φ2|〈a〉qφ
We say s |= 〈a〉qφ iff
∃A ∈ Σ.(∀s′ ∈ A.s′ |= φ) ∧ (τa(s,A) > q).
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 22 / 46
Probabilistic bisimulation
Logical Characterization
L ::== T|φ1 ∧ φ2|〈a〉qφ
We say s |= 〈a〉qφ iff
∃A ∈ Σ.(∀s′ ∈ A.s′ |= φ) ∧ (τa(s,A) > q).
Two systems are bisimilar iff they obey the same formulas of L.[DEP 1998 LICS, I and C 2002]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 22 / 46
Probabilistic bisimulation
That cannot be right?
s0
a
a
;;
;;;;
;
s1 s2
b
s3
t0
a
t1
b
t2
Two processes that cannot be distinguished without negation.The formula that distinguishes them is 〈a〉(¬〈b〉⊤).
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 23 / 46
Probabilistic bisimulation
But it is!
s0a[p]
a[q]
;;
;;;;
;
s1 s2
b
s3
t0
a[r ]
t1
b
t2
We add probabilities to the transitions.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 24 / 46
Probabilistic bisimulation
But it is!
s0a[p]
a[q]
;;
;;;;
;
s1 s2
b
s3
t0
a[r ]
t1
b
t2
We add probabilities to the transitions.
If p + q < r or p + q > r we can easily distinguish them.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 24 / 46
Probabilistic bisimulation
But it is!
s0a[p]
a[q]
;;
;;;;
;
s1 s2
b
s3
t0
a[r ]
t1
b
t2
We add probabilities to the transitions.
If p + q < r or p + q > r we can easily distinguish them.
If p + q = r and p > 0 then q < r so 〈a〉r 〈b〉1⊤ distinguishes them.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 24 / 46
* Proof of the logical characterization theorem
Digression on Analytic Spaces
An analytic set A is the image of a Polish space X (or a Borelsubset of X ) under a continuous (or measurable) function f : X−→ Y , where Y is Polish. If (S,Σ) is a measurable space where Sis an analytic set in some ambient topological space and Σ is theBorel σ-algebra on S.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 25 / 46
* Proof of the logical characterization theorem
Digression on Analytic Spaces
An analytic set A is the image of a Polish space X (or a Borelsubset of X ) under a continuous (or measurable) function f : X−→ Y , where Y is Polish. If (S,Σ) is a measurable space where Sis an analytic set in some ambient topological space and Σ is theBorel σ-algebra on S.
Analytic sets do not form a σ-algebra but they are in thecompletion of the Borel algebra under any measure. [Universallymeasurable.]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 25 / 46
* Proof of the logical characterization theorem
Digression on Analytic Spaces
An analytic set A is the image of a Polish space X (or a Borelsubset of X ) under a continuous (or measurable) function f : X−→ Y , where Y is Polish. If (S,Σ) is a measurable space where Sis an analytic set in some ambient topological space and Σ is theBorel σ-algebra on S.
Analytic sets do not form a σ-algebra but they are in thecompletion of the Borel algebra under any measure. [Universallymeasurable.]
Regular conditional probability densities can be defined onanalytic spaces.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 25 / 46
* Proof of the logical characterization theorem
Amazing Facts about Analytic Spaces
Given A an analytic space and ∼ an equivalence relation suchthat there is a countable family of real-valued measurablefunctions fi : S −→ R such that
∀s, s′ ∈ S.s ∼ s′ ⇐⇒ ∀fi .fi(s) = fi(s′)
then the quotient space (Q,Ω) - where Q = S/ ∼ and Ω is thefinest σ-algebra making the canonical surjection q : S −→ Qmeasurable - is also analytic.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 26 / 46
* Proof of the logical characterization theorem
Amazing Facts about Analytic Spaces
Given A an analytic space and ∼ an equivalence relation suchthat there is a countable family of real-valued measurablefunctions fi : S −→ R such that
∀s, s′ ∈ S.s ∼ s′ ⇐⇒ ∀fi .fi(s) = fi(s′)
then the quotient space (Q,Ω) - where Q = S/ ∼ and Ω is thefinest σ-algebra making the canonical surjection q : S −→ Qmeasurable - is also analytic.
If an analytic space (S,Σ) has a sub-σ-algebra Σ0 of Σ whichseparates points and is countably generated then Σ0 is Σ! TheUnique Structure Theorem (UST).
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 26 / 46
* Proof of the logical characterization theorem
The Quotient
Given (S,Σ, τa) an LMP, we define s ≃ s′ if s and s′ obey exactlythe same formulas of L0.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 27 / 46
* Proof of the logical characterization theorem
The Quotient
Given (S,Σ, τa) an LMP, we define s ≃ s′ if s and s′ obey exactlythe same formulas of L0.
The functions I[[φ]] : S −→ R defined by I[[φ]](s) = 1 if s |= φ and 0otherwise are a countable family of measurable functions suchthat s ≃ s′ if and only if all the functions agree on s and s′. Thusthe quotient space (Q,Ω) is analytic.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 27 / 46
* Proof of the logical characterization theorem
The Quotient
Given (S,Σ, τa) an LMP, we define s ≃ s′ if s and s′ obey exactlythe same formulas of L0.
The functions I[[φ]] : S −→ R defined by I[[φ]](s) = 1 if s |= φ and 0otherwise are a countable family of measurable functions suchthat s ≃ s′ if and only if all the functions agree on s and s′. Thusthe quotient space (Q,Ω) is analytic.
We define an LMP (Q,Ω, ρa) where ρa(t ,U) := τa(s,q−1(U));s ∈ q−1(t).
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 27 / 46
* Proof of the logical characterization theorem
ρ is well defined - I
Easy to check that q−1(q([[φ]])) = [[φ]]:s ∈ q−1(q([[φ]])) implies that q(s) ∈ q([[φ]]), i.e. ∃s′ ∈ [[φ]].s ≃ s′, so s |= φ so s ∈ [[φ]].
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 28 / 46
* Proof of the logical characterization theorem
ρ is well defined - I
Easy to check that q−1(q([[φ]])) = [[φ]]:s ∈ q−1(q([[φ]])) implies that q(s) ∈ q([[φ]]), i.e. ∃s′ ∈ [[φ]].s ≃ s′, so s |= φ so s ∈ [[φ]].
Thus q([[φ]]) is measurable.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 28 / 46
* Proof of the logical characterization theorem
ρ is well defined - I
Easy to check that q−1(q([[φ]])) = [[φ]]:s ∈ q−1(q([[φ]])) implies that q(s) ∈ q([[φ]]), i.e. ∃s′ ∈ [[φ]].s ≃ s′, so s |= φ so s ∈ [[φ]].
Thus q([[φ]]) is measurable.
Thus the σ-algebra generated -say, Λ - by q([[φ]]) is asub-σ-algebra of Ω.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 28 / 46
* Proof of the logical characterization theorem
ρ is well defined - I
Easy to check that q−1(q([[φ]])) = [[φ]]:s ∈ q−1(q([[φ]])) implies that q(s) ∈ q([[φ]]), i.e. ∃s′ ∈ [[φ]].s ≃ s′, so s |= φ so s ∈ [[φ]].
Thus q([[φ]]) is measurable.
Thus the σ-algebra generated -say, Λ - by q([[φ]]) is asub-σ-algebra of Ω.
Λ is countably generated and separates points so by UST it is Ω.Thus q([[φ]]) generates Ω.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 28 / 46
* Proof of the logical characterization theorem
ρ is well defined - II
The collection q([[φ]]) is a π-system (because L0 has conjunction)and it generates Ω; thus if we can show that two measures agreeon these sets they agree on all of Ω.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 29 / 46
* Proof of the logical characterization theorem
ρ is well defined - II
The collection q([[φ]]) is a π-system (because L0 has conjunction)and it generates Ω; thus if we can show that two measures agreeon these sets they agree on all of Ω.
If q(s) = q(s′) = t then τa(s, [[φ]]) = τa(s′, [[φ]]) (simpleinterpolation).
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 29 / 46
* Proof of the logical characterization theorem
ρ is well defined - II
The collection q([[φ]]) is a π-system (because L0 has conjunction)and it generates Ω; thus if we can show that two measures agreeon these sets they agree on all of Ω.
If q(s) = q(s′) = t then τa(s, [[φ]]) = τa(s′, [[φ]]) (simpleinterpolation).
Thus τa(s,q−1(q([[φ]]))) = τa(s′,q−1(q([[φ]]))) and hence ρ is welldefined. We have ρa(q(s),B) = τa(s,q−1(B)).
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 29 / 46
* Proof of the logical characterization theorem
Finishing the Argument
Let X be any ≃-closed subset of S.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 30 / 46
* Proof of the logical characterization theorem
Finishing the Argument
Let X be any ≃-closed subset of S.
Then q−1(q(X )) = X and q(X ) ∈ Ω.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 30 / 46
* Proof of the logical characterization theorem
Finishing the Argument
Let X be any ≃-closed subset of S.
Then q−1(q(X )) = X and q(X ) ∈ Ω.
If s ≃ s′ then q(s) = q(s′) and
τa(s,X ) = τa(s,q−1(q(X ))) = ρa(q(s),q(X )) =
ρa(q(s′),q(X )) = τa(s′,q−1(q(X ))) = τa(s′,X ).
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 30 / 46
Metrics
A metric-based approximate viewpoint
Move from equality between processes to distances betweenprocesses (Jou and Smolka 1990).
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 31 / 46
Metrics
A metric-based approximate viewpoint
Move from equality between processes to distances betweenprocesses (Jou and Smolka 1990).
Formalize distance as a metric:
d(s, s) = 0,d(s, t) = d(t , s),d(s,u) ≤ d(s, t) + d(t ,u).
Quantitative analogue of an equivalence relation.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 31 / 46
Metrics
A metric-based approximate viewpoint
Move from equality between processes to distances betweenprocesses (Jou and Smolka 1990).
Formalize distance as a metric:
d(s, s) = 0,d(s, t) = d(t , s),d(s,u) ≤ d(s, t) + d(t ,u).
Quantitative analogue of an equivalence relation.
Quantitative measurement of the distinction between processes.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 31 / 46
Metrics
Criteria on Metrics
Soundness:d(s, t) = 0 ⇔ s, t are bisimilar
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 32 / 46
Metrics
Criteria on Metrics
Soundness:d(s, t) = 0 ⇔ s, t are bisimilar
Stability of distance under temporal evolution:“Nearby states stayclose forever.”
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 32 / 46
Metrics
Criteria on Metrics
Soundness:d(s, t) = 0 ⇔ s, t are bisimilar
Stability of distance under temporal evolution:“Nearby states stayclose forever.”
Metrics should be computable (efficiently?).
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 32 / 46
Metrics
Bisimulation Recalled
Let R be an equivalence relation. R is a bisimulation if: s R t if:
(s −→ P) ⇒ [t −→ Q,P =R Q]
(t −→ Q) ⇒ [s −→ P,P =R Q]
where P =R Q if
(∀R − closed E) P(E) = Q(E)
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 33 / 46
Metrics
A putative definition of a metric analogue ofbisimulation
m is a metric-bisimulation if: m(s, t) < ǫ ⇒:
s −→ P ⇒ t −→ Q, m(P,Q) < ǫ
t −→ Q ⇒ s −→ P, m(P,Q) < ǫ
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 34 / 46
Metrics
A putative definition of a metric analogue ofbisimulation
m is a metric-bisimulation if: m(s, t) < ǫ ⇒:
s −→ P ⇒ t −→ Q, m(P,Q) < ǫ
t −→ Q ⇒ s −→ P, m(P,Q) < ǫ
Problem: what is m(P,Q)? — Type mismatch!!
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 34 / 46
Metrics
A putative definition of a metric analogue ofbisimulation
m is a metric-bisimulation if: m(s, t) < ǫ ⇒:
s −→ P ⇒ t −→ Q, m(P,Q) < ǫ
t −→ Q ⇒ s −→ P, m(P,Q) < ǫ
Problem: what is m(P,Q)? — Type mismatch!!
Need a way to lift distances from states to a distances ondistributions of states.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 34 / 46
Metrics
A detour: Kantorovich metric
Metrics on probability measures on metric spaces.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 35 / 46
Metrics
A detour: Kantorovich metric
Metrics on probability measures on metric spaces.
M: 1-bounded pseudometrics on states.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 35 / 46
Metrics
A detour: Kantorovich metric
Metrics on probability measures on metric spaces.
M: 1-bounded pseudometrics on states.
d(µ, ν) = supf
|
∫
fdµ−
∫
fdν|, f 1-Lipschitz
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 35 / 46
Metrics
A detour: Kantorovich metric
Metrics on probability measures on metric spaces.
M: 1-bounded pseudometrics on states.
d(µ, ν) = supf
|
∫
fdµ−
∫
fdν|, f 1-Lipschitz
Arises in the solution of an LP problem: transshipment.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 35 / 46
Metrics
An LP version for Finite-State Spaces
When state space is finite: Let P,Q be probability distributions. Then:
m(P,Q) = max∑
i
(P(si)− Q(si))ai
subject to:∀i .0 ≤ ai ≤ 1∀i , j . ai − aj ≤ m(si , sj).
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 36 / 46
Metrics
The Dual Form
Dual form from Worrell and van Breugel:
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 37 / 46
Metrics
The Dual Form
Dual form from Worrell and van Breugel:
min∑
i ,j
lijm(si , sj ) +∑
i
xi +∑
j
yj
subject to:∀i .
∑
j lij + xi = P(si)
∀j .∑
i lij + yj = Q(sj )∀i , j . lij , xi , yj ≥ 0.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 37 / 46
Metrics
The Dual Form
Dual form from Worrell and van Breugel:
min∑
i ,j
lijm(si , sj ) +∑
i
xi +∑
j
yj
subject to:∀i .
∑
j lij + xi = P(si)
∀j .∑
i lij + yj = Q(sj )∀i , j . lij , xi , yj ≥ 0.
We prove many equations by using the primal form to show onedirection and the dual to show the other.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 37 / 46
Metrics
Example
Let m(s, t) = r < 1. Let δs(δt) be the probability measureconcentrated at s(t). Then,
m(δs, δt) = r
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 38 / 46
Metrics
Example
Let m(s, t) = r < 1. Let δs(δt) be the probability measureconcentrated at s(t). Then,
m(δs, δt) = r
Upper bound from dual: Choose lst = 1 all other lij = 0. Then
∑
ij
lijm(si , sj ) = m(s, t) = r .
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 38 / 46
Metrics
Example
Let m(s, t) = r < 1. Let δs(δt) be the probability measureconcentrated at s(t). Then,
m(δs, δt) = r
Upper bound from dual: Choose lst = 1 all other lij = 0. Then
∑
ij
lijm(si , sj ) = m(s, t) = r .
Lower bound from primal: Choose as = 0,at = r , all others tomatch the constraints. Then
∑
i
(δt (si)− δs(si ))ai = r .
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 38 / 46
Metrics
The Importance of the Example
We can isometrically embed the original space in the metric space ofdistributions.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 39 / 46
Metrics
Return from Detour
Summary of detour: Given a metric on states in a metric space, can liftto a metric on probability distributions on states.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 40 / 46
Metrics
Metric “Bisimulation”
m is a metric-bisimulation if: m(s, t) < ǫ ⇒:
s −→ P ⇒ t −→ Q, m(P,Q) < ǫ
t −→ Q ⇒ s −→ P, m(P,Q) < ǫ
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 41 / 46
Metrics
Metric “Bisimulation”
m is a metric-bisimulation if: m(s, t) < ǫ ⇒:
s −→ P ⇒ t −→ Q, m(P,Q) < ǫ
t −→ Q ⇒ s −→ P, m(P,Q) < ǫ
The required canonical metric on processes is the least such: ie.the distances are the least possible.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 41 / 46
Metrics
Metric “Bisimulation”
m is a metric-bisimulation if: m(s, t) < ǫ ⇒:
s −→ P ⇒ t −→ Q, m(P,Q) < ǫ
t −→ Q ⇒ s −→ P, m(P,Q) < ǫ
The required canonical metric on processes is the least such: ie.the distances are the least possible.
Thm: Canonical least metric exists. Usual fixed-point theoryarguments.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 41 / 46
Continuous-state systems
What about Continuous-State Systems?
Develop a real-valued “modal logic” based on the analogy:Program Logic Probabilistic LogicState s Distribution µFormula φ Random Variable fSatisfaction s |= φ
∫
f dµ
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 42 / 46
Continuous-state systems
What about Continuous-State Systems?
Develop a real-valued “modal logic” based on the analogy:Program Logic Probabilistic LogicState s Distribution µFormula φ Random Variable fSatisfaction s |= φ
∫
f dµ
Define a metric based on how closely the random variables agree.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 42 / 46
Continuous-state systems
What about Continuous-State Systems?
Develop a real-valued “modal logic” based on the analogy:Program Logic Probabilistic LogicState s Distribution µFormula φ Random Variable fSatisfaction s |= φ
∫
f dµ
Define a metric based on how closely the random variables agree.
We did this before the LP based techniques became available.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 42 / 46
Continuous-state systems
Real-valued Modal Logic
f ::= 1 | max(f , f ) | h f | 〈a〉.f
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 43 / 46
Continuous-state systems
Real-valued Modal Logic
f ::= 1 | max(f , f ) | h f | 〈a〉.f
1(s) = 1 Truemax(f1, f2)(s) = max(f1(s), f2(s)) Conjunctionh f (s) = h(f (s)) Lipschitz〈a〉.f (s) = γ
∫
s′∈S f (s′)τa(s, ds′) a-transition
where h 1-Lipschitz : [0,1] → [0,1] and γ ∈ (0,1].
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 43 / 46
Continuous-state systems
Real-valued Modal Logic
f ::= 1 | max(f , f ) | h f | 〈a〉.f
1(s) = 1 Truemax(f1, f2)(s) = max(f1(s), f2(s)) Conjunctionh f (s) = h(f (s)) Lipschitz〈a〉.f (s) = γ
∫
s′∈S f (s′)τa(s, ds′) a-transition
where h 1-Lipschitz : [0,1] → [0,1] and γ ∈ (0,1].
d(s, t) = supf |f (s)− f (t)|
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 43 / 46
Continuous-state systems
Real-valued Modal Logic
f ::= 1 | max(f , f ) | h f | 〈a〉.f
1(s) = 1 Truemax(f1, f2)(s) = max(f1(s), f2(s)) Conjunctionh f (s) = h(f (s)) Lipschitz〈a〉.f (s) = γ
∫
s′∈S f (s′)τa(s, ds′) a-transition
where h 1-Lipschitz : [0,1] → [0,1] and γ ∈ (0,1].
d(s, t) = supf |f (s)− f (t)|
Thm: d coincides with the canonical metric-bisimulation.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 43 / 46
Continuous-state systems
Finitary syntax for Real-valued modal logic
1(s) = 1 Truemax(f1, f2)(s) = max(f1(s), f2(s)) Conjunction(1 − f )(s) = 1 − f (s) Negation
⌊fq(s)⌋ =
q , f (s) ≥ qf (s) , f (s) < q
Cutoffs
〈a〉.f (s) = γ∫
s′∈S f (s′)τa(s, ds′) a-transition
q is a rational.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 44 / 46
Conclusions
Recent results and related work
Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 45 / 46
Conclusions
Recent results and related work
Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 45 / 46
Conclusions
Recent results and related work
Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 45 / 46
Conclusions
Recent results and related work
Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 45 / 46
Conclusions
Recent results and related work
Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]Domains of LMPs, universal LMP [DGJP]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 45 / 46
Conclusions
Recent results and related work
Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]Domains of LMPs, universal LMP [DGJP]Metrics [van Breugel, Worrel]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 45 / 46
Conclusions
Recent results and related work
Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]Domains of LMPs, universal LMP [DGJP]Metrics [van Breugel, Worrel]Weak bisimulation and metric analogue [DGJP]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 45 / 46
Conclusions
Recent results and related work
Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]Domains of LMPs, universal LMP [DGJP]Metrics [van Breugel, Worrel]Weak bisimulation and metric analogue [DGJP]Duality theory for LMPs. [Worrell et al.]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 45 / 46
Conclusions
Recent results and related work
Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]Domains of LMPs, universal LMP [DGJP]Metrics [van Breugel, Worrel]Weak bisimulation and metric analogue [DGJP]Duality theory for LMPs. [Worrell et al.]Decision procedure for metric without discount. [van Breugel et al.]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 45 / 46
Conclusions
Recent results and related work
Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]Domains of LMPs, universal LMP [DGJP]Metrics [van Breugel, Worrel]Weak bisimulation and metric analogue [DGJP]Duality theory for LMPs. [Worrell et al.]Decision procedure for metric without discount. [van Breugel et al.]Beautiful coinduction principle for stochastic processes due toKozen.
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 45 / 46
Conclusions
Recent results and related work
Markov Decision Processes with continuous state spaces. [Ferns,P., Precup]Sampling based techniques for approximating the metric.Logical characterization for simulation via domain theory, but weneed disjunction. [DGJP]Approximation theory for continuous space LMPs. [DGJP, BFPP,DD, DDP]Domains of LMPs, universal LMP [DGJP]Metrics [van Breugel, Worrel]Weak bisimulation and metric analogue [DGJP]Duality theory for LMPs. [Worrell et al.]Decision procedure for metric without discount. [van Breugel et al.]Beautiful coinduction principle for stochastic processes due toKozen.New completeness theorems: Cardelli, Larsen, Mardare [ICALP2011]Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 45 / 46
Conclusions
Work in Progress
New approach to approximation based on averaging [Chaput,Danos, P., Plotkin, ICALP 2009]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 46 / 46
Conclusions
Work in Progress
New approach to approximation based on averaging [Chaput,Danos, P., Plotkin, ICALP 2009]
Approximations of the logic and convergence properties [Larsen,Mardare, P.]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 46 / 46
Conclusions
Work in Progress
New approach to approximation based on averaging [Chaput,Danos, P., Plotkin, ICALP 2009]
Approximations of the logic and convergence properties [Larsen,Mardare, P.]
New Stone-type duality theory [Larsen, Mardare, P.]
Panangaden (McGill) Bisimulation and Metrics for Labelled Markov Processes 13th June 2012 46 / 46