university-logo INF5140: Specification and Verification of Parallel Systems Lecture 11 – The Great Debates Gerardo Schneider Department of Informatics University of Oslo INF5140, Spring 2011 Gerardo Schneider (Ifi, UiO) INF5140 - Lecture 11: Great Debates 12.05.2011 1 / 33
65
Embed
INF5140: Specification and Verification of Parallel Systems · Open vs Closed Systems Backward vs Forward Reachability Compositional vs Non-compositional Verification Deductive
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
university-logo
INF5140: Specification and Verification of ParallelSystems
1 The Great DebatesBranching vs Linear TimeSymbolic vs Explicit VerificationBreadth-First Search vs Depth-First SearchTarjan’s SCC Algorithms vs Spin’s Nested Depth-First SearchEvents vs StatesReal-time vs Timeless VerificationProbabilities vs PossibilitiesAsynchronous vs Synchronous SystemsInterleaving Semantics vs True ConcurrencyOpen vs Closed SystemsBackward vs Forward ReachabilityCompositional vs Non-compositional VerificationDeductive vs Algorithmic Verification
1 The Great DebatesBranching vs Linear TimeSymbolic vs Explicit VerificationBreadth-First Search vs Depth-First SearchTarjan’s SCC Algorithms vs Spin’s Nested Depth-First SearchEvents vs StatesReal-time vs Timeless VerificationProbabilities vs PossibilitiesAsynchronous vs Synchronous SystemsInterleaving Semantics vs True ConcurrencyOpen vs Closed SystemsBackward vs Forward ReachabilityCompositional vs Non-compositional VerificationDeductive vs Algorithmic Verification
Two main types of temporal logic used in model checkingCTL (Computational Tree Logic): mainly used in hardwareverificationLTL (Linear Temporal Logic): almost exclusively used for softwareverification
Main debates concernRelative expressivenessRelative complexity
Their expressiveness are incomparableCTL can express reset properties (LTL cannot)
From every state there exists at least one execution that can makethe system return to the initial state
LTL can express fairness (CTL cannot, although it can beembedded into the verification algorithms)
Every cyclic execution must (or may not) traverse specific types ofstates infinitely often
Two main types of temporal logic used in model checkingCTL (Computational Tree Logic): mainly used in hardwareverificationLTL (Linear Temporal Logic): almost exclusively used for softwareverification
Main debates concernRelative expressivenessRelative complexity
Their expressiveness are incomparableCTL can express reset properties (LTL cannot)
From every state there exists at least one execution that can makethe system return to the initial state
LTL can express fairness (CTL cannot, although it can beembedded into the verification algorithms)
Every cyclic execution must (or may not) traverse specific types ofstates infinitely often
Two main types of temporal logic used in model checkingCTL (Computational Tree Logic): mainly used in hardwareverificationLTL (Linear Temporal Logic): almost exclusively used for softwareverification
Main debates concernRelative expressivenessRelative complexity
Their expressiveness are incomparableCTL can express reset properties (LTL cannot)
From every state there exists at least one execution that can makethe system return to the initial state
LTL can express fairness (CTL cannot, although it can beembedded into the verification algorithms)
Every cyclic execution must (or may not) traverse specific types ofstates infinitely often
The worst-case behavior of LTL converters is rarePractical LTL formula usually have two or three temporal operatorsWhat affects complexity is not the number of potentially reachablesates but the effectively reachable statesLTL verification algorithms can more easily be implemented withon-the-fly strategies (no need to build the whole state space)
1 The Great DebatesBranching vs Linear TimeSymbolic vs Explicit VerificationBreadth-First Search vs Depth-First SearchTarjan’s SCC Algorithms vs Spin’s Nested Depth-First SearchEvents vs StatesReal-time vs Timeless VerificationProbabilities vs PossibilitiesAsynchronous vs Synchronous SystemsInterleaving Semantics vs True ConcurrencyOpen vs Closed SystemsBackward vs Forward ReachabilityCompositional vs Non-compositional VerificationDeductive vs Algorithmic Verification
In software verification partial order reduction strategies performvery well
Complex and highly correlated data structures, common in softwareverification problems, are not easily exploited with BDDsPartial order reduction techniques apply well to asynchronousprocess executionSynchronous, clocked, operations are not well suitable for partialorder reduction
Computing the optimal reduction is NP-completeThe memory used by an explicit state method is determined bythe number of states storedIn terms of memory consumption (number of bytes used), there isno big difference between symbolic and explicit verificationmethods
In software verification partial order reduction strategies performvery well
Complex and highly correlated data structures, common in softwareverification problems, are not easily exploited with BDDsPartial order reduction techniques apply well to asynchronousprocess executionSynchronous, clocked, operations are not well suitable for partialorder reduction
Computing the optimal reduction is NP-completeThe memory used by an explicit state method is determined bythe number of states storedIn terms of memory consumption (number of bytes used), there isno big difference between symbolic and explicit verificationmethods
1 The Great DebatesBranching vs Linear TimeSymbolic vs Explicit VerificationBreadth-First Search vs Depth-First SearchTarjan’s SCC Algorithms vs Spin’s Nested Depth-First SearchEvents vs StatesReal-time vs Timeless VerificationProbabilities vs PossibilitiesAsynchronous vs Synchronous SystemsInterleaving Semantics vs True ConcurrencyOpen vs Closed SystemsBackward vs Forward ReachabilityCompositional vs Non-compositional VerificationDeductive vs Algorithmic Verification
1 The Great DebatesBranching vs Linear TimeSymbolic vs Explicit VerificationBreadth-First Search vs Depth-First SearchTarjan’s SCC Algorithms vs Spin’s Nested Depth-First SearchEvents vs StatesReal-time vs Timeless VerificationProbabilities vs PossibilitiesAsynchronous vs Synchronous SystemsInterleaving Semantics vs True ConcurrencyOpen vs Closed SystemsBackward vs Forward ReachabilityCompositional vs Non-compositional VerificationDeductive vs Algorithmic Verification
The classical way to detect the presence of infinite accepting runsin a finite reachability graph is to use Tarjan’s DFS algorithm forconstructing all the SCC of the graphIn the worst-case, Tarjan’s algorithm visit every reachable statetwiceAdvantage: it detects all accepting runsSpin nested DFS algorithm does not detect all accepting runsThe search is set up s.t. an accepting run corresponds to acounterexample of a correctness claimThe worst-case time complexity is the same as Tarjan’s algorithmbut the memory overhead is lowerTarjan’s algorithm makes the implementation of strong fairnessconstraints easy
The classical way to detect the presence of infinite accepting runsin a finite reachability graph is to use Tarjan’s DFS algorithm forconstructing all the SCC of the graphIn the worst-case, Tarjan’s algorithm visit every reachable statetwiceAdvantage: it detects all accepting runsSpin nested DFS algorithm does not detect all accepting runsThe search is set up s.t. an accepting run corresponds to acounterexample of a correctness claimThe worst-case time complexity is the same as Tarjan’s algorithmbut the memory overhead is lowerTarjan’s algorithm makes the implementation of strong fairnessconstraints easy
The classical way to detect the presence of infinite accepting runsin a finite reachability graph is to use Tarjan’s DFS algorithm forconstructing all the SCC of the graphIn the worst-case, Tarjan’s algorithm visit every reachable statetwiceAdvantage: it detects all accepting runsSpin nested DFS algorithm does not detect all accepting runsThe search is set up s.t. an accepting run corresponds to acounterexample of a correctness claimThe worst-case time complexity is the same as Tarjan’s algorithmbut the memory overhead is lowerTarjan’s algorithm makes the implementation of strong fairnessconstraints easy
1 The Great DebatesBranching vs Linear TimeSymbolic vs Explicit VerificationBreadth-First Search vs Depth-First SearchTarjan’s SCC Algorithms vs Spin’s Nested Depth-First SearchEvents vs StatesReal-time vs Timeless VerificationProbabilities vs PossibilitiesAsynchronous vs Synchronous SystemsInterleaving Semantics vs True ConcurrencyOpen vs Closed SystemsBackward vs Forward ReachabilityCompositional vs Non-compositional VerificationDeductive vs Algorithmic Verification
Some properties are difficult to establish in a state-based settingand others in an event-basedFor example, for the correctness property:“Always within a finite amount of time after the transmission of a
message, the message will be received at its destination”
If the message is sent to a buffered channel, the state changes as aresult of the send and receive eventsIf send and receive are rendez-vous handshakes, recording theexecution in a way observable to Spin is a bit subtle
The ideal is to have a combined approach without adding anyverification penalty
See “State/Event-based Software Model Checking” by Chaki et al(based on labeled Kripke structures)
Some properties are difficult to establish in a state-based settingand others in an event-basedFor example, for the correctness property:“Always within a finite amount of time after the transmission of a
message, the message will be received at its destination”
If the message is sent to a buffered channel, the state changes as aresult of the send and receive eventsIf send and receive are rendez-vous handshakes, recording theexecution in a way observable to Spin is a bit subtle
The ideal is to have a combined approach without adding anyverification penalty
See “State/Event-based Software Model Checking” by Chaki et al(based on labeled Kripke structures)
1 The Great DebatesBranching vs Linear TimeSymbolic vs Explicit VerificationBreadth-First Search vs Depth-First SearchTarjan’s SCC Algorithms vs Spin’s Nested Depth-First SearchEvents vs StatesReal-time vs Timeless VerificationProbabilities vs PossibilitiesAsynchronous vs Synchronous SystemsInterleaving Semantics vs True ConcurrencyOpen vs Closed SystemsBackward vs Forward ReachabilityCompositional vs Non-compositional VerificationDeductive vs Algorithmic Verification
1 The Great DebatesBranching vs Linear TimeSymbolic vs Explicit VerificationBreadth-First Search vs Depth-First SearchTarjan’s SCC Algorithms vs Spin’s Nested Depth-First SearchEvents vs StatesReal-time vs Timeless VerificationProbabilities vs PossibilitiesAsynchronous vs Synchronous SystemsInterleaving Semantics vs True ConcurrencyOpen vs Closed SystemsBackward vs Forward ReachabilityCompositional vs Non-compositional VerificationDeductive vs Algorithmic Verification
Standard model checking algorithm may be modified in order toinclude probabilityIn most cases the inclusion of probabilities increase theverification complexity
In some cases (e.g. there are examples in real-time verification),adding probability simplifies the verification analysis
Promela/Spin does not have probabilities and only deals withpossible system behaviors
1 The Great DebatesBranching vs Linear TimeSymbolic vs Explicit VerificationBreadth-First Search vs Depth-First SearchTarjan’s SCC Algorithms vs Spin’s Nested Depth-First SearchEvents vs StatesReal-time vs Timeless VerificationProbabilities vs PossibilitiesAsynchronous vs Synchronous SystemsInterleaving Semantics vs True ConcurrencyOpen vs Closed SystemsBackward vs Forward ReachabilityCompositional vs Non-compositional VerificationDeductive vs Algorithmic Verification
Most hardware model checkers have a synchronous viewAll process actions are clock-drivenEvery process take a step at every clock tick
Asynchronous behavior can be modeled in the synchronoussettingDistributed systems are essentially asynchronousSpin is one of the few asynchronous model checkersAdvantages of asynchronous model checkers:
Greater verification efficiencyOnly w.r.t. to explicit state verification methods (not compared withsymbolic algorithms)
Main disadvantage of asynchronous model checkers:Difficult to model synchronous behavior
That’s why Spin is not good for hardware verification
Most hardware model checkers have a synchronous viewAll process actions are clock-drivenEvery process take a step at every clock tick
Asynchronous behavior can be modeled in the synchronoussettingDistributed systems are essentially asynchronousSpin is one of the few asynchronous model checkersAdvantages of asynchronous model checkers:
Greater verification efficiencyOnly w.r.t. to explicit state verification methods (not compared withsymbolic algorithms)
Main disadvantage of asynchronous model checkers:Difficult to model synchronous behavior
That’s why Spin is not good for hardware verification
Most hardware model checkers have a synchronous viewAll process actions are clock-drivenEvery process take a step at every clock tick
Asynchronous behavior can be modeled in the synchronoussettingDistributed systems are essentially asynchronousSpin is one of the few asynchronous model checkersAdvantages of asynchronous model checkers:
Greater verification efficiencyOnly w.r.t. to explicit state verification methods (not compared withsymbolic algorithms)
Main disadvantage of asynchronous model checkers:Difficult to model synchronous behavior
That’s why Spin is not good for hardware verification
Most hardware model checkers have a synchronous viewAll process actions are clock-drivenEvery process take a step at every clock tick
Asynchronous behavior can be modeled in the synchronoussettingDistributed systems are essentially asynchronousSpin is one of the few asynchronous model checkersAdvantages of asynchronous model checkers:
Greater verification efficiencyOnly w.r.t. to explicit state verification methods (not compared withsymbolic algorithms)
Main disadvantage of asynchronous model checkers:Difficult to model synchronous behavior
That’s why Spin is not good for hardware verification
1 The Great DebatesBranching vs Linear TimeSymbolic vs Explicit VerificationBreadth-First Search vs Depth-First SearchTarjan’s SCC Algorithms vs Spin’s Nested Depth-First SearchEvents vs StatesReal-time vs Timeless VerificationProbabilities vs PossibilitiesAsynchronous vs Synchronous SystemsInterleaving Semantics vs True ConcurrencyOpen vs Closed SystemsBackward vs Forward ReachabilityCompositional vs Non-compositional VerificationDeductive vs Algorithmic Verification
True concurrency semantics allows the simultaneous execution ofactions, in addition to the interleaving of actionsIn distributed systems, two asynchronous processes may executeactions at the same timeIn terms of verification, true concurrency implies the addition ofmore transitions to the verification modelIs true concurrency really needed?
1 If the process actions access either distinct data objects or none atall
Interleaving gives a correct interpretation: the simultaneous executionof actions is indistinguishable from any sequential interleaving
2 If two processes access shared dataBy representing the data objects at some level of granularity it isalways possible to accurately describe the possible behavior withinterleaving semantics
Interleaving semantics is often preferred (as the simplest sufficientmodel)
True concurrency semantics allows the simultaneous execution ofactions, in addition to the interleaving of actionsIn distributed systems, two asynchronous processes may executeactions at the same timeIn terms of verification, true concurrency implies the addition ofmore transitions to the verification modelIs true concurrency really needed?
1 If the process actions access either distinct data objects or none atall
Interleaving gives a correct interpretation: the simultaneous executionof actions is indistinguishable from any sequential interleaving
2 If two processes access shared dataBy representing the data objects at some level of granularity it isalways possible to accurately describe the possible behavior withinterleaving semantics
Interleaving semantics is often preferred (as the simplest sufficientmodel)
True concurrency semantics allows the simultaneous execution ofactions, in addition to the interleaving of actionsIn distributed systems, two asynchronous processes may executeactions at the same timeIn terms of verification, true concurrency implies the addition ofmore transitions to the verification modelIs true concurrency really needed?
1 If the process actions access either distinct data objects or none atall
Interleaving gives a correct interpretation: the simultaneous executionof actions is indistinguishable from any sequential interleaving
2 If two processes access shared dataBy representing the data objects at some level of granularity it isalways possible to accurately describe the possible behavior withinterleaving semantics
Interleaving semantics is often preferred (as the simplest sufficientmodel)
True concurrency semantics allows the simultaneous execution ofactions, in addition to the interleaving of actionsIn distributed systems, two asynchronous processes may executeactions at the same timeIn terms of verification, true concurrency implies the addition ofmore transitions to the verification modelIs true concurrency really needed?
1 If the process actions access either distinct data objects or none atall
Interleaving gives a correct interpretation: the simultaneous executionof actions is indistinguishable from any sequential interleaving
2 If two processes access shared dataBy representing the data objects at some level of granularity it isalways possible to accurately describe the possible behavior withinterleaving semantics
Interleaving semantics is often preferred (as the simplest sufficientmodel)
1 The Great DebatesBranching vs Linear TimeSymbolic vs Explicit VerificationBreadth-First Search vs Depth-First SearchTarjan’s SCC Algorithms vs Spin’s Nested Depth-First SearchEvents vs StatesReal-time vs Timeless VerificationProbabilities vs PossibilitiesAsynchronous vs Synchronous SystemsInterleaving Semantics vs True ConcurrencyOpen vs Closed SystemsBackward vs Forward ReachabilityCompositional vs Non-compositional VerificationDeductive vs Algorithmic Verification
Traditionally, model checking is based on two requirements on themodel: finiteness and closenessTo be closed, a system must include all the possible inputs andpossible interactions with the environmentOpen systems are more difficult to analyze since in many casesthe environment behavior is not knownIn practice open systems are verified by making worst-caseassumptions about the environment
This is compatible with the assume-guarantee style of reasoning
Traditionally, model checking is based on two requirements on themodel: finiteness and closenessTo be closed, a system must include all the possible inputs andpossible interactions with the environmentOpen systems are more difficult to analyze since in many casesthe environment behavior is not knownIn practice open systems are verified by making worst-caseassumptions about the environment
This is compatible with the assume-guarantee style of reasoning
1 The Great DebatesBranching vs Linear TimeSymbolic vs Explicit VerificationBreadth-First Search vs Depth-First SearchTarjan’s SCC Algorithms vs Spin’s Nested Depth-First SearchEvents vs StatesReal-time vs Timeless VerificationProbabilities vs PossibilitiesAsynchronous vs Synchronous SystemsInterleaving Semantics vs True ConcurrencyOpen vs Closed SystemsBackward vs Forward ReachabilityCompositional vs Non-compositional VerificationDeductive vs Algorithmic Verification
Many verification problems can be reduced to reachability analysisReachability may be done using a forward or a backward analysisForward: Verification start in the initial state and computes the setof successors in the reachability graph
It terminates when the intersection of reachable states with theintended (bad) state is non-empty
Backward: Verification start in a final state (or a state marked asbad) and computes the set of predecessors in the reachabilitygraph
It terminates when the intersection of reachable states with theinitial state is non-empty
In both cases heuristics may direct the search spaceWhich one to use depends on the particular application and thekind of properties
E.g., in real-time verification backward reachability is widely usedfor safety properties (the timed automata containing an error state)
Many verification problems can be reduced to reachability analysisReachability may be done using a forward or a backward analysisForward: Verification start in the initial state and computes the setof successors in the reachability graph
It terminates when the intersection of reachable states with theintended (bad) state is non-empty
Backward: Verification start in a final state (or a state marked asbad) and computes the set of predecessors in the reachabilitygraph
It terminates when the intersection of reachable states with theinitial state is non-empty
In both cases heuristics may direct the search spaceWhich one to use depends on the particular application and thekind of properties
E.g., in real-time verification backward reachability is widely usedfor safety properties (the timed automata containing an error state)
Many verification problems can be reduced to reachability analysisReachability may be done using a forward or a backward analysisForward: Verification start in the initial state and computes the setof successors in the reachability graph
It terminates when the intersection of reachable states with theintended (bad) state is non-empty
Backward: Verification start in a final state (or a state marked asbad) and computes the set of predecessors in the reachabilitygraph
It terminates when the intersection of reachable states with theinitial state is non-empty
In both cases heuristics may direct the search spaceWhich one to use depends on the particular application and thekind of properties
E.g., in real-time verification backward reachability is widely usedfor safety properties (the timed automata containing an error state)
Many verification problems can be reduced to reachability analysisReachability may be done using a forward or a backward analysisForward: Verification start in the initial state and computes the setof successors in the reachability graph
It terminates when the intersection of reachable states with theintended (bad) state is non-empty
Backward: Verification start in a final state (or a state marked asbad) and computes the set of predecessors in the reachabilitygraph
It terminates when the intersection of reachable states with theinitial state is non-empty
In both cases heuristics may direct the search spaceWhich one to use depends on the particular application and thekind of properties
E.g., in real-time verification backward reachability is widely usedfor safety properties (the timed automata containing an error state)
1 The Great DebatesBranching vs Linear TimeSymbolic vs Explicit VerificationBreadth-First Search vs Depth-First SearchTarjan’s SCC Algorithms vs Spin’s Nested Depth-First SearchEvents vs StatesReal-time vs Timeless VerificationProbabilities vs PossibilitiesAsynchronous vs Synchronous SystemsInterleaving Semantics vs True ConcurrencyOpen vs Closed SystemsBackward vs Forward ReachabilityCompositional vs Non-compositional VerificationDeductive vs Algorithmic Verification
In fact there is no big debate about this topic since everybodywants to have a compositional verification methodCompositionality is good for
Increasing local reasoningMinimizing the state-space search (using less memory)Parallelizing (distributing) the verification algorithm withoutrepeating computation
If it so good, why not to write only compositional algorithms?Obtaining compositional algorithms is not easy in general
Many approaches are non-compositional or they are restricted tocertain kind of subsystems and propertiesIn many cases the underlying system is not compositional by natureSometimes it is only possible to obtain partial compositionalalgorithms, still needing a global computation but in a reducedstate-space
In fact there is no big debate about this topic since everybodywants to have a compositional verification methodCompositionality is good for
Increasing local reasoningMinimizing the state-space search (using less memory)Parallelizing (distributing) the verification algorithm withoutrepeating computation
If it so good, why not to write only compositional algorithms?Obtaining compositional algorithms is not easy in general
Many approaches are non-compositional or they are restricted tocertain kind of subsystems and propertiesIn many cases the underlying system is not compositional by natureSometimes it is only possible to obtain partial compositionalalgorithms, still needing a global computation but in a reducedstate-space
In fact there is no big debate about this topic since everybodywants to have a compositional verification methodCompositionality is good for
Increasing local reasoningMinimizing the state-space search (using less memory)Parallelizing (distributing) the verification algorithm withoutrepeating computation
If it so good, why not to write only compositional algorithms?Obtaining compositional algorithms is not easy in general
Many approaches are non-compositional or they are restricted tocertain kind of subsystems and propertiesIn many cases the underlying system is not compositional by natureSometimes it is only possible to obtain partial compositionalalgorithms, still needing a global computation but in a reducedstate-space
1 The Great DebatesBranching vs Linear TimeSymbolic vs Explicit VerificationBreadth-First Search vs Depth-First SearchTarjan’s SCC Algorithms vs Spin’s Nested Depth-First SearchEvents vs StatesReal-time vs Timeless VerificationProbabilities vs PossibilitiesAsynchronous vs Synchronous SystemsInterleaving Semantics vs True ConcurrencyOpen vs Closed SystemsBackward vs Forward ReachabilityCompositional vs Non-compositional VerificationDeductive vs Algorithmic Verification
Verification can be done following a deductive approach (likeManna & Pnueli’s formalism) or algorithmically (like Spin)Deductive verification
Usually uses theorem provers/assistantsDifficult to automatizeApplication domain: any kind of system (finite and infinite)Needs a lot of expertise for using it
Algorithmic verificationCompletely automatic in most cases; semi-automatic in someapplicationsUsually restricted to finite-state systems
Automatic procedures for infinite-state systems has been developedusing special techniques and/or suitable abstractions
A push-button procedure: almost everybody can use it
Some attempts have been done to combine both approaches
Verification can be done following a deductive approach (likeManna & Pnueli’s formalism) or algorithmically (like Spin)Deductive verification
Usually uses theorem provers/assistantsDifficult to automatizeApplication domain: any kind of system (finite and infinite)Needs a lot of expertise for using it
Algorithmic verificationCompletely automatic in most cases; semi-automatic in someapplicationsUsually restricted to finite-state systems
Automatic procedures for infinite-state systems has been developedusing special techniques and/or suitable abstractions
A push-button procedure: almost everybody can use it
Some attempts have been done to combine both approaches
Verification can be done following a deductive approach (likeManna & Pnueli’s formalism) or algorithmically (like Spin)Deductive verification
Usually uses theorem provers/assistantsDifficult to automatizeApplication domain: any kind of system (finite and infinite)Needs a lot of expertise for using it
Algorithmic verificationCompletely automatic in most cases; semi-automatic in someapplicationsUsually restricted to finite-state systems
Automatic procedures for infinite-state systems has been developedusing special techniques and/or suitable abstractions
A push-button procedure: almost everybody can use it
Some attempts have been done to combine both approaches
Verification can be done following a deductive approach (likeManna & Pnueli’s formalism) or algorithmically (like Spin)Deductive verification
Usually uses theorem provers/assistantsDifficult to automatizeApplication domain: any kind of system (finite and infinite)Needs a lot of expertise for using it
Algorithmic verificationCompletely automatic in most cases; semi-automatic in someapplicationsUsually restricted to finite-state systems
Automatic procedures for infinite-state systems has been developedusing special techniques and/or suitable abstractions
A push-button procedure: almost everybody can use it
Some attempts have been done to combine both approaches