Laboratory for Advanced Software Engineering Research Laboratory for Advanced Software Engineering Research UMASS UMASS Lori A. Clarke University of Massachusetts [email protected]http://laser.cs.umass.edu/ Finite State Verification: An Emerging Technology for Validating Software Systems
65
Embed
Finite State Verification: An Emerging Technology for Validating Software Systems
Finite State Verification: An Emerging Technology for Validating Software Systems. Lori A. Clarke University of Massachusetts [email protected] http://laser.cs.umass.edu/. UMASS. Laboratory for Advanced Software Engineering Research. Outline of Presentation. Lay of the Land: - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Solution exists e.g., x2, x10 = 0, all other xi = 1 => property does not hold
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Seeing the counter example
Property: For all paths, event a occurs more than event b
x1x2
x4
x5x6 x9x10
x0 x8
x3
x7
x11
a a’b’ b
x2, x10 = 0, all other xi = 1
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Some Limitations
• Integer Linear Programming has an exponential worst case bound
• Inter-process order information is not preserved– only checks whether event counts are
consistent– Like most static techniques, may produce
spurious results
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Some Benefits
• Does not enumerate the state space!
• Integer linear Programming is often very efficient– Empirical evidence: linear inequality systems
usually grow linearly and take sub-exponential times to solve
• In practice, INCA is usually an effective technique
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Data Flow Based Verification: some history
• Mid-70’s: Originally proposed for def-ref anomalies in FORTRAN (Osterweil and Fosdick)
• Early 80’s: Extended to general properties (Olender and Osterweil) & concurrency (Taylor and Osterweil)
• 90’s: Deadlock detection (Masticola and Ryder); Efficient representation of concurrency & incremental precision improvement (Dwyer and L. Clarke)
• Recent: Optimizations, Java (Avrunin, L. Clarke, Cobleigh, Naumovich, and Osterweil)
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Data Flow Analysis: FLAVERS
• Represents property as a finite state automaton
• System model is collection of annotated control flow graphs– Inter-process communication and interleavings are
represented with additional edges– does not enumerate all reachable states– over-approximates relevant executable behaviors
• Reasoning engine based on data flow analysis
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Property
System
Property Translator
SystemTranslator
State Propagation
Collection of annotated CFG’s
Property Verified
FSA
High-Level Architecture of High-Level Architecture of FSV SystemsFSV Systems
Counter Examples from Model
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Modeling the System
3
4
2
9
T1 T2
5
7
1,6
2,6
5,9
3,6
4
8
1,71 6
2,7 1,8
3,7 2,8
3,8
•State explosion
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Modeling the System
3
4
2
9
T1 T2
5
7
8
1 6 •Automatically creates the program model from source code
•Instead of the state space, explicitly represents interleaved execution via edges
•Smaller model
•Loss of precision
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Representing PropertiesRepresenting Properties
Example:
close,open,move
0
1
openclose
2
move
closemove
open
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
State Propagation
• States of the property are propagated through the model
• The property is proved if only accepting (non-accepting) states are contained in the final node of the model
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Example
public static void main (String [] args){ … if (elevatorStopped) {... openDoors(); } recordState(); if (elevatorStopped) {... closeDoors(); } moveToNextFloor();}
if
open
close
if
move
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Example
if
open
close
if
move close,open,move
0
1
openclose
2
move
closemove
open
{0}
{1}
{0,1}
{0}
{0,2}
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Example with Constraints
if
open
close
if
move
S==true S==false
S==falseS==true
0
21
viol
S==true S==false
S==true
S==true
S==false
S==false
S==falseS==true
Constraint
close,open,move
0
1
openclose
2
move
closemove
open
Property(0,0)
(0,1)
(1,1) (1,1)
(1,viol)
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Example with Constraints
if
open
close
if
move
S==true S==false
S==falseS==true
0
21
viol
S==true S==false
S==true
S==true
S==false
S==false
S==falseS==true
Constraint
close,open,move
0
1
openclose
2
move
closemove
open
Property(0,0)
(0,1)
(1,1){(1,1), (0,2)}
(0,2)
{(1,1), (0,viol)} {(1,viol), (0,2)}
{(0,1)}
{(0,1), (0,2)}
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Some Observations: Data Flow Analysis• Overall complexity is O(N2S)
– N is the # nodes in the model – S is the number of states: property x constraints– Experimentally: performance subexponential
• Usually requires several iterations to determine needed constraints
• Constraints– Many automatically generated on request– Can be used to model other information
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Experimental Comparisons
• All these approaches are:– very effective on some problems– disappointing on some problems
• Hard to predict how they will perform
• Experimental results– George S. Avrunin, James C. Corbett, Matthew B. Dwyer, Corina S.
Pasareanu, and Stephen F. Siegel, Comparing Finite-State Verification Techniques for Concurrent Software
Very Big Disclaimer!
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
url: laser.cs.umass.edu
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Can we move beyond academic prototypes to practitioners’ tools?
• Yes, but there is more work to be done– Optimization, optimization, optimization– Process support– Better support for specifying properties– Better support for generating, selecting, visualizing
counter example traces – Better approaches for dealing with dynamism– Full support for real languages– Full lifecycle support
• Integration with testing
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Specifying Properties
• It is very hard to specify properties precisely– E.g., open and close file repeatedly
• Must file always be opened?Or, IF it is opened, then it must be closed?
• Can file be opened repeatedly before it is closed?
• Need notations that are easy to use– Specification patterns
• Need tools to help understand properties– need to test the properties
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Counter Example Traces
• Want “short” but “useful” counter examples
• How to select the “next” counter example?
• How to incorporate user guidance?
• How to go from traces in the model to traces in the program?
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Dynamism
• FSV is a static analysis approach that deals with static models– Must create a specific instance of the model
• E.g., N philosophers => 5 philosphers
– Can not handle • dynamic objects
• dynamic process creation
• Need hybrid techniques that integrate theorem proving with FSV
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Support for Real Languages
• Many language features have not been addressed– Aliasing – Exception handling– Event based notification
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS
Lifecycle-based Verification
• High-level architectural design– Extremely important for distributed systems
• Detect problems early
– Need to support heterogeneous interaction models
• Low-level design– Additional detail leads to additional properties– Need to maintain consistency with the HLA
Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS