This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Test Input Generation With Java Pathfinder – Visser, Pasareanu, Khurshid
Generalized Symbolic Execution for Model Checking and Testing – Khurshid, Pasareanu, Visser
Symbolic Execution and Model Checking for Testing - Pasareanu, Visser
JPF–SE - A symbolic execution extension to Java PathFinder – Anand, Pasareanu, Visser
JPF is an explicit state software model checker for Java bytecode
Usage
• As an exhaustive testing tool
• As a model checker
• Deploys abstractions and optimizations
• Extensibility– Search-/VMListeners
– Model Java Interface (MJI)
– Configurable ChoiceGenerators
– Bytecode Factories (New, no tutorials yet)
What’s New• Symbolic Test Data Generation The symbolic execution extension uses a BytecodeFactory to
override JPF's core bytecodes to generate concrete test cases. In a nutshell, this works by using the JPF field/stackframe attribute system to collect symbolic path conditions, which are then fed into a contraint solver to obtain concrete test data. The choice points in this execution mode are branch instructions.
• Symbolic Threadsafety Test Generation This is a very simple "symbolic" execution mode that tries to identify potential thread safety problems, and then creates code to test this with concrete value JPF execution mode. The symbolic information is much more simple than with the symbc extension, and can be kept in the execution trace. The first implemented test category detects un-synchronized access of fields from within the same public (exported) method, to identify candidates for the PreciseRaceDetector listener.
• Compositional Verification Framework The CV extension implements a machine learning algorithm that can be used for assume/guarantee reasoning, to partition a system into components that can be verified separately. Ultimately, the goal of this extension is to dramatically improve the scalability of JPF, but it can be also used to generate environment assumptions for the UML model checking, to determine "valid" event sequences. This extension is mostly implemented with JPF listeners.
• Numeric Property Verification This bytecode replacement extension started as a set of numeric instruction classes to detect over/underflow, but got extended to also include inexact value propagation (NaN, Inf), exact floating point comparison, and potential catastrophic cancellation (loss of precision by subtraction/addition)
• UML State Chart Model Checking The state chart framework is a variation of the user interface model checking, for the purpose of model checking UML state chart diagrams. It is a combination of MJI libraries that are based on a unique translation scheme for UML state charts, together with generic applications (test drivers) to execute these state charts. The framework supports both testing and model checking, using the same scripting language for environment specification.
Symbolic Execution and
Test-input GenerationWillem Visser
&Corina Pasareanu and Peter Mehlitz
RIACS/Kestrel Technology/CSCNASA Ames Research Center
• Motivation for Model Checking Programs• Introduction to Java PathFinder• Symbolic Execution• Coverage based Test-input generation• Conclusions
13
Concrete Execution Path (example)
x = 1, y = 0
1 >? 0
x = 1 + 0 = 1
y = 1 – 0 = 1
x = 1 – 1 = 0
0 – 1 >? 0
int x, y;
if (x > y) {
x = x + y;
y = x – y;
x = x – y;
if (x – y > 0)
assert(false);
}
14
Symbolic Execution Tree (example)
x = X, y = Y
X >?
Y
[ X > Y ] y = X + Y – Y = X
[ X > Y ] x = X + Y – X = Y
[ X > Y ] Y - X >? 0
[ X <= Y ] END [ X > Y ] x = X + Y
[ X > Y, Y – X <= 0 ] END [ X > Y, Y – X > 0 ] END
int x, y;
if (x > y) {
x = x + y;
y = x – y;
x = x – y;
if (x – y > 0)
assert(false);
}
15
Forward Symbolic Execution
• technique for executing a program on symbolic input values
• explore program paths– for each path, build a path condition
– check satisfiability of path condition
• state– symbolic values of variables, path condition, and counter
• various applications– test generation
– program verification
• traditional use– programs with fixed number of int variables
16
Challenges in Generalizing Symbolic Execution
• how to handle fields in dynamic structures?
• how to handle aliasing?
• how to generate tests?– satisfy criteria– satisfy precondition– are in-equivalent
17
Example
class Node {int elem;Node next;
Node swapNode() { if (next != null) if (elem > next.elem) { Node t = next; next = t.next; t.next = this; return t; } return this;}
}
? null
E0 E1
E0
E0 E1 null
E0 E1 ?
E0 E1
E0 E1
Input list + Constraint Output list
E0 > E1
none
E0 <= E1
none
E0 > E1
E0 > E1
E0 > E1
E1 E0 ?
E1 E0
E1 E0
E1 E0 null
E0 E1
E0
? null
NullPointerException
18
Generalized Symbolic Execution
• model checker generates and explores “symbolic” execution tree– path conditions are added at each branch point
• off-the-shelf decision procedures check path conditions• model checker backtracks if not satisfiable
– non-determinism handles aliasing• explore different heap configurations explicitly
– concurrency
• lazy initialization– initializes program’s inputs on an “as-needed” basis– no a priori bound on input sizes
• preconditions to initialize inputs only with valid values
19
Algorithm (aliasing)
• when method execution accesses field fif (f is uninitialized) { if (f is reference field of type T) { non-deterministically initialize f to null a new object of class T (with uninitialized fields) an object created during prior field initialization (alias) } if (f is numeric/string field) initialize f to a new symbolic value }
boolean repOk(Entry e) { // root has no parent, root is black,… // RedHasOnlyBlackChildren workList = new LinkedList(); workList.add(e); while (!workList.isEmpty()) { Entry current=(Entry)workList.removeFirst(); Entry cl = current.left; Entry cr = current.right; if (current.color == RED) { if(cl != null && cl.color == RED) return false; if(cr != null && cr.color == RED) return false; } if (cl != null) workList.add(cl); if (cr != null) workList.add(cr); } // equal number of black nodes on left and right sub-tree… return true;}
30
Black-box TIG
• Generate inputs based on analysis of input structure– e.g. Rover plan generation, Korat
• 100% “coverage” of input structures up to a predetermined upper-bound– e.g. all red-black trees with 6 or less nodes
• Complex data requires that only valid structures be considered– Use class invariant to reduce number of input structures to consider
• a predicate characterizing all the instances of a class• boolean repOk()
• Check code coverage using generated structures as input
• Advantage – test code for which no source is available
31
Symbolic execution for black-box TIG
• Symbolic execution of repOk()– Generate new structures only when repOk() returns true– Limit the size of the structures generated– Only correct structures will be generated
• repOk() returns true after all nodes in the tree have been visited, hence they must all be concrete
• symbolic (partial) structures can fail repOk()
• Similar to Korat– Except we can also deal with data constraints
32
Symbolic Execution of repOk()Example
public static boolean repOk(Entry e) {
if (e == null) return true;
if (e.color == RED) return false;
…
33
White-box TIG
• Consider code coverage criterion when generating test inputs
• Challenge– Treating complex data with symbolic execution
• Use repOk() as a method precondition during symbolic execution of source code:– use repOk() to convert “symbolic” input structures into
concrete structures that cover the code and pass repOk()– use repOk() also to eliminate “symbolic” structures during
lazy initialization, thus reducing the search space
34
repOk() x 2abstract and concrete
Symbolic Execution of Code During Lazy Initializationcheck Abstract repOK()
When coverage is achieved, solve the symbolic constraints
to create concrete inputs
To concretize inputs by symbolic execution of
Concrete repOk()over symbolic structures
- as with Black-box TIG -
35
White-box TIG: cover branches in deleteEntry(Entry p)
The symbolic structure is used as input to repOk() and lazily executed to obtain the concrete structure
Symbolic structure(s)that coverthe branch
This structure “passes”the abstract repOk()
37
Conservative repOk()
• Used to eliminate symbolic structures that cannot be converted to a concrete structure that satisfy repOk() and therefore do not describe valid inputs
• Because of symbolic structures we use abstraction– conservative_RepOk() can return TRUE, FALSE or Don’t Know
• if FALSE, ignore that structure by backtracking• if TRUE or Don’t Know, continue ...
• Example: (2) Red nodes have only black children.
FALSE TRUE Don’t Know
38
Conservative repOk()
// root has no parent, root is black,… // RedHasOnlyBlackChildren workList = new LinkedList(); workList.add(e); while (!workList.isEmpty()) { Entry current=(Entry)workList.removeFirst(); Entry cl = current.left; Entry cr = current.right; if (current.color == RED) {