Carnegie Mellon University System Modeling System Modeling and Verification and Verification with UCLID with UCLID http://www.cs.cmu.edu/~bryant Randal E. Bryant Contributions by former graduate students: Sanjit Seshia, Shuvendu Lahiri
Carnegie Mellon University
System ModelingSystem Modelingand Verificationand Verification
with UCLIDwith UCLID
System ModelingSystem Modelingand Verificationand Verification
with UCLIDwith UCLID
http://www.cs.cmu.edu/~bryant
Randal E. Bryant
Contributions by former graduate students:Sanjit Seshia, Shuvendu Lahiri
– 2 –
Applying Data Abstraction to Hardware VerificationApplying Data Abstraction to Hardware VerificationIdeaIdea
Abstract details of data encodings and operations Keep control logic precise
ApplicationsApplications Verify overall correctness of system Assuming individual functional units correct
TechnologyTechnology Use restricted subset of first-order logic Implement efficient decision procedures Multiple methods of performing verification
– 3 –
Alpha 21264 Microprocessor
Microprocessor Report, Oct. 28, 1996
Challenge: System-Level VerificationChallenge: System-Level Verification
Verification TaskVerification Task Does processor
implement its ISA?
Why is it Hard?Why is it Hard? Lots of internal state Complex control
logic Complex
functionality
– 4 –
Sources of ComplexitySources of Complexity
StateState ISA: registers, memory Microarchitectural: caches, buffers, reservation stations Conceptually finite state, but practically unbounded
ControlControl Pipelines spread execution across multiple cycles Out-of-order execution modifies processing order Superscalar operation creates parallelism Control logic coordinates everything
Resulting behavior matches that of sequential ISA model
FunctionalityFunctionality Arithmetic functions, instruction decoding
– 5 –
Existing Verification MethodsExisting Verification Methods
Simulators, equivalence checkers, model checkers, …
All Operate at Bit LevelAll Operate at Bit Level RTL model
State encoded as words and arrays of wordsComprised of bits
Most Operate at Cycle or Subcycle LevelMost Operate at Cycle or Subcycle Level How each bit of state gets updated
System Modeling LanguagesSystem Modeling Languages Abstract time up to transaction level Still view state as collection of bits
– 6 –
Word-Level AbstractionWord-Level Abstraction
Data: Abstract details of form & functions
Control: Keep at bit level
Timing: Keep at cycle level
Control LogicControl Logic
Data PathData Path
Com.Log.
1
Com.Log.
2
– 7 –
Data Abstraction #1: Bits → IntegersData Abstraction #1: Bits → Integers
View Data as Symbolic WordsView Data as Symbolic Words Arbitrary integers
No assumptions about size or encodingClassic model for reasoning about software
Can store in memories & registers
x0x1x2
xn-1
x
– 8 –
Modeling Data SelectionModeling Data Selection
If-Then-Else OperationIf-Then-Else Operation Mulitplexor Allows control-dependent data flow
1
0
xy
p
ITE(p, x, y)1
0
xy
1
x1
0
xy
0
y
– 9 –
Data PathData Path
Com.Log.
1
Com.Log.
2
Abstracting Data BitsAbstracting Data Bits
Control LogicControl Logic
Data PathData Path
Com.Log.
1
Com.Log.
1? ?
What do we do about logic functions?
– 10 –
Abstraction #2:Uninterpreted Functions
Abstraction #2:Uninterpreted Functions
For any Block that Transforms or Evaluates Data:For any Block that Transforms or Evaluates Data: Replace with generic, unspecified function Only assumed property is functional consistency:
a = x b = y f (a, b) = f (x, y)
ALUf
– 11 –
Abstracting FunctionsAbstracting Functions
For Any Block that Transforms Data:For Any Block that Transforms Data: Replace by uninterpreted function Ignore detailed functionality Conservative approximation of actual system
Data PathData Path
Control LogicControl Logic
Com.Log.
1
Com.Log.
1F1 F2
– 12 –
Modeling Data-Dependent ControlModeling Data-Dependent Control
Model by Uninterpreted PredicateModel by Uninterpreted Predicate Yields arbitrary Boolean value for each control + data
combination Produces same result when arguments match
Pipeline & reference model will branch under same conditions
Cond
Adata
Bdata
Branch?
Bra
nch
Lo
gic
p
– 13 –
Abstraction #3: Modeling Memories as Mutable FunctionsAbstraction #3: Modeling Memories as Mutable Functions
Memory M Modeled as FunctionMemory M Modeled as Function
M(a): Value at location a
InitiallyInitially
Arbitrary state Modeled by uninterpreted function m0
Ma
M
a m0
– 14 –
Effect of Memory Write OperationEffect of Memory Write Operation
Writing Transforms MemoryWriting Transforms Memory M = Write(M, wa, wd)
Reading from updated memory:
Address wa will get wdOtherwise get what’s
already in M
Express with Lambda NotationExpress with Lambda Notation Notation for defining
functions M =
a . ITE(a = wa, wd, M(a))
M
Ma 1
0
wd
=wa
– 15 –
Systems with BuffersSystems with Buffers
Modeling MethodModeling Method Mutable function to describe buffer contents Integers to represent head & tail pointers
• • • ••• •••
tailtail headhead
In Use
••••••
tailtailheadheadheadhead
In Use
•••
0 0 0 0 MaxMax--11MaxMax--11
Unbounded Buffer Circular Queue
– 16 –
Some History of Term-Level ModelingSome History of Term-Level Modeling
HistoricallyHistorically Standard model used for program verification Widely used with theorem-proving approaches to hardware
verificationE.g, Hunt ’85
Automated Approaches to Hardware VerificationAutomated Approaches to Hardware Verification Burch & Dill, ’95
Tool for verifying pipelined microprocessors Implemented by form of symbolic simulation
Continued application to pipelined processor verification
– 17 –
UCLIDUCLID
Seshia, Lahiri, Bryant, CAV ‘02
Term-Level Verification SystemTerm-Level Verification System Language for describing systems
Inspired by CMU SMV
Symbolic simulatorGenerates integer expressions describing system state after
sequence of steps
Decision procedureDetermines validity of formulas
Support for multiple verification techniques
Available by DownloadAvailable by Downloadhttp://www.cs.cmu.edu/~uclid
– 18 –
Challenge: Model GenerationChallenge: Model Generation
How to generate term-level model How to guarantee faithfulness to RTL description
Comparison of ModelsComparison of Models RTL
Abstracts functional elements from gate-level modelSynthesis allows automatic map to gate level
Term levelAbstracts bit-level data representations to wordsAbstracts memories to mutable functionsNo direct connection to synthesizable model
– 19 –
Generating Term-Level ModelGenerating Term-Level Model
Manually Generate from RTLManually Generate from RTL How do we know it is a valid abstraction? Hard to keep consistent with changing RTL
Automatically Generate from RTLAutomatically Generate from RTL Andraus & Sakallah, DAC ‘04 Must decide which signals to keep Boolean, which to
abstractConfused by bit field extraction primitives of HDL
Synthesize RTL from Word-Level ModelSynthesize RTL from Word-Level Model Difficult to make efficient
– 20 –
Underlying LogicUnderlying Logic
Existing Approaches to Formal VerificationExisting Approaches to Formal Verification E.g., symbolic model checking State encoded as fixed set of bits
Finite state systemAmenable to Boolean methods (SAT, BDDs)
Our TaskOur Task State encoded with unbounded data types
Arbitrary integersFunctions over integers
Must use decision proceduresDetermine validity of formula in some subset of first-order logicAdapt methods historically used by automated theorem provers
– 21 –
EUF: Equality with Uninterp. FunctsEUF: Equality with Uninterp. Functs Decidable fragment of first order logic
Formulas (Formulas (F F )) Boolean ExpressionsBoolean ExpressionsF, F1 F2, F1 F2 Boolean connectives
T1 = T2 Equation
P (T1, …, Tk) Predicate application
Terms (Terms (T T )) Integer ExpressionsInteger ExpressionsITE(F, T1, T2) If-then-else
Fun (T1, …, Tk) Function application
Functions (Functions (FunFun)) Integer Integer Integer Integerf Uninterpreted function symbol
x1, …, xk . T Function lambda expression
Predicates (Predicates (PP)) Integer Integer Boolean Booleanp Uninterpreted predicate
symbol
– 22 –
UCLID OperationUCLID Operation
OperationOperation Series of
transformations leading to propositional formula
Except for lambda expansion, each has polynomial complexity
LambdaExpansion
Function&
PredicateElimination
FiniteInstantiation
BooleanSatisfiability
SymbolicSimulation
file.ucl
Model+
Specification
UCLIDFormula
-freeFormula
TermFormula
BooleanFormula
– 23 –
Pipeline
UCLID ExampleUCLID ExampleDLX PipelineDLX Pipeline
Single-issue, 5-stage pipeline
pc
pPC
fd
Valid
PC
Type
Instr
de
Valid
PC
Type
Instr
Arg2
Arg1
mw
Valid
Dest
Data
em
Branch
Valid
Type
Instr
Arg2
Value
Target
RF Mem
Fetch Decode Execute MemoryWriteBack
Integer state
Boolean state
Function state
– 24 –
Writing & Reading Register FileWriting & Reading Register File
fd
Instr
de
Arg2
Arg1
mw
Valid
Dest
Data
RF
DecodeWriteBack
src1
src2
– 25 –
Writing Register FileWriting Register File
mw
Valid
Dest
Data
RF
WriteBack
init[RF] := rf0; (* Uninterpreted Function *) next[RF] := Lambda(a) . case mw_Valid & (a = mw_Dest) : mw_Data; default : RF(a); esac;
– 26 –
Reading Register FileReading Register File
fd
Instr
de
Arg2
Arg1
RF
Decode
src1
src2
init[de_Arg1] := dea10; (* Initially arbitary *) next[de_Arg1] := next[RF](src1(fd_Instr)); init[de_Arg2] := dea20; (* Initially arbitary *) next[de_Arg2] := next[RF](src2(fd_Instr));
Write-before-read semantics
– 27 –
ReachableStates
Verifying Safety PropertiesVerifying Safety Properties
State Machine ModelState Machine Model State encoded as Booleans, integers, and functions Next state function expresses how updated on each step
Prove: System will never reach bad stateProve: System will never reach bad state
ResetStates
BadStates
PresentState
NextState
Inputs(Arbitrary)
Reset
– 28 –
Reachable
• • •
Rn
R2
Bounded Model CheckingBounded Model Checking
Repeatedly Perform Image Repeatedly Perform Image ComputationsComputations Set of all states reachable
by one more state transition
Easy to ImplementEasy to Implement
Underapproximation of Underapproximation of Reachable State SetReachable State Set But, typically catch most
bugs with 8–10 steps
BadStates
R1
ResetStates
– 29 –
• • •
Rn
R2
True Model CheckingTrue Model Checking
Reach Fixed-PointReach Fixed-Point Rn = Rn+1 = Reachable
Impractical for Term-Level Impractical for Term-Level ModelsModels
Many systems never reach fixed point
Can keep adding elements to buffer
Convergence test undecidable
BadStates
R1
ResetStates
– 30 –
I
Inductive Invariant CheckingInductive Invariant Checking
Key Properties of System that Make it Operate CorrectlyKey Properties of System that Make it Operate Correctly Formulate as formula I
Prove InductiveProve Inductive Holds initially I(s0)
Preserved by all state changes I(s) I((i, s))
ReachableStates
ResetStates
BadStates
– 31 –
An Out-of-order Processor (OOO)An Out-of-order Processor (OOO)
Data Dependencies Resolved by Register RenamingData Dependencies Resolved by Register Renaming Map register ID to instruction in reorder buffer that will generate
register value
Inorder Retirement Managed by Retirement BufferInorder Retirement Managed by Retirement Buffer FIFO buffer keeping pending instructions in program order
Reorder BufferFields
PC
Programmemory
Reorder Buffer
validvaluesrc1validsrc1valsrc1tagsrc2validsrc2valsrc2tagdestop
result bus
DECODE Register
Rename Unit
valid tag val
ALU
head tail
incrdispatch
retire
execute
result
1st
Operand
2nd
Operand
– 32 –
Verifying OOOVerifying OOO Lahiri, Seshia, & Bryant,
FMCAD 2002
GoalGoal Show that OOO implements
Instruction Set Architecture (ISA) model
For all possible execution sequences
ChallengeChallenge OOO holds partially executed
instructions in reorder bufferStates of two systems match
only when reorder buffer flushed
ISA
Reg.File
PC
OOO
Reg.File
PCReorder Buffer
– 33 –
Adding Shadow StateAdding Shadow State
McMillan, ‘98 Arons & Pnueli, ‘99
Provides Link Between ISA Provides Link Between ISA & OOO Models& OOO Models Additional entries in ROB
Do not affect OOO behavior
Generated when instruction dispatched
Predict values of operands and result
From ISA model
ISA
Reg.File
PC
OOO
Reg.File
PCReorder Buffer
– 34 –
Invariant CheckingInvariant Checking
Formulas Formulas II11, …, , …, IInn
Ij(s0) holds for any initial state s0, for 1 j n
I1(s) I2(s) … In(s) Ij(s ) for any current state s and successor state s for 1 j n
Invariants for OOO (13)Invariants for OOO (13) Refinement maps (2)
Show relation between ISA and OOO models
Shadow state (3)Shadow values correctly predict OOO values
State consistency (8)Properties of OOO state that ensure proper operation
Overall CorrectnessOverall Correctness Follows by induction on time
– 36 –
OOO InvariantsOOO Invariants
Split into Formulas Split into Formulas II11, …, , …, IInn
Ij(s0) holds for any initial state s0, for 1 j n
I1(s) I2(s) … In(s) Ij(s ) for any current state s and successor state s for 1 j n
Invariants for OOO (13)Invariants for OOO (13) Refinement maps (2)
Show relation between ISA and OOO models
State consistency (8)Properties of OOO state that ensure proper operation
Added state (3)Shadow values correctly predict OOO values
Overall CorrectnessOverall Correctness Follows by induction on time
– 37 –
State Consistency Invariant Examples State Consistency Invariant Examples Register Renaming invariants (2)Register Renaming invariants (2)
Tag in a rename-unit should be in the ROB, and the destination register should matchr.reg.valid(r) (rob.head reg.tag(r) < rob.tail
rob.dest(reg.tag(r)) = r )
For any entry, the destination should have reg.valid as false and tag should contain this or later instructionrobt.(reg.valid(rob.dest(t))
t reg.tag(rob.dest(t)) < rob.tail)
– 38 –
Extending the OOO ProcessorExtending the OOO Processor
baseExecutes ALU instructions only
excHandles arithmetic exceptionsMust flush reorder buffer
exc/brHandles branchesPredicts branch & speculatively executes along path
exc/br/mem-simpAdds load & store instructionsStore commits as instruction retires
exc/br/memStores held in bufferCan commit laterLoads must scan buffer for matching addresses
– 39 –
Comparative Verification EffortComparative Verification Effort
base exc exc / br exc / br / mem-simp
exc / br / mem
Total Invariants
13 34 39 67 71
Manually instantiate
0 0 0 4 8
UCLID time
54 s 236 s 403 s 1594 s 2200 s
Person time
2 days 7 days 9 days 24 days 34 days
(Person time shown cumulatively)
– 40 –
“I Just Want a Loaf of Bread”“I Just Want a Loaf of Bread”Ingredients
Recipe Result
– 41 –
Cooking with InvariantsCooking with InvariantsIngredients: Predicates
Recipe: Invariants
Result: Correctness
reg.valid(r)
r,t.reg.valid(r) reg.tag(r) = t (rob.head reg.tag(r) < rob.tail rob.dest(t) = r )
rob.head reg.tag(r)
reg.tag(r) = t
rob.dest(t) = r
– 42 –
Automatic Recipe GenerationAutomatic Recipe Generation
Want Something MoreWant Something More Given any set of ingredients Generate best recipe possible
Ingredients
Recipe Creator Result
– 43 –
Automatic Predicate AbstractionAutomatic Predicate Abstraction
Graf & Saïdi, CAV ‘97
IdeaIdea Given set of predicates P1(s), …, Pk(s)
Boolean formulas describing properties of system state
View as abstraction mapping: States {0,1}k
Defines abstract FSM over state set {0,1}k
Form of abstract interpretationDo reachability analysis similar to symbolic model checking
ImplementationImplementation Early ones had weak inference capabilities
Call theorem prover or decision procedure to test each potential transition
Recent ones make better use of symbolic encodings
– 44 –
Abstract State SpaceAbstract State Space
ConcreteStates
AbstractStates
P1(s), …, Pk(s)
s
AbstractionFunction
t
Abstraction
ConcreteStates
AbstractStates
s t
ConcretizationFunction
Concretization
– 45 –
Abstract State MachineAbstract State Machine
Transitions in abstract system mirror those in concrete
Abstract
ConcreteSystem
AbstractSystem
s
Concretize
t t
s
Concrete Transition
Abstract Transition
– 46 –
Generating Concrete InvariantGenerating Concrete Invariant
Reach Fixed-Point on Reach Fixed-Point on Abstract SystemAbstract System Termination guaranteed,
since finite state
Equivalent to Computing Equivalent to Computing Invariant for Concrete Invariant for Concrete SystemSystem Strongest possible
invariant that can be expressed by formula over these predicates
• • •Rn
R2
R1
ResetStates
A
AbstractSystem
Concretize
ConcreteSystem
I
ResetStates
C
– 47 –
Systems Verified with Predicate AbstractionSystems Verified with Predicate Abstraction
Very general modelsUnbounded processes, buffers, cache lines, …
Safety properties only
Model Predicates Iterations CPU Time
Out-Of-Order Execution Unit 25 9 1,207s
German’s Cache Protocol 13 9 14s
German’s Protocol, unbounded channels
24 17 427s
Bounded Retransmission Buffer 22 9 11s
Lamport’s Bakery Algorithm 33 18 471s
– 48 –
Automatic Predicate DiscoveryAutomatic Predicate Discovery
Strength of Predicate AbstractionStrength of Predicate Abstraction If give it right set of predicates, PA will put them together
into invariant
WeaknessWeakness Gets nowhere without right set of predicates Typical failure mode: Generate “true” as invariant
ChallengesChallenges Too many predicates will overwhelm PA engine Our use of quantified invariants precludes counterexample-
generated refinement techniques
– 49 –
Implementation of Predicate DiscoveryImplementation of Predicate Discovery
Lahiri & Bryant, CAV ’04 Initially: Extract predicates from verification condition Iterate: Add new predicates by composing next-state
formulasWith some heuristics thrown in
ExperienceExperience Can automatically generate invariants for real examples ~10X slower than for hand-selected predicates
– 50 –
Future ProspectsFuture Prospects
EvaluationEvaluation Important to abstract data & data functions while
maintaining details about control Demonstrated ability to verify complex, parameterized
systems
Model Generation is Weakest LinkModel Generation is Weakest Link Big jump from bit-level to term-level
Look at intermediate levels of abstraction with bit-vectors
Need algorithmic connection between our model and RTL
Predicate Abstraction Shows PromisePredicate Abstraction Shows Promise Provides key automation advantage of model checking