Introduction to Formal Property Verification (FPV) Erik Seligman CS 510, Lecture 8, January 2009.

Introduction to Formal Property Verification (FPV)

Erik Seligman

CS 510, Lecture 8, January 2009

Agenda

Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints

Agenda

Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints

Definitions

• Assertion • Statement that must be true in all cases.

• Assumption • Assertion treated as always-true constraint for FPV.

• Cover Point• Condition that must be reachable for valid proof env

• Formal Property Verification (FPV)• Mathematical proofs, not simulation

• Proves assertions: all possible test vectors

Simulation: spot coverage of design space

Motivation for Formal Property Verification

Formal Property Verification (ideal case): full coverage of design space

Simulation: spot coverage of design space

Motivation for Formal Property Verification

Formal Property Verification (ideal case): full coverage of design space

Simulation: spot coverage of design space

Motivation for Formal Property Verification

Formal Property Verification (real life): full coverage in some areas

Major Benefits of FPV for ASIC Projects

Improving Design Process

Major Benefits of FPV for ASIC Projects

Improving Design Process• Force Designer to Think Through Logic

Major Benefits of FPV for ASIC Projects

Improving Design Process• Force Designer to Think Through Logic

• Help Identify Hidden Assumptions

Major Benefits of FPV for ASIC Projects

Improving Design Process• Force Designer to Think Through Logic

• Help Identify Hidden Assumptions

Bug Hunting

Major Benefits of FPV for ASIC Projects

Improving Design Process• Force Designer to Think Through Logic

• Help Identify Hidden Assumptions

Bug Hunting• Unit-Level Validation (before testbench)

Major Benefits of FPV for ASIC Projects

Improving Design Process• Force Designer to Think Through Logic

• Help Identify Hidden Assumptions

Bug Hunting• Unit-Level Validation (before testbench)

• Find Corner Cases Missed in Simulation

Major Benefits of FPV for ASIC Projects

Improving Design Process• Force Designer to Think Through Logic

• Help Identify Hidden Assumptions

Bug Hunting• Unit-Level Validation (before testbench)

• Find Corner Cases Missed in Simulation

• Quickly Verify Design Changes

Major Benefits of FPV for ASIC Projects

Improving Design Process• Force Designer to Think Through Logic

• Help Identify Hidden Assumptions

Bug Hunting• Unit-Level Validation (before testbench)

• Find Corner Cases Missed in Simulation

• Quickly Verify Design Changes

“Peace of Mind”

Useful Assertions for FPV Focus on high-level intent

• Assertions = “executable comments”• Add insight to design

– Micro-assert on a couple of RTL lines less usefulassign foo = bar & baz;A1: assert property (foo == bar & baz);

Don’t be afraid of some modeling code• Auxiliary calculations / wires are fine

– Provide `ifdef to exclude from synthesis• Full reference models in areas of concern• Smaller “shadow models” often very useful

Agenda

Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints

Prerequisites for FPV

RTL design with assertions Clocks must be identified

• Critical since FPV runs over time

• Clocks are ‘special’: driven 1/0/1/0/…

• Need explicit ratios if multiple clocks

Reset pattern must be identified• FPV resets model to known state at start

• Common method: single rst signal (easy)

• More complex design may have reset sequence– Hold RST 10 cycles, then PowerGood for 5 cycles, …

FPV Run Verilog RTL with

Assertions

FPV

FPV Run Verilog RTL with

Assertions

Passing Assertions

FPV

FPV Run Verilog RTL with

Assertions

Passing Assertions

Bounded-Passing

Assertions

FPV

Bounded vs Full Proofs Which of these do you need?

• “Assertion can NEVER be violated”

• “Assertion can never be violated by any possible simulation of length up to <n>”

Bounded proof usually easier for tools

• Use cover point proofs to judge good bound

• Bound == lengths of interesting scenarios

• Some coverage lost vs full proofs

– But often at point of diminishing ROI

Consider modifying starting state

• Fill queue at start of proof…?

FPV Run Verilog RTL with

Assertions

Passing Assertions

Bounded-Passing

Assertions

FPV

Failing Assertions

FPV Run Verilog RTL with

Assertions

Passing Assertions

Bounded-Passing

Assertions

FPV

Failing Assertions

Unknown Assertions

FPV Debug Loop

Verilog RTL with Assertions

Passing Assertions

Bounded-Passing

Assertions

FPV

Failing Assertions

Unknown Assertions

Analyze Failures: RTL error, assertion error, or assumption

needed?

Edit RTL: Fix bugs, add

assumptions

FPV Debug Loop

Verilog RTL with Assertions

Passing Assertions

Bounded-Passing

Assertions

FPV

Failing Assertions

Unknown Assertions

Analyze Failures: RTL error, assertion error, or assumption

needed?

Edit RTL: Fix bugs, add

assumptions

• This is where FPVers spend their time!

Cover Points and FPV Cover Point is opposite of assertion

• “Good”: FPV creates example trace

• “Bad”: FPV proves point unreachable

• Also may aid simulation coverage checks

Examplecover property (opcode == `ADD);

• If it passes, FPV reports trace with ADD op

• If it fails, ADD op cannot exist in FPV env– Maybe bad assumption prevents ADD op

– All proofs are suspect unless this was expected

Add cover points when doing FPV!• Good tools auto-generate some

Agenda

Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints

Running Jasper

Just run ‘JG’ from command line• Can use –batch to run without GUI

Runs are automatically logged• See jgproject/jg.log

GUI hints• Right-clicking usually gets useful options

• Pass over button with cursor for name

Basic Jasper Command File# Load into JG using "source <file>.tcl".# load modelanalyze -clearanalyze -sva traffic_start.velaborate -top traffic # set clocks and resetsclock -clearclock clkreset -clearreset rst# Set engine mode & run proofsset_engine_mode {H D B3 H2}prove -all

Proof Results

View Violation Trace

• How would this work for liveness properties, like a |-> ##[0:$] b?

Violation Trace: “Why”?

Violation Trace: Why? Again

Violation Trace for Liveness

• Trace shows possible infinite violation loop

Alternate JG Debug Tool: The Visualizer

Using The Visualizer

Visualizing Constraints

Visualize Options

Replot With Constraints

Adding More Constraints

Replot Again

Agenda

Introduction To FPV The FPV Process Running FPV Using Jasper FPV Hints

Failing Assertions

Your first FPV run on a block *will* fail• Nobody writes right assumptions in advance!

• Always something you didn’t think of

Thus most of FPVers time is debug• This is OK– debug process gives insight

• Often debugging one assert can help identify other issues

• More assumptions improve counterexample

Assumption Creation Loop: Majority of FV Time

Mindset change from sim; prepare team!

• Early runs have many false negatives More assumptions == more interesting CEX

• Interesting bugs not found on first run

• Several rounds of assumes deep traces

• Be sure to check assumptions too, in simulation or FV

Analyze Failures

Run FV

Add Assumes

Example: Adding an Assumption

In one unit, many assertions failed

• Mid-transaction address changes Needed input assumption to prove

• Bug found when assumption fired in simulation!

RTL Under Test(several cycles per

transaction)

Address Bus

ASSUME (held for ASSUME (held for nn cycles)cycles)

Assumption Count Exploding?

Possible bad choice of boundary• Effectively reimplementing neighbor block?

Consider increasing hierarchy level• Add upper level & many blackboxes?

Also consider simplifying problem•Only cover certain modes

–PCIE: prove for x16, not x4, x8?•Restrict data

–Will one bit test most major logic?–Are fully general payloads needed?

Example: FV too hard?

MPE0

MRA1

MRA0

MPE1

MPE = Memory Protocol Engine

MRA = Memory Read Arbiter

Correct Hierarchy Makes FV Easy

MPE0

MRA1

MRA0

MPE1

MSB

Pick Your Battles

FPV can be effort-intensive• Need good understanding of requirements

• Should concentrate on high-risk areas

FPV owner needs deep understanding• Tool pokes at unusual behaviors, not typical

– Very different from simulation

• If not author of block, need to study intensely– Block owner should be available for questions

• Don’t assign random intern to FPV!

Effective FPV At Intel (Info from public DAC 2008 talk) Chipsets: Found numerous bugs missed in

simulation (PCIe, memory controllers)

• Also uncovered flaw in one validation env CPU designs expanding this usage mode

• Competitive: Recent project devoted 8% of validation resources in front-end design, found 8% of bugs

• 30-35% of bugs found by assertion FV were unlikely to be found in simulation

References / Further Reading•http://www.aracnet.com/~eseligma/docs/dvcon_2006.pdf

• Jasper documentation in /pkgs/jasper/current/doc on ECE systems

•http://www.systemverilog.org/pdf/3b_AssertionsUserTechnology.pdf

•http://www.aycinena.com/index2/index3/iccd%202006%20verification%20panel.html

• http://oskitech.com/papers/datta-mc-vlsi08.pdf