Stop Writing Assertions! Efficient Verification Methodology Dave Whipp – DVClub SV, August 2008.

Stop Writing Assertions!Efficient Verification Methodology

Dave Whipp – DVClub SV, August 2008

NVIDIA Confidential

Todays Talk

An Overview of Methodology CreationMethodology as User Interface

Making ESL work as a Hardware Design FlowEvolution of a Interface Definition Language

Transaction Level Assertions

Transaction Level Debug

NVIDIA Confidential

A HW Development Flow

Big Paper Spec

Design

Verification

RTL

Testbench Checkers Tests

Scoreboard

Assertions

C Model

Synthesis

Directed

RandomBFMs,

TLMs

Clocks, ResetsAssertions

ISS Model

CoverageDebug

Formal

NVIDIA Confidential

User Model for Running Tests

Build Testbench

Generate Test

Run Predictor

Run DUT

Compare Behaviors

*

Part 1

Methodology Design

NVIDIA Confidential

Purpose of a Flow

Conceptual FrameworkWhere do things live?

How do things work?

Keep out of the wayDon’t make life difficult for people

Define MetaphorsSubtly influence future directions

Anchor for VariationGive people something to complain about

NVIDIA Confidential

Two Philosophies

ConformityEmphasize Standardization

DiversityEmphasize Innovation

Synergy or Conflict?

NVIDIA Confidential

Understanding Variation

To understand what to standardize:you need to understand what not to standardize

Personal Preferences

Technical Aspects of the Designs

Supporting Legacy

Seeking the Next Big Thing

NVIDIA Confidential


Emacs!

vi

NVIDIA Confidential


Choice of editor doesn’t affect othersAt least, not much

Choice of scripting language has greater impactBut is encapsulated

A script’s users don’t see the implementation language

Choice of HVL affects whole teamCan’t write “E” tests for a “Vera” testbench!

But a unit testbench isn’t seen by other units

A good flow will allow encapsulation of preferencesI can go to any unit and build & run its tests

Enables rapid localization of infrastructure issues

NVIDIA Confidential

Technological Differences

NVIDIA Confidential

Technical Characteristics

PCIE Video Frame

Buffer

Off-chip Memory

Graphics Pipe

NVIDIA Confidential

Reuse Vs Stagnation

Reuse considered GoodAvoid reinventing the wheel

Build on the shoulders of giants

Reuse invites InertiaReuse can propagate dependencies

Dependencies make things harder to change

Resistance to change is known as inertia

Inertia can lead to StagnationImproper reuse accumulates dependencies

Reused code that is not understood will bit-rot

To avoid stagnation, inject agitation

NVIDIA Confidential

Are Single Paradigm Projects Possible?

Project 1

Unit A

Unit B

Unit C

Project 2 Project 3

Unit D

Unit B

Unit C

Unit B

Paradigm 1

Paradigm 2

Paradigm 3

time

NVIDIA Confidential

Watch some Real Users

Image courtesy of DAZ Productions

NVIDIA Confidential

How to “Watch” Users

Meetings and Discussions

Coffee-Break Grousing

Bug Reports

Keep Track of Support Requestscreate FAQs

VNC (Remote Desktop)

Instrumentation

NVIDIA Confidential

Build Time Distribution: 10,000 per sample

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

> 1 hour

< 1 hour

< 30 min

< 15 min

< 10 min

< 5 min

< 2 min

< 1 min

< 30 sec

< 20 sec

NVIDIA Confidential

Build Time Distribution: 1000 per sample

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

> 1 hour

< 1 hour

< 30 min

< 15 min

< 10 min

< 5 min

< 2 min

< 1 min

< 30 sec

< 20 sec

Part 2

ESL for Transaction Design

NVIDIA Confidential


Big Paper Spec

Design

Verification

RTL


Scoreboard

Assertions

C Model

Synthesis

Directed

RandomsBFMs,

TLMs


ISS Model

CoverageDebug

(BAD)

NVIDIA Confidential


Big Paper Spec

Design

Verification

RTL


Scoreboard

Assertions

C Model

Synthesis

Directed

RandomsBFMs,

TLMs


ISS Model

CoverageDebug

(BAD)

NVIDIA Confidential

A HW Development Flow (Better)

RTL Scoreboards

Synthesis

Randoms

Clocks, Resets

Design

Verification

Testbench

Small Paper Spec

ISS Model ESL

C Model

Interfaces

Assertions

Directed Tests BFMs

TLMs

Validation

Assertions

Triage

Debug

Coverage

NVIDIA Confidential

Who Writes Assertions?

DesignersBottom Up Assumptions

Verification EngineersTop-down Intent

NVIDIA Confidential

Who Writes Assertions?

DesignersBottom Up Assumptions

Verification EngineersTop-down Intent

ArchitectsThe Specification

Top Down Assumptions

Bottom Up Intent

NVIDIA Confidential

Where to Write Assertions

The RTLInline

Bound

The TestbenchScoreboard

EnvironmentE.g. Post Process Log file

NVIDIA Confidential

Where to Write Assertions

The RTLInline

Bound

The TestbenchScoreboard

EnvironmentPost Process Log files

The SpecificationC Models (?)

NVIDIA Confidential

Where To Write Specification Assertions

FunctionalityModel

PerformanceModel

RTLDesign

NVIDIA Confidential


TransactionModel

PerformanceModel

RTLDesign

CorrelationModel

ISS Model

Debug/TriageModel

NVIDIA Confidential


PerformanceModel

RTLDesign

StructuralModel

TransactionModel

NVIDIA Confidential

Interfaces Vs State

Two approaches to comparing models:

Compare “Architectural State”Registers/flops within the design whose existence is required by the specification

Compare externally visible behaviorCompare interface traffic

B. F. Skinner?

NVIDIA Confidential

Birth of an IDL

Interface Description Language

Initially, a language just to define signals

Interface a2b

clock clk

down U valid 1

up U busy 1

down U cmd 24

down U data 32

NVIDIA Confidential

Evolution of an IDL

Quickly added flow-control protocol abstraction

Interface a2b

clock clk

flow valid_busy

down U cmd 24

down U data 32

From this we can generate:Testbench components (BFMs: producers, consumers)

Protocol Assertions

…

NVIDIA Confidential

Continued Evolution of an IDL

Separation of packet structure from interface

group SOP

down U cmd 24

group MOP

down U data 32

group EOP

down U checksum 32

Interface a2b

clock clk

flow valid_busy

packet SOP, MOP, EOP

NVIDIA Confidential

Transaction Assertions in YACC

valid_interface_traffic:

| valid_interface_traffic transaction;

transaction: begin middle end;

begin: SOP;

middle:

| middle MOP;

end: EOP

NVIDIA Confidential

Cycle Level Assertions in SVA

sequence valid_trans;(cmd==SOP)(##1 cmd != SOP && cmd != EOP) [*0:$]##1 cmd == EOP

endsequence

a_well_formed_transaction: assert @(posedge clk)cmd == SOP |-> sequence (valid_trans)

clk

cmd SOP EOPMOP MOP

NVIDIA Confidential

Transaction Level Assertions in SVA

sequence valid_transaction;

(cmd==SOP)

(##1 cmd != SOP && cmd != EOP) [*0:$]

##1 cmd == EOP

endsequence

clk

valid

cmd SOP EOPMOP MOP

NVIDIA Confidential


event sample;

always @(posedge clk)

if (valid && ! busy) -> sample

assert @(sample)

cmd == SOP |-> sequence (valid_transaction)

clk

valid

busy

cmd SOP EOPMOP MOP

NVIDIA Confidential


event sample;

always @(posedge clk)

if (valid && ! busy) -> sample

assert @(sample)

cmd == BEGIN |-> sequence (valid_transaction)

clk

valid

busy

cmd SOP EOPMOP MOP

NVIDIA Confidential

Grammer using IDL

assert “temporal representation”data SOP => past( EOP )

data MOP => past( SOP | MOP )

data EOP => past( SOP | MOP )

assert “bnf representation”bnf SOP

-> MOP ** past( SOP.length-1 :sample( SOP ) )

-> EOP

NVIDIA Confidential

Multi-Unit Assemblies

A EB C D F G

a2b b2c c2d d2e e2f f2g

A simple pipeline

NVIDIA Confidential


A

E

B C D

F G

a2b b2c c2d

d2ee2f f2g

Simple rearrangement

NVIDIA Confidential


A

E

B C D

F G

a2b b2c c2d

d2ee2f f2g

Identify units with similar behaviors

NVIDIA Confidential


BE

CF

D

a2be

be2cf

cf2dd2be

cf2gG

A

Extract common behavior into unified components

be2cf === b2c + e2f

NVIDIA Confidential

Reusing Interface Definitions

A EB C D F G

D

GA

BE

CF

How to maximize reuse between these two architectures?

NVIDIA Confidential

Packets as Traffic Streams

group b2c

down U data 32

group e2f

down U data 32

Interface be2cf

clock clk

flow valid_credit

packet b2c, e2f

NVIDIA Confidential

Time Units of Temporal Expressions

Group b2cdown U value 4

assert value != past( value )

Group e2fdown U value 4

assert ( value == 0 ) => ( past( value ) != 0 )

Interface be2cfpacket b2c, e2f

assert b2c => ( b2c.value != past( b2c.value :sample(b2c) ) )

assert past( e2f && e2f.value == 0 ) => (b2c && b2c.value != 0)

NVIDIA Confidential

Example

Renderer Memory

NVIDIA Confidential

The Traffic

group mem_write

down U address 16

down U data 1

group sync

down U shape 2

enum SQUARE, CIRCLE, TRIANGE, BLANK

down U radius 3

NVIDIA Confidential

Accumulate Memory State

group mem_write

down U address 16

down U data 1

assign mem[ x = 0 .. 15 ][ y = 0 .. 15 ]

= past( data :sample( address == {x,y} ))

NVIDIA Confidential

Add Predicates

group_more mem_write

assign is_circle[ r = 0..7 ] = “&&”(

[ x = -8 .. 7 ]

[ y = -8 .. 7 ]

mem[ x+8 ][ y+8 ] == ( x**2 + y**2 <= r**2 )

)

NVIDIA Confidential

Define the interface

interface render2memory

clock mem_clk

flow valid_busy

packet sync, mem_write

assert “correct sync shape”

sync && sync.shape == CIRCLE

=> past( mem_write.is_circle[ sync.radius ] )

NVIDIA Confidential

Example

Renderer Memory

assert “correct sync shape”sync && sync.shape == CIRCLE => past( mem_write.is_circle[ sync.radius ] )

NVIDIA Confidential

Summary

Efficient VerificationUnderstand Variation

Encourage Diversityavoid anarchy

Encourage Standardizationavoid stagnation

Frontload The ScheduleProvide Verification with Executable Spec (ESL)

C Model, Correlation Model (pre validated)

Interfaces, Transactions (definitions for code generators)

Assertions, Testpoints (neutral language, pre validated)

Questions

http://dave.whipp.name/dv



NVIDIA Confidential

Summary

Architects should write assertionsValidated assertions are input to Verification

Assertions must be directly reusable across modelsManual recoding invites errors

Explicitly model the structure that is common to architectural models and to design

Tie assertions to these common points

NVIDIA Confidential

Threading Models

Thread Follows ResourceTraditional SystemC approach

Thread Follows TransactionThink “Multithreaded ISS”

Cycle-based ModelThe “Old” way of doing things

A non-threaded version of “thread follows resource”

Different models are convenient at various timesBut SystemC forces you to commit early to one approach

Therefore need extra code for transaction level debug

NVIDIA Confidential

70%

70% of schedule is Verification

What is correct figure

0%synthesize architectural models, constraints

Correct by construction!

100%synthesize verification models, constraints

no design needed!

NVIDIA Confidential

Transactions Vs Cycles

Data min_val (Addr a1, Addr a2){ Data d1 = mem_read(a1); Data d2 = mem_read(a2);

if (d1 < d2)return d1;

elsereturn d2;

}

Address

Data

a1

d1

Pipelined Bus

a2

d2

t1 t2 t3 t4 t5 t6

NVIDIA Confidential

Unit Level System Testing

Stage N-1

Stage N+1

Stage N

Graphics Pipeline (Transaction Model)

RTL Unit DIFFDIFF

NVIDIA Confidential

New Challenges

New Tools

New Platforms

New People

New Ideas

Refactoring

Testability

D.R.Y.

Avoiding Stagnation

Stop Writing Assertions! Efficient Verification Methodology Dave Whipp – DVClub SV, August 2008.

Documents

nvidia confidential

time slide

nvidia confidential

nvidia confidential

sample slide

formal slide

agitation slide

nvidia confidential