- Verifying “Golden” reused IPs The Evil’s in the Edits William C Wallace Texas Instruments Nitin Jayaram Texas Instruments Nitin Mhaske Atrenta Inc Vijay.

- Verifying “Golden” reused IPs

The Evil’s in the EditsThe Evil’s in the Edits

William C WallaceTexas Instruments

Nitin JayaramTexas Instruments

Nitin MhaskeAtrenta Inc

Vijay KanumuriTexas Instruments

Author DetailsWilliam C Wallace:

Senior Digital Designer, Texas Instruments

20+ years of experience, focused on using new methods to reduce risk and increase productivity when making modification on legacy IP

Nitin Jayaram:

Lead DV engineer, Texas Instruments

7+ years of experience, focused on new methods to increase DV productivity

Nitin Mhaske:

Sr Manager, Atrenta Inc

[email protected]

13+ years of experience, leading verification for Semis and EDA

AbstractTime To Market makes IP reuse imperative in TI’s SoCs with expectations that reused IP development should be less efforts and less risks. However, most of the IPs need to go through some edits to meet current marketing requirement or meet constraints such as power, and timing. Lack of micro-architecture knowledge, poor documentation and rudimentary testing for reused IPs makes verification of new/modified RTL challenging and time consuming. Traditional quality control techniques fails to capture dependencies of these IP edits, risking the SoC.

Assertion Synthesis techniques that learns design behavior based on simulation tests and RTL was used to solve the problem. These techniques automate process of finding functional coverage holes for the IP edits and guided test development to better coverage. With limited design knowledge, these coverage holes are harder to find using traditional coverage techniques. Assertions generated out of these techniques made sure that design edits did not affect the un-modified RTL and respected the SoC requirements.

Backdrop

SOCs are Heterogeneous

TI - SoC

Typical Internally Reused IP

Typical Third Party Reused

IP

The Problem Statement

Challenges With Reused IPs

Almost never satisfies “current” SoC marketing requirements

Supported features, configurations

SoC specific Power Intent Other constraints such as

timing, area etc

Original (Micro-architecture) design Intent lost over time

Original developers not available

Changed hand multiple times at multiple locations

Poorly maintaineddocuments

How much verification good enough?(Any design edits may affect many features)

Existing Env often directed;so cannot trust quality

No standardization like UVM; makes reuse harder

Lacks functional Coverage, assertions

Verification Efforts For Reused IPsExpectations: Reused IP should consume less time/resources

“Many unknowns” makes test development daunting Modification of existing Env is challenging and time

consuming Regressions of un-modified feature tests may not work as-is

(due to latency changes etc)

Poor documentation, lack of localized assertion makes debugging tougher

Summary: Verification of reused IPs, if not planned properly, may take as much efforts as new IP

Standard Quality Metrics Not Effective

IP sign-off to SoC primarily based on two metrics Code coverage Functional coverage

Code coverage consume lot of time for Reused IPs Deprecated Features Waivers Modification

Functional coverage monitors are subjective, error prone DV writes high level coverage monitors for modified/new features Important cross coverage due to micro-arch. dependencies of IP

edits is difficult to plan and implement

Need For Improving The Methodology Quality: Need automated checkers and coverage to find bugs at IP level; otherwise may be found during late stage in SoC verification.

Productivity: Need guidance to test development Verifying reused IP from scratch is not practical Coverage associated to IP edits and their dependencies highly

important

Other verification techniques do not work Formal Model checking difficult to setup and runtime too long Sequential equivalence does not understand functional impact

Assertion Synthesis techniques and

how to apply them

Solution: Assertion Synthesis(Generates assertions and coverage automatically from tests + RTL)

Assertion Synthesis

Un-modifiedModules

New/ModifiedModules

IP RTL

Legacy Tests

New Tests

IP Verification Env

1 to 2 cycle properties generated by analyzing all tests specific to control modules

Trivial un-reachable properties removed automatically

Redundant properties from others removed automatically

Properties divided into different levelsL0 (single signal), L1 (Two signal) and L2(rest)

“Feature_A == 1’b0” Tests never covers feature_A set“!(stall && wr) Test never asserts stall and wr same cycle“stall |->@cs == IDLE” In all Tests, when stall == 1, next cycle cs is always idle

If signal “stall” is tied low in RTL, property “stall == 1’b0” or “!(stall && wr)” removed automatically

If “feature_A == 1’b0”, redundant low imp properties such as “!(feature_a && feature_b) removed autmatically

L0 Properties: onehot0(sel), cnt < 4’ha L1 properties: !(rd && wr)

Properties capturing design assumption/intent categorized as assert; otherwise as cover

If design does not support rd & wr in the same cycle “!(rd && w)” is an assertion. Otherwise captures TB limitations; so a coverage

Methodology on Reused IPs

SoCM1

M4

IP1 – RTL(Reused)

M2

M3

IP2-RTL

IP3-RTL

M5

M6

• Module M4 being new/modified

• Rest of the modules in IP1 unmodified

• IP1 has legacy verification env+tests

Before modification in IP1 starts, run all legacy tests and generate properties for unmodified modules M1, M2..

All properties converted as assertions automatically w/o review. Assumption is that for unmodified RTL

legacy tests has complete coverage; no coverage holes

Integrated the properties back to IP simulations. So any assertion that fires indicates changes in module M4 that violation assumption of unmodified RTL; hence find bugs

Once sufficient tests are written for module M4, generated properties using all tests for M4 and review

Classified assertions are added back into simulation. Coverage holes targeted w/ additional tests.

When IP coverage matures(holes taken care), integrate all IP assertions into SoC simulations

SoC Phase# 1

Any assertion firing can indicate design bug or limitation of IP tests (did not cover fully as SoC intends too)

Phase# 2

Phase# 3

Example Property Stats From 2 IPs

10-15 Lines/Property for control intensive modules

20-50 Lines/Property for datapath intensive modules

Performance Numbers PLI overhead during simulation when collecting properties 10-15% When assertions are added back into simulation, overhead 5-10%

Classification of properties : Avg. 1 min per property

Module RTL Lines

Properties Lines/Property

L0(1 signal)

L1(2 signals)

L2(>2 signals)

sch_rd ~1800 19 55 50 ~14

sch_wr ~1500 15 60 62 ~11

galois_eng_pool ~3000 6 29 12 ~50

rtc_vbusp ~1000 18 22 7 ~20

rtc_func ~250 4 3 2 ~25

Assertions Found and Impact

Property: mrd_aes_data_in_index != mwr_aes_data_in_index

Captures micro-architecture design intent: rd and wr index generated in the RTL based on algorithm; Should never be the same.

Property: mrd_aes_data_valid == 1’h1 |-> !isunknown(mrd_aes_data)

Captures X propagation: If data driven unknown in SoC, it would localize failure

Property:!(rd_mac_start && rd_mac_data_out_pop)

Captures code coverage waivers: Known coverage waiver; but an assertion will catch that it is truly not hit in any IP tests or SoC in any corner scenario

Assertion Synthesis captures design usage assumption and intent. Integrated Assertion in IP simulation and SoC simulation

Coverage Holes Found and impact

Property: interrupts <= 3'h5

Finds holes missed by code coverage: Property indicates that higher values of interrupts possible; but missed. This hole could not be found by toggle coverage as all bits toggled

Property: mutex(cache_enable, roll_2k_page_active)

Helps to understand & code micro-arch dependent functional coverage: Cache enable and roll_rk_page_active can be high in the same cycle; but DV missed important dependency

Property: emif_SDataAceept |-> n_mac_seq != MAC_WRT_CMD_STALL

Finds BFM limitation: Data accept high and command accept low can happen in stall; but BFM did not support this functionality

Assertion Synthesis discovered individual and cross coverage holes missed Analysis of coverage holes was efficient due to removal of un-reachable and

redundant coverage holes Guided our future test development based on the coverage holes found

Risk mitigation by Reusing Properties in SoC

Assertions generated using IP tests are directly ported/bound to SoC After review for modified RTL and W/O review for unmodified RTL

When assertions fire in SoC tests, it indicates either of following Design bug: IP assumption/intent violated in SoC Misconfiguration: IP is not used correctly in SoC; saves debug time IP Coverage holes: IP is missing important coverage; important to particular SoC

Findings When properties from IP used in SoC Context and no assertions fired implies:

SOC did not violate an IP requirement and/or SOC did not exercises a unique state-space that IP DV did not cover

Performance overhead of running assertions in SoC was comparable as IP (5-10%)

IP Tests Simulation +

Assertion Synthesis

Assertion Properties

(Reviewed+ non-reviewed)

SoC TestsSimulation

Assertion Fires

Assertion Holds True

Backup

Conclusions & Next Steps

Assertion synthesis guides test development for reused IPs by finding important functional coverage holes

Integrating assertions generated using Assertion Synthesis back to IP and SoC simulations give extra quality assurance

To-Do Assertions generated from IP need to be ported to SoC Emulation

Env Find out how effective flow can be for even third party IP

Lessons Learned Some learning curve involved in understanding properties Aggravated due to less knowledge of RTL Some properties need couple of minutes to review; improves over

time

Review & target properties based on Level (Lx) to reduce property review time and prioritize test development

Deliver properties to SOC after coverage matures at IP; not early

DV should still code high level functional coverage for modified/new features Co-relates to spec better & complement micro-architecture ones

from Assertion Synthesis