1 SoC Verification Strategies for Embedded Systems Design November 5-6, 2003/ Seoul Chong-Min Kyung, KAIST SOC Design Conference
1
SoC Verification Strategies for Embedded Systems Design
November 5-6, 2003/ Seoul
Chong-Min Kyung,
KAIST
SOC Design Conference
2
Various Embedded Mobile Systems
Desktop PC
Data Processing Consumer
Communication Automotive & etc.
DTV
MediaCenter
GameConsole
DSC
DVC
Car Navigation
DVD
Telematics
MP3 Player
Credit Card
Cellular Phone
Notebook PC
Internet
Smart Phone PDA
Bluetooth
Audio
3
Difficulties in Embedded Systems Design(What‟s special in ES design?)
Real operating environment of ES is difficult to reproduce. -> In-system verification is necessary.
Total design flow of ES until the implementation is long and complex. -> Verification must be started early.
Design turn-around time must be short as the life-time of ES itself is quite short. -> Short verification cycle
4
Three Most Important Issues in Verification
1. Verify Right :
Always make sure you have correct specifications to start with. (Frequent interaction with SPECIFIER, customer, marketing, etc.)
In-System Verification
Check Properties in Formal Techniques.
2. Verify Early
System-level, Heterogeneous Models, SW-HW
3. Verify Appropriately
HW-SW Co-simulation
4. Verify Fast
Hardware Acceleration, Emulation
5
Strongly Required Features of Verification
Accommodate Multiple Levels of Design Representation
Exploit Hardware, Software and Interfacing mechanisms as Verification Tools
6
Virtual Chip : “Verify Early, In-System”
Virtual Chip: Making Functional Models Work on Real Target System [DAC98]
Simulating ISS of a Processor Chip along with real target environment
Target board
Host computeras Virtual Chip cable
Pin Signal Generator with Buffers
Chip Model;ISS
Chip Socket
7
PC
running
ISS
Without
CPU
Picture taken in 1993 or 1994
8
Reducing TTM using Virtual Chip
idle
Conventional design flow
Architectural
model
RTL
model
Gate-level
model
H/W
Emulation
Verification w/ H/W
H/W prototype
(H/W emulation)
Board
design
H/W System
Application
S/W
idle
design
Virtual Chip design flow; EARLY ,IN-SYSTEM
Architectural
model
H/W prototype
(Virtual Chip)
Board
design
H/W System
design
RTL
model
Gate-level
model
H/W
Emulation
Verification w/ H/W
Application
S/W
Design time is
drastically reduced
9
Microprocessor Design Verification Methodology
Instruction
Behavior
In C
(Polaris)
Micro-
architecture
in C
RTL
Verilog
Gate-Level
Verilog
Real
Mother-board
H/WVirtual PC in C (VPC)
C Language
Target Environment
HDL
MCV : Microcode Verifier PLI : Programming Language Interface
more refined model
MCVFlexPCVirtual Chip PLI
CPU Model
10
HK386 (1995)
Design Specification Instruction level, Pin-to-Pin compatible with
i386 Operation speed : 40 MHz 0.8 m DLM CMOS ASIC
Test Programs MS DOS 6.0, Windows 3.1, Office 4.0 CAD tools, games, etc..
MS Win. 3.1 MS Office MaxPlus II
11
What about Emulation?
Swallowed a lot of Money
Required a separate, air-conditioned room
Needed a lot of time for Compile
Needed an expensive slowed-down PC ($20,000 for 500kHz clocking)
However…worked at the last minute
Finding bugs is not easy
12
x86 Emulation Configuration
HardwareEmulator
500kHz Slowed-Down PCProbeModule
Target Interface
Board
13
Booting Windows 20M instructions on Marcia
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
0 5 10 15 20 25 31 36 41
Time (weeks)
Instr
uctions (
thousand)
HDL saver
Attached
Windows
DOS
HDL Simulation Hardware
Emulation
setup
version
update 1
version
update 2
version
update 3
Simulation debugs,
Emulation approves.
14
Agenda
Verify Early and Altogether (In-System)
Why Verification ?
Verification Alternatives
Verification with Progressive Refinement
SoC Verification
Concluding Remarks
15
Trend of Verification Effort in the Design
Verification portion of design increases to anywhere from 50 to 80% of total development effort for the design.
Code Verify (30 ~ 40%)Synthesis P&R
Code Verify (50 ~ 80%) Synthesis P&R
1996300K gates
20001M SoC
Verification methodology manual, 2000-TransEDA
16
Percentage of Total Flaws
About 50% of flaws are functional flaws.
Need verification method to fix logical & functional flaws
From Mentor presentation material, 2003
Clocking
5%
Race
5%
Power
4%
Other
9%
Yield
7%
Noise
12%Slow Path
13%
Logical/
Functional
45%
17
Another recent independent study showed that more than half of all chips require one or more re-spins,
and that functional errors were found in 74% of these
re-spins.
With increasing chip complexity, this situation could worsen.
Who can afford that with >= 1M Dollar NRE cost?
18
Bug Fixing Cost in Time
Cost of fixing a bug/problem increases as design progresses.
Need verification method at early design stage
Verification methodology manual, 2000 – TransEDA
BehavioralDesign
RTLDesign
Gate LevelDesign
DeviceProduction
Cost ofFixing
a Problem
19
Verification Performance Gap; more serious than the design productivity gap
Growing gap between the demand for verification and the simulation technology offered by the various options.
System-on-a-chip verification, 2001 – P.RashinkarDesign complexity
Sim
ula
tio
n p
erf
orm
an
ce
Ve
rifi
ca
tio
n c
om
ple
xit
y
Verification Performance Gap
SmallASIC
MediumASIC
ComplexASIC
SOC
20
Completion Metrics; How do we know when the verification is done?
Emotionally, or Intuitively;
Out of money? Exhausted?
Competition‟s product is there.
Software people are happy with your hardware.
There have been no bugs reported for two weeks.
More rigorous criteria;
All tests passed
Test Plan Coverage
Functional Coverage
Code Coverage
Bug Rates have flattened toward bottom.
21
Verification Challenges
Specification and/or Operating Environment is Incomplete/Undetermined. (Just like last-minute ECO!)
Cannot avoid using Yesterday‟s tool for Today‟s Design.
Design productivity grows faster than Verification productivity. (Verification productivity Gap)
Besides Functional Verification, Estimation on Performance, Power Consumption becomes crucial issues in SoC design.
22
Agenda
Verify early and in-system
Why Verification ?
Verification Alternatives Simulation
Hardware-accelerated simulation
Emulation
Prototyping
Formal verification
Semi-Formal (Dynamic Formal) verification
Verification with Progressive Refinement
SoC Verification
Concluding Remarks
23
Overview of Verification Methodologies
Simulation
Hardware
Accelerated
Simulation
Emulation
Formal
Verification
Semi-formal
Verification
Prototyping
Basicverificationtool
24
Software Simulation
Pros
The design size is limited only by the computing resource.
Simulation can be started as soon as the RTL description is finished.
Set-up cost is minimal.
Cons
Slow (~100 cycles/sec) ; Speed gap between the speed of software simulation and real silicon widens. (Simulation speed = size of the circuit simulated / speed of the simulation engine)
The designer does not exactly know how much percentage of the design have been tested.
25
Hardware-Accelerated Simulation
Simulation performance is improved by moving the time-consuming part of the design to hardware.
Usually, the software simulation communicates with FPGA-based hardware accelerator.
Simulation environment
Testbench
Module
0
Module
1
Module 2
Hardware
Accelerator
Module 2 is
synthesized &
compiled into
FPGAs
26
Hardware-Accelerated Simulation
Pros
Fast (100K cycles/sec)
Cheaper than hardware emulation
Debugging is easier as the circuit structure is unchanged.
Not an Overhead : Deployed as a step stone in the gradual refinement
Cons (Obstacles to overcome)
Set-up time overhead to map RTL design into the hardware can be substantial.
SW-HW communication speed can degrade the performance.
Debugging of signals within the hardware can be difficult.
27
Emulation
Imitating the function of another system to achieve the same results as the imitated system.
Usually, the emulation hardware comprises an array of FPGA‟s (or special-type processors) and interconnection scheme among them.
About 1000 times faster than simulation.
Simulation
Hardware
Accelerated
Simulation
Emulation
Prototyping
28
Emulation
Pros
Fast (500K cycles/sec)
Verification on real target system.
Cons
Setup time overhead to map RTL design into hardware is very high.
Many FPGA‟s + resources for debugging high cost
Circuit partitioning algorithm and interconnectionarchitecture limit the usable gate count.
29
Emulation
Challenges
Efficient interconnection architecture and Hardware Mapping efficiency for Speed and Cost
RTL debugging facility with reasonable amount of resource
Efficient partitioning algorithm for any given interconnection architecture
Reducing development time (to take advantage of more recent FPGA‟s)
30
Prototyping
Special (more dedicated and customized) hardware architecture made to fit a specific application.
Simulation
Hardware
Accelerated
Simulation
Emulation
Prototyping
31
Prototyping
Pros
10X higher clocking than emulation
Components as well as the wiring can be customized.
Can be carried and dispatched for demo or customer evaluation
Cons
Not flexible for design change
(Even a small change requires a new PCB.)
32
Overview of Verification Methodologies
Formal verification
Application of logical reasoning to the development of digital system
Both design and its specification are described by a language in which semantics are based on mathematical rigor.
Semi-formal verification
Combination of simulation and formal verification.
Formal verification cannot fully cover large designs, and simulation can come to aid in verifying the large design.
SimulationFormal
Verification
Semi-formal
Verification
More complete verification
33
Formal Verification
Classes Model Checking : verifies that the design satisfies properties
specified using temporal logic
Equivalence Checking
Symbolic Simulation
Theorem Proving
Pros Assures 100% coverage.
Run fast.
Cons Works only for small-size finite state systems due to memory
explosion. (so far, <500 flip-flops)
Uncomfortable, i.e., need to learn temporal logic for “property” description in Model Checking
34
Efforts by Formal People to reduce the Complexity
Reachability analysis
Design state abstraction
Design decomposition
35
Semi-Formal Verification - Assertion
Assertion-based verification (ABV)
“Assertion” is a statement on the intended behavior of a design.
The purpose of assertion is to ensure consistency between the designer‟s intention and the implementation.
36
Semi-Formal Verification - Assertion
Simulation Quality of assertion-based verificationN
um
be
r o
f b
ug
s f
ou
nd
Time, EffortSetup
testbenchDescribe
assertions
Formal verificationSimulation
Simulation with assertionsEfficiency of
assertion
By IBM in “Computer-Aided Verification” 2000
37
Semi-Formal Verification - Coverage
Coverage-directed verification
Increase the probability of bug detection by checking the „quality‟ of stimulus
Used as a guide for the generation of input stimulus
Test Plan
(Coverage
Definition)Directives
Random
Test
GeneratorTest Vectors
SimulationCoverage
analysis
Coverage
Reports
38
Semi-Formal Verification
Pros
Designer can measure the coverage of the test environment as the formal properties are checked during simulation.
Cons
Simulation speed is degraded as the properties are checked during simulation.
Challenges
Guiding the direction of test vectors to increase the coverage of the design.
Development of more efficient coverage metric to represent the behavior of the design.
39
Speed Comparison
0 kHz
Software
Simulation
10 kHz
1MHz
Hardware-
Accelerated
Simulation
(from
Quickturn/Dynalith
Presentation)
Hardware
emulation
(from Quickturn
presentation)
100kHz
500KHz
100Hz
100 kHz
Speed (Cycles/sec, log scale)
10MHz1~10MHz
Prototyping Semi-formal
(Assertion-
based
verification)
50-70Hz
40
Verification Time vs. Coverage (not to exact scale): Which best fits your application?
Coverage
Verification Time
Simulation
Semi-formal
Prototyping
Emulation/Simulation Acceleration
Simulation setup
Semi-formal setup
Emulation/SA setup
Prototyping setup
41
Agenda
Verify early and in-system
Why Verification ?
Verification Alternatives
Verification with Progressive Refinement
Flexible SoC verification environment
Debugging features
Cycle vs. transaction mode verification
SoC Verification
Concluding Remarks
42
Criteria for Good SoC Verification Environment
Support various abstraction levels
Support heterogeneous design languages
Trade-off between verification speed and debugging features
Co-work with existing tools
Progressive refinement
Platform-based design
43
Core model
ISS
Transaction-Level Modeling
Model the bus system in transaction-level
No notion of exact time.
But precedence relation of each functional block is properly modeled.
Rough estimation on performance is possible.
Used as the fastest reference model by each block designer
Memory IP IP
Transaction-level Bus model
Rough
Performance
Estimation
...
...
Behavior
model
---
---
---
---
---
---
Software design Hardware design
transaction
44
Core model
ISS
Cycle-Accurate Bus Modeling
For more accurate modeling
Build a cycle-accurate system model in C or SystemC
Replace the transaction-level bus model with a cycle-accurate bus model
ARM released a “Cycle-Level Interface Specification” for this abstraction level.
Memory
IP
IP
Cycle-accurate Bus model
Accurate
Performance
Estimation
...
...
Behavior
model
---
---
---
---
---
---
transaction
BFM BFM BFM
RTL
model
45
AMBA AHB CLI Specification
AMBA AHB Cycle Level Interface (CLI) Specification
CLI spec defines guidelines for TLM of AHB with SystemC.
Interface methods
Data structures
Header files for SystemC models
CLI spec leaves the detailed implementation of the AHB bus model to the reader.
master
slave
master
HADDR
HWRITEHRDATA
HTRANS
slave
Read 10 data from slaveX startingAddress 0x10
Cycle-level modeling Transaction-level modeling
46
AMBA AHB CLI Specification
Example master implementation Transactions are represented as methods in transaction-level modeling.
Abstraction levels of each method can be decided as needed such as cycle-count accurate, cycle-accurate, transaction accurate, etc.
void masterA::run(){bus_request();has_grant();init_transaction();read_from_slave();...
}
Header Body
The granularity of transactions are decided according to the verification needs.
SC_MODULE(masterA) {......void run();
};
masterA.h masterA.cpp
47
Flexible SoC Verification Environment
Build C reference model for the target application.
Setup of platform-level verification environment as early as possible
JPEG decoding
HEAD VLD IDCT DISP
Verific
atio
n E
nviro
nm
ent S
etu
p
Functional block model
TRS TRS TRS TRS
HEAD VLD IDCT DISP
Platform-level model
Algorithm
Memory Timer INTC
SW Model
HW Model
48
Flexible SoC Verification Environment
Transactor connects various types of SW block models with HW bus system.
Several types of transactors must be prepared, for example
AHB C model
AHB HDL model
OPB Testbuilder
PLB SystemC
JPEG decoding
HEAD VLD IDCT DISP
Verific
atio
n E
nviro
nm
ent S
etu
p
Functional block model
TRS TRS TRS TRS
HEAD VLD IDCT DISP
Platform-level model
Algorithm
Memory Timer INTC
49
Flexible SoC Verification Environment
Socketize IP representation
HW: C HDL EDIF
SW: native C ISS Processor Core
C
Transactor
HDL
AHB
C to HDL
EDIF
Synthesis
Transactor
C
AHB
ISS
AHB
Processor
Cross-compiler
Transactor
Transactor
Transactor
SW partmodel
HW partmodel
50
Cycle-Level Transactor
Generate stimulus at every clock cycle
Check the result of DUT at every clock cycle
PCIController
DUTPCIChannel
Testbench
Testbench DUT
FPGA part
DeviceDriver
S/W simulation part
Cycle-leveltransactor
51
Transaction-Level Transactor
Only information to generate transaction is transferred to DUT, i.e., address and data
No need to synchronize at every clock cycle
PCIController
DUTDMAChannel
Testbench
Testbench DUT
FPGA part
DeviceDriver
S/W simulation part
Tra
nsacto
r
MainMemory
52
Cycle vs. Transaction-level Transactor
Cycle-level transactor
Synchronized at every clock cycle.
Operating speed depends on the number of signals to be transferred.
Transaction-level transactor
Synchronized at the end of each transaction.
Transactor must be designed for each interface standard
ex) AHB transactor, SDRAM transactor, IIS transactor
53
DFV (Design for verification) : OpenVera (OV) verification IP
Reusable verification modules, i.e.,
1) bus functional models,
2) traffic generators,
3) protocol monitors, and
4) functional coverage blocks.
54
RTL Debugging Feature
Gate-level netlist generated from the synthesis tools is difficult to trace manually.
Techniques to resolve RTL symbol names from the gate-level symbol names and to provide debugging environment in RTL name spaces is required.
Insert RTL instrumentation IP for debugging
Design flow
Read RTL design (Verilog, VHDL)
Generate instrumented RTL design (spiced with triggering and dump logic)
Synthesis
Compile (mapping & PAR)
DiaLite (Temento), Identify (Synplicity)
55
Synthesizable Testbench
Prepare a set of programmable test bench module which can be also synthesized into hardware.
To verify a DUT, build a test bench using the programmable test bench.
The test bench is applicable to both simulation and emulation in the same fashion.
from Duolog TechnologGeneric Mini-Controller(GMC)
gmc_cfg_datasize 3 # Set to 32 Bits
gmc_cfg_msmode 0xF # Set to Master TX
gmc_cfg_datapath_0x00 # Read/Write to Mem
gmc_cmd_send_data 100 # Send 100 Bytes
uart_cfg_odd_parity 1 # Set ODD parity
56
Agenda
Verify early and in-system
Why Verification ?
Verification Alternatives
Verification with Progressive Refinement
SoC Verification
Co-simulation
Co-emulation
Concluding Remarks
57
SoC Verification
Co-simulation
Connecting ISS with HDL simulation environment
Seamless, N2C
Co-emulation
Emulation/rapid-prototyping equipments supporting co-emulation
ARM Integrator, Aptix System Explorer,
AXIS XoC, and Dynalith iPROVE
58
What‟s the point in SoC Verification?
Mixture of SW and HW
Let the HW model to cooperate with Processor Model such as ISS or BFM (Bus functional model)
Mixture of pre-verified, unverified components
Utilize legacy IPs already verified
Mixture of different language, different abstraction levels
Provide common interface structure between SoC components
59
Canonical SoC design flow
SystemSpec.
SystemDesign
HW/SWPartitioning
HWDevelopment
SWDevelopment
HW refinement(UT->T->RTL)
Gate
HW IP
SW IP
SoftwareVerification
FunctionalVerification
Gate-LevelVerification
HW-SWCo-Design
HW-SWCo-
Verification
SW refinement(RTOS
mapping)
Final code Emulator
In-system emulator
HW-SW co-debugging
60
Tools for HW-SW Co-Verification
HW-SW co-simulation
ISS
RTOS simulator
HW/SWPartitioning
HWDevelopment
SWDevelopment
HW refinement(UT->T->RTL)
SoftwareVerification
FunctionalVerification
Co-Verification
SW refinement(RTOS
mapping)
HW-SW
High-level synthesis
Testbench automation
IP accelerator
SystemSpec.
SystemDesign
HW/SW
HW IP
SW IP
61
Tools for System-level Verification
System-level design (Performance analysis tools)
Hot-spot analyzer
High-level cycle count estimation
High-level power analysis
High-level chip area estimation
On-chip-bus traffic estimation
SystemSpec.
SystemDesign
HW/SWPartitioning
HW IP
SW IP
HW-SWCo-Design
62
Co-Simulation Tools
Software debugging in ISS and hardware verification in HDL simulator are done in parallel way.
Co-simulation stub manages the communication between HDL simulator and ISS.
The most accurate solution albeit very slow
ISSHDL
Simulator
Co-simulationStubDebugger
RemoteDebuggingInterface
63
Co-Emulation Tools
Link hardware emulators with a processor model running in host machine or an actual processor core module
Most emulation and rapid prototyping products support linkage with ISS running in host machine
As the emulator runs very fast, speed of ISS including memory access as well as synchronization between emulation and ISS rise as new bottlenecks in verification
64
Example) ARM ETM (Embedded Trace Macrocell)
Real-time trace module capable of instruction and data tracing dedicated to the ARM core family
Triggering enables to focuscollection around the region of interest
Trace module controls theamount of trace data by filtering instructions or data
Only applicable to the ARMcore debugging
ARMcore
Memories Peripherals
Oth
er
com
ponents
ETM
Target system with ARM based ASIC
PC-based debugger
Trace
Trigger
Traceport
analyzer
JTAGinterface
Conf.
65
Typical Co-Emulation Environment
Connect ARM ISS or ARM board to the emulation system.
ARM ISSEmulationSystem
ARMDevelopment
Board
EmulationSystem
ARM ISS can be configured to user‟s target SoC architecture.
SW debugger can be fully utilized.
Faster than ISS.
Ready-made ARM development boards has fixed architecture which may be different from the user architecture.
66
Example 1: ARM Integrator
ARM prototyping platform
Composed of
Platform : AMBA backbone + system infrastructure
Core Module : various ARM core modules (up to four)
Logic Module : FPGA boards for AMBA IP‟s
Allows fast prototypingof ARM-based SoC
Enables co-emulationof both software in ARMprocessor core andhardware in the FPGA
Difficult to debug hardwarelogic
from ARM
67
FPGA
Hardware Sessions
C/SystemC/HDL sessions
HardwareProcessor
Logic Designin EDIF/FPGA
Transactor Transactor
Target board
I/F protocol I/F protocol
Inter-Lingual Communication
Designin C
Example 2 : iSAVE/iPROVE (Dynalith)
All-in-one verification :
Heterogeneous models including ARM ISS, C hardware model, HDL hardware description
SW models run in Linux-based PC
HW models run in FPGA‟s, or
as a processor chip
HW/SW Co-debugging with
Pin Signal Analyzer
On-the-fly probing of FPGA
internal signals to SRAM
Communicate with C modelusing PCI DMA
ISS sessions
ISSDesignin SC
Designin HDL
68
Agenda
Verify early and in-system
Why Verification ?
Verification Alternatives
Verification with Progressive Refinement
SoC Verification
Concluding Remarks
69
So, in Summary…
Verification is challenging; It needs strategy!
Strategy is to apply each method when appropriate.
Verify with Correct Specification
In-System Verification
Verify as early as possible; Kill the bug when it is small and still isolated in a smaller region. (Kill it before it grows big and kills you(r design and your company).)
Accelerate the verification cycle using
Efficient debugging tools
Hardware Accelerator
70
A Tactical Sequence of Applying Various Verification Tools
1st step: Apply formal methods
Static formal verification
Assertion-based verification
2nd step: Simulate IP
with transaction level test-bench
with cycle-accurate test-bench
3rd step: Emulate design
Emulate IP operation in FPGA
In-system IP verification
Cycle-level vs. transaction level test-bench
71
Concluding…
Main differences of SoC with ASIC design are
Planned IP-reuse
Reuse of pre-verified platform
Focus on co-verification with software
Newly added IP‟s must be thoroughly verified before its application to SoC.
Well-established verification platform allows progressive verification/refinement of each IP, i.e., each SoC component while assuring correct implementation.
Powerful HW/SW co-simulation and co-debugging features are required.
72
Thank you for your kind attention!