Page 1
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 1/76
Industry Pulse:
Trends in Functional Verification
Chief Scientist Verification
Design Verification Technology
Memo CODE 2013
Harry Foster
Page 2
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 2/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Extrapolating From Current ConditionsDisregards Future Innovation
HF, MemoCODE, 20132
“In 1910, in the early history telephony, a Bell telephone statistician
projected a massive ramp-up in switchboard operator jobs as telephoneuse grew, until “every woman in America” would be required.”
Source: Future Savvy: Identifying trends to Make Better Decisions, Manage Uncertainty, and Profit From Change Adam Gordon, 2008
Page 3
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 3/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Functional Verification Market According to EDAC
700
965
0
200
400
600
800
1000
2010 2012
M i l l i o n s ( $
)
HF, MemoCODE, 20133
38% Growth Between 2010 2012
EDAC: Market Statistics Service 2007 Annual Summary Report
Page 4
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 4/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Functional Verification Market According to EDAC
380430
190
365
130
170
0
100
200
300
400
2010 2012
M i l l i o n s
( $ )
Simulation 13% GrowthEmulation 94% Growth
Formal 31% Growth
HF, MemoCODE, 20134
EDAC: Market Statistics Service 2007 Annual Summary Report
Page 5
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 5/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
HF, MemoCODE, 20135
Conducted by Wilson Research Group
— Commissioned by Mentor Graphics
— Format followed 2002, 2004 Collett studies for trend analysis, as
well as the 2007 FarWest Research Study
Worldwide study
— Overall confidence of 95% plus/minus 4.05%
This was a blind study!
— To eliminate any bias in the results
This was a balanced study!
— No single vendor dominated responses
Wilson Research Group
2012 Wilson Research GroupFunctional Verification Study
Page 6
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 6/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Who Participated In The SurveyParticipant’s market segment
HF, MemoCODE, 20136
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
0%
10%
20%
30%
40%
50%
60%
ASIC Vendor Fabless IC Vendor
ICManufacturer
SystemsCompany
DesignServicesCompany
IP VendorCompany
Other
S t u d y P a r t i c i p a
n t s
Non-FPGA 2012
FPGA 2012
Page 7
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 7/76www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Who Participated In The SurveyParticipant’s job title
HF, MemoCODE, 20137
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
0%
10%
20%
30%
40%
50%
60%
S t u d y P
a r t i c i p a n t s
Non-FPGA 2012
FPGA 2012
Page 8
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 8/76www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Overview
Beyond Theory
Beyond Standards
Beyond the Status Quo
HF, MemoCODE, 20138
Page 9
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 9/76www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
BEYOND THEORY
Beyond Theory in Terms of Rising Complexity
HF, MemoCODE, 20139
Page 10
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 10/76www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Difference Between Theory and Practice
HF, MemoCODE, 201310
In theory there is no difference betweentheory and practice, but in practice there is.
Page 11
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 11/76www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Difference Between Theory and Practice
HF, MemoCODE, 201311
Theory: Everything is clear, but nothing works.
Page 12
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 12/76www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Difference Between Theory and Practice
HF, MemoCODE, 201312
Practice: Everything works, but nothing is clear.
Page 13
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 13/76www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Difference Between Theory and Practice
HF, MemoCODE, 201313
The problem is sometimes theory meets practice:
Nothing works and nothing is clear.
Page 14
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 14/76www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Beyond Theory in Terms of Rising Complexity
HF, MemoCODE, 201314
What does this really mean?
What makes things complex?
How do we measure complexity?
Page 15
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 15/76www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
What Makes Something Complex?
HF, MemoCODE, 201315
System consisting of many interconnected parts — Examining the individual parts tells you nothing about the system
Complex does not necessarily mean complicated
Page 16
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 16/76www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Designs are Getting More Complex
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
0%
5%
10%
15%
20%
25%
30%
35%
40%
S t u d y P
a r t i c i p a n t s 2007
2010
2012
2007: Mean 90nm2010: Mean 65nm2012: Mean 45nm
Process Geometry
16
Page 17
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 17/76www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Designs are Getting More Complex
HF, MemoCODE, 2013
0%
5%
10%
15%
20%
25%
30%
N
o n - F P G A S t u d y P a r t i c i p a n t s
2002
2007
2010
2012
About a 1/3rd of designs below 5M gates About a 1/3rd of design between 5M - 20M gates About a 1/3rd of designs great than 20M gates
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
Number of gates of logic and datapath, excluding memories
17
Page 18
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 18/76www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Designs are Getting More Complex
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
0.4
2.7
6.1
11.1
0
2
4
6
8
10
12
2002 2007 2010 2012
N o n - F P
G A M e a n D e s i g n S
i z e G a t e s ( M )
Mean number of gates of logic and datapath, excluding memories trends
18
Page 19
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 19/76www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Designs are Getting More Complex
21% 22%
28%
6%
10% 13%
0%
10%
20%
30%
40%
50%
60%
NONE 1 2 3 4 5 or MORE
N o n - F P G A S t u d y P
a r t i c i p a n t s
Number of Embedded Processors for Non-FPGA Designs
2004
2007
2010
2012
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 201319
79% of designs contain one or more embedded processors
Page 20
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 20/76www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Designs are Getting More Complex
1.06
1.46
1.96
2.25
0
1
2
3
2004 2007 2010 2012
N o n - F P G A M e a n
n u m b e r
o f E m b e d d e d P r o
c e s s o r s
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 201320
Mean number of embedded processors continues to rise
Page 21
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 21/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
FPGAs are Getting Complex Too!
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
44%
38%
11%
1%4%
2%
21% 22%
28%
6%
10% 13%
0%
10%
20%
30%
40%
NONE 1 2 3 4 5 or MORE
S u r v e y P a r t i c i p
a n t s
Number of embedded processors
FPGA
Non-FPGA
56% of FPGAs contain one or more embedded processors
21
Page 22
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 22/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
How do we measure complexity?
HF, MemoCODE, 201322
Computational complexity theory used in computer science
There are no generally accepted metrics!
O(1)
O(logn)
O(n)
O(nlogn)
O(n^2)
O(2^n)
O(n!)
O p e r a t i o n s
Elements
Page 23
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 23/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Is bug density a good proxy?
Single, sequential data
streams — Floating point unit — Graphics shading unit — DSP convolution unit — MPEG decode
— . . .
Multiple, concurrent datastreams — Cross bar — Bus traffic controller — DMA controller — Standard I/F (e.g., PCIe)
— . . .
Channel
Compressed
Audio
Data Link LayerTX
RX
PHY
Sequential data streams
1x number of bugs
Concurrent data streams
5x number of bugs
Encoder Decoder
HF, MemoCODE, 201323
- Ted Scardamalia, internal IBM study
Page 24
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 24/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Packet-Based Design
Transaction
Layer Packet
Reformater
Data Link
Layer Packet
Reformater
Retry BufferArbiter
Tx
Rx
From
Fabric ToPHY
From Rx
Channel
Concurrency is Complicated to Verify
HF, MemoCODE, 201324
Page 25
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 25/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Maybe effort is a good proxy?
HF, MemoCODE, 201325
A f f o r d a
b l e
E x p e n s i v e
P r o h i b i t i v e
C o s t / E f
f o r t
Complexity
CriticalThreshold
Page 26
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 26/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Verification Consumes Majority of Project Time
0%
5%
10%
15%
20%
25%
1%-20% 21%-30% 31%-40% 41%-50% 51%-60% 61%-70% 71%-80% >80%
N o n - F P G A S t u
d y P a r t i c i p a n t s
Time (Percent)
Total Project Time Spent in Verification
2007
2010
2012
2007: Mean 49%2010: Mean 56%2012: Mean 56%
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 201326
Page 27
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 27/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
More and More Verification EngineersMean peak number of design vs. verification engineers
7.8 8.1 8.5
4.8
7.6 8.4
2007 2010 2012
~ 1-to-1 ratioof peak design
and verificationengineers
58% 11%
4% 5%
VerificationEngineers
DesignEngineers
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 201327
Page 28
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 28/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Where Verification Engineers Spend Their Time
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
16%
22%
23%
36%
4%Test Planning
Testbench Development
Creating and Running Test
Debug
Other
More time spent in debugthan any other task!
28
Page 29
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 29/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
54%
2007
46%
2007
47%
2012
53%
2012
Doing Design Doing Verification
Design Engineer Project Time2007 - 2012
HF, MemoCODE, 201329
Designers Doing More and More Verification
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
Page 30
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 30/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
0
20
40
60
80
100
2007 2012 2017 2022 2027 2032 2037
At this rate… In 25 years, ALL of a designer’s
time will bedevoted to
verification
Project Time 2007 - 2037
Design Verification
Time Design EngineersSpends Doing:
T i m e ( P e r c e n
t )
HF, MemoCODE, 201330
Time Designers Spends in Design vs. Verification
Page 31
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 31/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Design Reuse Trends
HF, MemoCODE, 201331
41%
33%
13% 13%
28%
35%
22%
15%
0%
10%
20%
30%
40%
NEW LOGIC REUSED LOGIC(Developed in-house)
PURCHASED IP ANALOG, RF AND/OR MIXED SIGNAL
N o n - F P G A S t u d y P a r t i c i p a n t s
Design Composition
2007
2012
Source: Wilson Research Group and Mentor Graphics.
Page 32
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 32/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Verification Reuse
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
50%
41%
8%
39%
44%
17%
0%
10%
20%
30%
40%
50%
NEW REUSED FROM OTHER DESIGNS
ACQUIRED EXTERNALLY
N o n - F P G A S t u d y P a r t i c i p a n t s
Testbench Composition
2007
2012
Mean testbench composition trends
32
Page 33
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 33/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
With All This Effort, How are We Doing?
HF, MemoCODE, 201333
Page 34
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 34/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Project’s Schedule Completion Trends
0%
5%
10%
15%
20%
25%
30%
35%
More than 10%EARLY
ON-SCHEDULE 20% 40% >50% BEHINDSCHEDULE
S t u d y P a r t i c i p a n t s
2007
2010
2012
2007: 67% behind schedule2010: 66% behind schedule2012: 67% behind schedule
Behind Schedule Ahead of schedule
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 201334
Page 35
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 35/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
FPGA vs. Non-FPGA Completion Trends
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
0%
5%
10%
15%
20%
25%
30%
35%
More than 10%EARLY
ON-SCHEDULE 20% 40% >50% BEHINDSCHEDULE
S t u d y P a r t i c i p a n t s
Non-FPGA vs. FPGA completion compared to project's original schedule
Non-FPGA
FPGA
Non-FPGA: 67% behind schedule FPGA: 67% behind schedule
35
Page 36
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 36/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Required Number of Spins
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
0%
10%
20%
30%
40%
50%
1FIRST
SILICONSUCCESS
2 3 4 5 6 7SPINS or
MORE
N o n - F P G A S t u d y P a r t i c i p a n t s
Number of Required Spins
2004
2007
2010
2012
36
Page 37
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 37/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Types of Flaws
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
0%
10%
20%
30%
40%
50%
60%
N o n - F P G A S t u d y P a r t i c i p a n t s
Trends in Types of Flaws Resulting in Respins
2004
2007
2010
2012
* Multiple answers possible
37
Page 38
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 38/76
Page 39
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 39/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential© 2010 Mentor Graphics Corp. Company Confidential
www.mentor.comHF, MemoCODE, 201339
Cost of Find Functional Flaws
Silicon Debug, Doug Josephson and Bob Gottlieb, (Paul Ryan)D. Gizopoulos (ed.), Advances in Electronic Testing: Challenges and Methodologies, Springer, 2006
Relative Cost Of Finding Bugs
$1
$10$100
$1,000
$10,000
$100,000
$1,000,000
$10,000,000
I n i t i a l D
e s i g
n
D e s i g
n R e v i e
w
L a y o u
t
T a p e
R e l e
a s e
E a r
l y S i l i c
o n
S a m
p l i n g
V o l u m
e P r o d u c t i o
n
Design Cycle
C
ostTo
Fix
Page 40
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 40/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
BEYOND STANDARDS
Beyond arguing over who won the standards war
HF, MemoCODE, 201340
Page 41
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 41/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Standardization of Languages
0%
20%
40%
60%
80%
VHDL Verilog Synopsys Vera System C SystemVerilog Specman e C/C++ OTHER Testbench
N o n - F P G A S t u d y P a r t i c i p a n t s
Languages Used for Verification (testbenches)
2007
2010
2012
SystemVerilog grew 8.3% between 2010 and 2012
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 201341
Page 42
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 42/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
SystemVerilog Adoption by Design Size
58.8%
71%
89%
0%
20%
40%
60%
80%
100%
< 5M 5 - 20M > 20M
N o n - F P G A S t u d
y P a r t i c i p a n t s
SystemVerilog Adoption by Design Size(Gate Count Excluding Memories)
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 201342
Page 43
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 43/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Standardization in Base Class Libraries
0%
10%
20%
30%
40%
AccelleraUVM
OVM Mentor AVM
Synopsys VMM
SynopsysRVM
CadenceeRM
CadenceURM
Other
N o n - F P G A S t u d
y P a r t i c i p a n t s
Testbench Methodologies and Base-Class Libraries
2010
2012
486% UVM growth between 2010 and 201246% UVM projected growth in the next twelve months
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 201343
Page 44
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 44/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Standardization of the SoC Verification Process
Ten years ago, IC/ASIC verification was partitioned intotwo main steps:
HF, MemoCODE, 201344
Integration Verification
Block-Level Verification
Block Full Chip
Page 45
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 45/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Standardization of the SoC Verification Process
Emerging from ad hoc to systematic processes
HF, MemoCODE, 201345
Block -Level Verification
Interconnect Verification
Integration Verification
Application/ SW
Verification
IP Subsystem SoC System
Page 46
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 46/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
BEYOND THE STATUS QUO
Beyond surviving by maintaining the status quo
HF, MemoCODE, 201346
Page 47
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 47/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
The Verification Paradox
A good verification process lets you get the most out ofbest-in-class verification tools
Start Tools Ad Hoc
Processes
6-9%
Cost Increase
Start Process Tools20-30%
Cost Savings
Source: Cisco Momentum Research Group
HF, MemoCODE, 201347
Page 48
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 48/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Standardization of the SoC Verification Process
HF, MemoCODE, 201348
Block -Level Verification
Interconnect Verification
Integration Verification
Application/ SW
Verification
Page 49
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 49/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Use of Advanced Verification Techniques
HF, MemoCODE, 201349
62%
71%
68%
70%
41%
40%
37%
48%
0% 20% 40% 60% 80%
Constrained-Random Simulation
Functional coverage
Assertions
Code coverage
Non-FPGA Study Participants
2007
2012
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
Page 50
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 50/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Directed vs Constrained-Random Simulation
HF, MemoCODE, 201350
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
44%51%
56%49%
0%
20%
40%
60%
80%
100%
2010 2012
N o n - F P G A S t u d y P
a r t i c i p a n t s
Mean Directed vs. Constrained-Random Simulation Trends
Directed
Constrined-Random
Page 51
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 51/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Larger Designs Use More Formal
20%
26%
41%
0%
10%
20%
30%
40%
< 5M 5 - 20M > 20M
N o n - F P G A S t u d y P a r t i c i p a n t s
Formal Property Checking Adoption by Design Size(Gate Count Excluding Memories )
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 201351
Page 52
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 52/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
The Evolution of Formal Technology
FormalPropertyChecking
Fully Automatic
Formal
Automated Applications
LowEffort
High
Effort
FormalExperts
Everyone
1990s Today2000s
HF, MemoCODE, 201352
Page 53
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 53/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Standardization of the SoC Verification Process
HF, MemoCODE, 201353
A57 A57 A57 A57
Block -Level Verification
Interconnect Verification
Integration Verification
Application/ SW
Verification
Page 54
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 54/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Standardization of the SoC Verification Process
HF, MemoCODE, 201354
Block -Level Verification
Interconnect Verification
Integration Verification
Application/ SW
Verification
— IP Blocks connectivity — Access all memories
— Access all registers, such as control — Configurations work — Functional scenarios and use-cases — Verify multiple clock domain crossings
Page 55
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 55/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Mean Number of Clock Domains by Design Size
5.21
9.36
11.48
0
2
4
6
8
10
12
14
< 5M 5 - 20M > 20M
M e a n N u m b e r
o f C l o c k D o m a i n s
Mean Number of Clock Domains by Design Size(Gates Excluding Memories)
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 201355
Page 56
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 56/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Standardization of the SoC Verification Process
HF, MemoCODE, 201356
Block -Level Verification
Interconnect Verification
Integration Verification
Application/ SW
Verification
— Boot OS — Load System Drivers
— Run Application SW
Page 57
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 57/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
SoC Design & Verification Involves Lots of SW
HF, MemoCODE, 201357
$0
$20
$40
$60
$80
$100
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
Total SW Engineering Costs + ESA Tool Costs
Total HW Engineering Costs + EDA Tool CostsSource: ITRS 2010, Impact of Design Technology on SoC Consumer Portable Implementation Cost
( $
M )
It’s the software, stupid! -Gary Smith
As Design Sizes Increase…Emulation Up,
Page 58
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 58/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
As Design Sizes Increase…Emulation Up,FPGA Prototyping Down in 2012
18%
32%
50%
45%
57%
28%
0%
10%
20%
30%
40%
50%
60%
HW Acceleration/Emulation FPGA Prototyping
N o n - F P G A S t u d y P a r t i c i p a n t s
Adoption by design size for those doingHW acceleration/emulation and FPGA Prototyping
< 5M
5 - 20M
> 20M
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study
HF, MemoCODE, 201358
Integrated Simulation/Emulation/Software
Page 59
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 59/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Integrated Simulation/Emulation/Software Verification Environments Emerge
HF, MemoCODE, 2013
Virtual Prototype
Processor Debug
JTAG
SW Debug
OVM/UVMSystemC/C++
Monitors
Testbench Acceleration
Assertions &Checkers
Physical Devices
Protocol Solutions
System Level
Virtual Devices &Transactors
Protocol Solutions
Design under Test
59
Page 60
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 60/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Coverage and Power
HF, MemoCODE, 201360
Block -Level Verification
Interconnect Verification
Integration Verification
Application/ SW
Verification
Block Subsystem SoC System
Across all aspects of verification
Page 61
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 61/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Signoff criteria trends
The Rising Importance of Coverage
HF, MemoCODE, 2013
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission * Multiple answers possible
0% 10% 20% 30% 40% 50%
OTHER
WHEN THE PROJECT PLAN SAYS SIGN-OFF,REGARDLESS OF STATUS
WHEN WE CAN NO-LONGER THINK OF ANY MORETESTS TO WRITE
WHEN THE RATE OF BUGS FOUND PER WEEK DROPSBELOW A SPECIFIED GOAL
WHEN THE EMULATED OR PROTOTYPED DESIGN ISWORKING IN-SITU
WHEN COVERAGE SAYS WE HAVE ACHIEVED ATARGET
WHEN THE PROJECT PLAN SAYS SIGN-OFF, ASSUMING VERIFICATION LOOKS OK
WHEN ALL TESTS DOCUMENTED IN THE VERIFICATION PLAN ARE COMPLETE AND PASS
Non-FPGA Study Participants
20072012
61
Page 62
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 62/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Unified Coverage Interoperability Standard
HF, MemoCODE, 201362
New Accellera UCIS Standard Announced at DAC 2012
UCIS API
CoverageDatabase
Simulation
Formal
Emulation
Analysis Testplan
Page 63
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 63/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Trends in Power Management Verification
0% 20% 40% 60% 80%
Power domain powerdown/power up
Power domain statereset/restoration
Transitions between systempower states
Hardware power controlsequence generation
Interactions between power domains
Operation in each system power state
Application-level power management
Hypervisor/OS control of power
management
Aspects of Non-FPGA Power Managed Design That Are Verified
67% of the industry actively
manages power
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
HF, MemoCODE, 201363
Page 64
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 64/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Power Trends
HF - January 2013 Master Set, WRG & MG Study Results
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
0%
5%
10%
15%
20%
25%
30%
35%
N o n - F P G A S t u d
y P a r t i c i p a n t s
Percentage of total simulations that were power-aware
About 10% of power managed designsperform no power-aware simulation!
Page 65
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 65/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Power Trends
HF - January 2013 Master Set, WRG & MG Study Results
Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
0%
5%
10%
15%
20%
25%
30%
35%
N o n - F P G A S t u d y P a r t i c i p a n t s
Percentate of Verification Resources Focused on PowerManagement
Median Verification Resources: 20% -29%
Page 66
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 66/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Power Trends
HF, MemoCODE, 201366
6% 6%
24%
15%
27%
22%
0%
5%
10%
15%
20%
25%
N o n - F P G A S t u d y P a
r t i c i p a n t s
* Multiple answers possible
Notation used to describe power intent
Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study, Used with permission
Page 67
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 67/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Beyond the Status Quo
HF, MemoCODE, 201367
Block -Level Verification
Interconnect Verification
Integration Verification
Application/ SW
Verification
IP Subsystem SoC System
Standardization of the SoC Verification Process
Page 68
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 68/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
BACK TO THE FUTURE
The Productivity Gap
HF, MemoCODE, 201368
Page 69
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 69/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Design Productivity Gap
SiliconDensity
DesignProductivity
C a p a c i t y
Time
Source: SEMATECH
HF, MemoCODE, 201369
Page 70
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 70/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
VerificationProductivity
Verification Productivity Gap
Source: SEMATECH
SiliconDensity
DesignProductivity
C a p
a c i t y
Time
HF, MemoCODE, 201370
Page 71
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 71/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Closing The Verification Gap
HF, MemoCODE, 20137171
Reuse Abstraction
Acceleration Methodology
?
Page 72
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 72/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential72
100s of lines
of TLM
Billions of Transistors
100s of Millions of Gates
Millions ofLines of RTL
#include "systemc.h"SC_MODULE(adder) // module
(class) declaration {sc_in<int> a, b; // portssc_out<int> sum; voiddo_add() // process { sum =a + b; } SC_CTOR(adder) //constructor {SC_METHOD(do_add); //register do_add to kernelsensitive << a << b; //sensitivity list of do_add } };
Managing Complexity
HF, MemoCODE, 201372
Page 73
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 73/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential73
RTL
clk
Cycle Accurateclk
Timed TLM
Untimed TLM
C/C++
Simulation
1x (7 days)
10,000x (1 min)
10x
100x
1,000x
Protocol
Protocol
Transaction
Functionarguments
Transaction
Productivity Gains Through Abstraction
HF, MemoCODE, 201373
Page 74
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 74/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
What’s the advantage of SystemC compared with RTL?
There are two different aspects – quality and time schedule.
Today, a full chip on SystemC will run around 10 MHz, and
you will never reach that speed using RTL or a lower-levelabstraction. It’s similar to a prototype speedup. Previously
with RTL designs, our bug rate was in the range of 10 to 50
bugs per square millimeter. Now we are at less than one bug
per millimeter squared. So we have both quality and speed
of development.
Source: EETimes, 2007, Laurent Ducousso, who manages intellectual-property (IP) verification forSTMicroelectronics’ Home Entertainment Division
Bug Prevention vs. Bug Hunting
HF, MemoCODE, 201374
Page 75
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 75/76
www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential
Summary
Beyond theory in terms of rising complexity Beyond arguing over who won the standards wars
Beyond surviving by maintaining the status quo
HF, MemoCODE, 201375
Page 76
7/21/2019 Keynote 3 Foster TrendsFunctionVerification
http://slidepdf.com/reader/full/keynote-3-foster-trendsfunctionverification 76/76