Consume. Collaborate. Contribute. Open. Together.
Consume. Collaborate. Contribute.
Open. Together.
Consume. Collaborate. Contribute.Consume. Collaborate. Contribute.
O p e n D o m a i n S p e c i f i c A r c h i te c t u r e :T h e I m p a c t o f O D S A
By:Bapi Vinnakota – OCP ODSA Sub-Project LeadKevin Drucker – Facebook ODSA Lead
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)Agenda
• Background on the Open Compute Project
• Open Domain Specific Architecture (ODSA) Sub-Project
• Drivers for Domain Specific Architectures
• Evolution of the industry
• Chiplet and SIP use cases
• Market Opportunity
• ODSA Collaboration
Consume. Collaborate. Contribute.
ODSA: Open Domain-Specific Architecture
• Domain-specific architecture: Programmable devices optimized for specific applications or class of applications. Meet the demands of high-intensity workloads in the data center and at the edge – e.g. machine learning, video processing
• Chiplets: Implement an integrated product as a collection of die in a single package, instead of a single die. Each die is a chiplet. Can reduce development and manufacturing costs.
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)History of Open Compute Project (OCP)
• In 2009 Facebook, confronting exponential growth, was forced to rethink infrastructure architecture
• Small team of engineers spent next two years re-envisioning and designing software, servers, racks, power supplies, cooling and data center design
• In 2011 Facebook shared it’s designs and released them to the public
• Along with Intel, Rackspace, Goldman Sachs and Andy Becholtsheim, the Open Compute Project (OCP) was launched and the Open Compute Project Foundation created
• Today, OCP has grown into a massive global community with over 150 member companies
Over 150 Companies !!!
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)ODSA within OCP: A new Server Sub-Project)
• ODSA organized within the Open Compute Project (OCP):
⎻ ODSA aligned as OCP Server sub-project
⎻ Initial concept brought to OCP by Netronome
⎻ First silicon focused effort within OCP
• Extending Moore’s Law:
⎻ Domain-Specific Architectures: Typically programmable silicon products that accelerate high-intensity workloads (e.g. Tensorflow, Network Processor, Antminer…)
⎻ Chiplets / SIPs go beyond MCM → Build complex products from multiple die, instead of monolithic devices, to reduce TTM, R&D expense and manufacturing costs.
• Open Domain-Specific Architecture: An architecture for a domain-specific products
⎻ Today: All multi-chiplet products are based on proprietary interfaces
⎻ Tomorrow: Select best-of-breed chiplets from multiple vendors
⎻ Incubating a new group, to define a new open interface and standards to build a PoCSpecifications
SERVER
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)Cambrian Explosion of Workloads
Early Compute Era
Standard Computation HPC Computation
Standard Computation
Standard Computation
Graphics Computation
Modern Computation
Bloc
k Cha
in
Cognitiv
e Se
curit
y
NLPRobo
tic P
As
Auton
omou
s Veh
icles
Mac
hine
Lea
rning
Neura
l Net
works
Deep Le
arning
Patte
rn R
ecog
nitio
n
RT T
rans
lation
AI and Machine-learning and data-heavy workloads have
exploded in past 5 years and will diversify as new applications
are discovered constantly…
Spon
ges
Mollusk
s
Prok
aryo
tes
Euka
ryot
es
Hemich
orda
ta
Echino
derm
s
Comb Je
llies
Early Precambrian Era
Late Precambrian
Prokaryotes
Eukaryotes
Early Cambrian
Plac
ozoa
Arthro
pods
Wor
ms
Chord
ata
Bio-Diversity Exploded from single cells into multi-cell
organisms during the Cambrian explosion; all major
phyllawere established in this transition
Metazoans
All images from Creative Commons
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)Chiplet and SIP Technology Is Happening Today!
PROVEN IN EXISTING BUSINESS MODELS
[L. Su, IEDM’17]
CHIPLET VS MONOLITHICS
DIE COST 1X DIE COST 0.59X
I/O I/O
I/O I/O
L1 L1
L1 L1
CPU
CPU
CPU CPU
CPU
CPU
CPU CPU
I/O L1
CPUCPU
I/O L1
CPUCPU
I/O L1
CPUCPU
I/O L1
CPUCPU
• Many major semiconductor companies have brought to market chiplet / SIP based products
• Developed using hybrid flows of internally developed tools and EDA industry available point solution
• ODSA seeks to enable internally developed and 3rd party chiplets to be readily integrated
• ODSA seeks to democratize this evolution of chiplet and SIP technology for the larger mass market through an open eco-system marketplace
• Focus on three (3) use cases:
⎻ IO Disaggregation, Core Disaggregation, System Integration
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)Major Chiplet / System in a Package (SIP) use cases: IO Disaggregation
• Use case: IO Disaggregation
• What:
⎻ Separating core and un-core
⎻ High speed IO Interfaces moved to chiplets
⎻ Ex: PCIe I/F, Memory I/F, Network I/F
• Why:
⎻ Focus resources on value added core functions
⎻ Reduce development and qual intervals → shorter TTM
⎻ PPA: Target IP / Core to process nodes that yield highest Performance, Power, Area for a given function
I/O I/O
I/O I/O
L1 L1
L1 L1
CPU
CPU
CPU CPU
CPU
CPU
CPU CPU
L1
L1
CPU
CPU
CPU CPU
CPU
CPU
CPU CPU
I/O
L1
L1
I/O
I/O
I/O
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)Major Chiplet / System in a Package (SIP) use cases: Logic Disaggregation
• Use case: Logic Disaggregation
• What:
⎻ Sub-divide large area of monolithic low yield device into multiple lower area / higher yielding devices
• Why:
⎻ Shift final product from costly, non-linear portion of semiconductor yield curve to lower-cost, higher yielding linear portion
I/O I/O
I/O I/O
L1 L1
L1 L1
CPU
CPU
CPU CPU
CPU
CPU
CPU CPU
Illustration Of
Semiconductor
Yield Curve
Die Area
Yie
ld L
oss
I/O L1
CPUCPU
I/O L1
CPUCPU
I/O L1
CPUCPU
I/O L1
CPUCPU
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)Major Chiplet / System in a Package (SIP) use cases: Combined
I/O I/O
I/O I/O
L1 L1
L1 L1
CPU
CPU
CPU CPU
CPU
CPU
CPU CPU
L1 CPUCPU
Now how about we add KGD memory…
I/O
I/O I/O
I/O
Combined IO and Logic Disaggregation
L1 CPUCPU
L1 CPUCPU L1 CPUCPU
• Optimize for process node vs. PPA→Select the best node for the given function
• Maximize yield curve for linear intercepts
• Amortize cost of development and qualification of the chiplets and re-use them for subsequent products
• Combine with known good die chiplets (KGD) from 3rd parties
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)Major Chiplet / System in a Package (SIP) use cases: System Integration
• Use case: System Integration
• What:
⎻ Integrate multiple SoCs and components into a single SIP package
⎻ Obtain KGD from multiple 3rd party suppliers
⎻ Ex: co-packaged optics and switch
• Why:
⎻ Significant system footprint reduction
⎻ IO power reduction: driving inches of PCB vs. mm of substrate ➔ mJ→ pJ
⎻ Huge win @ scale: even 5W ~ 10W savings power system adds up to MW savings
I/OLASER I/O
I/O I/O
I/OSWITCH SWITCH
SWITCH SWITCHI/OLASER
I/O LASER
I/O LASER
I/OLASER SWITCH SWITCH
SWITCH SWITCHI/OLASER
I/O LASER
I/O LASER
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)Transformative Market Opportunity: The Tip of the Spear
• Independent research from IHS Markit
• Four (4) segments for chiplets analyzed (SoCs, MPU, GPU, PLD (FPGA/CPLDs)
• Six Verticals (wireless, wireline, consumer, computing, industrial, automotive)
Subset of IHS Markit Data: •System on a Chip (SoC)
segment•Conclusion: Immediate
opportunity for chipletsand an open interface
Tip of the spear !!!0
100
200
300
400
500
600
2018 2019 2020 2021 2022 2023 2024
Wireless Communications Consumer Electronics Wired Communications
Computing & Data Storage Industrial Electronics Automotive Electronics
SoC SAM by Market Segment
© 2019 IHS Markit
SoC
s built
wit
h C
hip
lets
$(M
)
Excludes embedded x86 MPUs marketed as SoCs. Includes Configurable SoCs from PLD Vendors.
Source: IHS Markit
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)Transformative Market Opportunity: The End Game
• Initially dominated by compute uses case, other market segments grow to dominate• Takeaway: an open chiplet eco-system with open interface standards is a huge opportunity
0
500
1,000
1,500
2,000
2,500
3,000
3,500
2018 2019 2020 2021 2022 2023 2024
Computing & Data Storage Wired Communications
Wireless Communications Consumer Electronics
Industrial Electronics
Overall Chiplet SAM by Market Segment
Aggre
gat
e P
roce
ssor
Mar
ket
built
wit
h C
hip
lets
$(M
)
0
10,000
20,000
30,000
40,000
50,000
60,000
2020 2025 2030 2035
Computing & Data Storage Wired Communications
Wireless Communications Consumer Electronics
Industrial Electronics Automotive Electronics
Extended Chiplet SAM by market segment
Aggre
gat
e P
roce
ssor
Mar
ket
built
wit
h C
hip
lets
$(M
)
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)Growth of ODSA since Inception
7 10
35
52
65
85
0
10
20
30
40
50
60
70
80
90
OCT DEC JAN MAR JUN SEPT
Cumulative Company Count
0
20
40
60
80
100
120
140
160
180
Jan March June Sept
Attendance in ODSA Events
Source: OCP ODSA Survey, Workshop Registration Data
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)ODSA Vision of Chiplet Eco-System
PCIeSerDes
XSR, Kandou,..BoW
Parallel
AIB, HBM,…
PIPE Interface Adapter Common Abstraction
PCIe Link Layer
SA
TAC
CIX
Inter-Chiplet
Link Layer
Intra-Chiplet
Link Layer
Routing
Cache
Coherent
Non-
CoherentISF
Pac
kage
agg
rega
tion S
tack
Die
dis
aggr
ega
tion S
tack
Inter-chiplet PHY and packaging options
CX
L
Tile
Lin
k
Organic substrate
Interposers/bridges
HOST
STORAGE
NET
WO
RK
X8 PCIe G3 (64Gbps)40G Ethernet optical
40G Ethernet copper
NFP
FPGA
CPUF
F
QSFP 40G
QSFP 40G
MTP
MTP
F
F
QSFP 40G
QSFP 40G
MTP
MTPDRAM
DRAM
F
F
F
F
PCIe x4 Gen3
Chiplet
Marketplace
OCP Form Factors drive Power,I/O
Footprint, Performance
Multi-technology
ODSA stack
Reference
architectures for:
Networking, Storage,
Inferencing, Training,
Video and Image
processingBusiness, Tools
Workflow to assemble
Product
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)ODSA Vision of Chiplet Eco-System
KEY INDUSTRY ASKS
• Readily available, integrated End-to-End SIP level Modelling
and Development Flows and Tools:
• Functional, Electrical, Mechanical, Thermal, Testability,
Manufacturing
• Common chiplet data exchange formats to share chiplet
information between 3rd parties
• Coalesce around common chiplet model formats
• Shared standards for integration of Known Good Die (KGD)
Consume. Collaborate. Contribute.
!! Coming Along Nicely !!
On-die custom protocol (could be open e.g. AXI)
Off-package protocol (e.g. PCIe)
DMACache
Coherent
PCIeSerDes
XSR, USR,..BoW
Parallel
AIB, HBM,…
PIPE Interface Adapter PHY-type specific Abstraction
PCIe Link Layer
SA
TAC
CIX
Inter-Chiplet Link Layer Intra-Chiplet
Link Layer
Routing
Cache
Coherent
Non-
CoherentISF
Pa
ck
age
aggre
ga
tio
n S
tack
Die
dis
aggre
ga
tio
n S
tack
Inter-chiplet PHY and
packaging options
CX
L
Organic substrate
Interposers/bridges
CH
I B
us
Tile
Lin
k
NO
C
PC
Ie
Off-package bus adapter New chiplet-aware stackOn-die bus adapter
Package aggregation Die disaggregation
Interface(CXS/CXL/OPENCAPI)
On-die Bus
AC
E L
ite
Bu
s
Chiplet agnostic logical design Chiplet-aware design
The Impact of Open Domain Specific Architecture (ODSA)ODSA Stack: Current View
Consume. Collaborate. Contribute.
PCIeSerDes
XSR, Kandou,..BoW
Parallel
AIB, HBM,…
PIPE Interface Adapter Common Abstraction
PCIe Link Layer
SA
TA
CC
IX
Inter-Chiplet Link
Layer
Intra-Chiplet
Link Layer
Routing
Cache
Coherent
Non-
CoherentISF
Pa
ck
age
aggre
ga
tio
n S
tack
Die
dis
aggre
ga
tio
n S
tack
Inter-chiplet PHY and packaging options
CX
L
Tile
Lin
k
Organic substrate
Interposers/bridges
Please Help! Join a Workstream!
Join Interface/Standards:(Mark Kuemerle/Ramin Farjad/
Robert Wang/David Kehlet)
DevelopPackaging + Socket, Dev Board
Provide FPGA IP
Provide ODSA chiplets
Provide PHY Technology
Join the PoC, Build fast:(Quinn Jacobson/Jawad Nasrullah/
Jayaprakash Balachandran)
Join Business, IP and workflow:(Sam Fuller/Dharmesh Jani)
Develop software Define test andassembly workflow
Provide Chiplet IP
Workstream contact information at the ODSA wiki
Define ArchitecturalInterface
The Impact of Open Domain Specific Architecture (ODSA)ODSA Workstreams
Consume. Collaborate. Contribute.
Project Objective Organizations Participating Recent Results Upcoming Milestones Needs
PHY Analysis PHY requirementsPHY analysisCross-PHY abstraction (PIPE)
Alphawave, AnalogX, Aquantia, Avera Semi, Facebook, Intel, Kandou, Netronome, zGlue,
PHY Analysis paper(published at Hot Interconnect)
PIPE abstraction
BoW Interface No technology license fee, easy to port inter-chiplet interface spec
Aquantia, Avera Semi, Netronome
BoW Interface proposal(published at Hot Interconnect)
BoW specification 0.7End September, 2019
Test chips, Chiplet library supporting interface
Prototype product that integrates existing die from multiple companies into one package
Achronix, Cisco, Netronome, NXP, Samtec, Sarcina, zGlue,Macom, Facebook
Decomposable design flow. Committed schedule End user End user participation~30% funding is open
Chiplet design exchange
Open chiplet physical description format. Ayar, Cadence, NXP, zGlue, Draft spec ZEF Exchange format draft specification
Link and Network Layer
Interface and implementations –requirements and proposals
Achronix, Avera Semi, Intel, Netronome, NXP, Xilinx
Multi-chiplet test Test requirements for an open-chipletinterface
Chiplet monitoring Monitoring infrastructure for chipletoperation
Business workflow Formalize learnings from prototype effort
Wiki: https://www.opencompute.org/wiki/Server/ODSA, meet Fridays at 8 AM Pacific Time. Please join us.
Engineers from:
Achronix, AnalogX, ASE, Avera Semi, Ayar, Cadence, Cisco, Facebook,
Ferric, Intel, Kandou, Macom, Marvel, Netronome, NXP, On Semi,
Samtec, Sarcina, Synopsys, Xilinx, zGlue
The Impact of Open Domain Specific Architecture (ODSA)Active Projects
Consume. Collaborate. Contribute.Open. Together.
The Impact of Open Domain Specific Architecture (ODSA)ODSA Collaboration Results
• Physical Interface Definition:
⎻ PHY: Multi-company (zGlue, Netronome, Alphawave, Aquantia) analysis. First cross-PHY analysis published at Hot Interconnect
⎻ Bunch of Wires (BoW) PHY – new low-cost inter-chiplet PHY. Developed cooperatively by Aquantia, Avera Semi and Netronome with input from Xilinx, Global Foundries. Published at Hot Interconnect, IEEE Micro (Jan’20).
⎻ PIPE Abstraction – a new layer to “abstract” the interface PHY. Supported by Intel in their most recent PHY specification.
• SIP Proof-of-concept / Prototype:
⎻ Software development vehicle: Designed collaboratively by Cisco, Facebook, Macom, zGlue. Components from Achronix, NXP, Netronome. Working toward integrated into SIP. Design is highly scalable → able to add new participants easily.
• Chiplet Design Exchange (CDX) Project:
⎻ Project to facilitate the design of physical chiplet information for CAD tool integration (zGlue, ASE, Cadence, Macom)
• Independent Market survey conducted by IHS Markit
• Everything is on our wiki - https://www.opencompute.org/wiki/Server/ODSA
Consume. Collaborate. Contribute.
Time to begin planning Implementation• PHY Layer:
• Open or nearly-open PHY layer
• Association to share development costs for proprietary PHY layer
• Combination of the two
• Adapter Logic
• PIPE adapter logic
• IO Coherent link layer protocol –an ODSA member company may offer
• Coherent link layer protocol – open
• Bus adapters
• Easy to use no-friction interface will be widely used
• Open source implementation is a good start
The Impact of Open Domain Specific Architecture (ODSA)Looking Ahead
Consume. Collaborate. Contribute.Open. Together.
Next ODSA Workshop: 2019-Dec-18 @ Facebook, Menlo Park Ca
IF YOU WANT TO GO FAST, GO ALONEIF YOU WANT TO GO FURTHER, GO TOGETHER
AFRICAN PROVERB
Consume. Collaborate. Contribute.
Open. Together.