An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families Gabriel Loh August 24 th , 2021 IEEE SCV – Industry Spotlight
An Overview of Chiplet Technology
for the AMD EPYC™ and Ryzen™
Processor Families
Gabriel Loh
August 24th, 2021
IEEE SCV – Industry Spotlight
2 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Insatiable Performance Demands
1995 2000 2005 2010 2015 2020
1 exaFLOP
1 petaFLOP
0.1
1
10
100
1000
10000
Apr-18 Oct-18 May-19 Dec-19 Jun-20 Jan-21
Para
mte
rs (
Billions)
Number of ML Model Parameters
Switched
Transformers
GPT-3
T-NLG
T5Megatron
GPT-2
BERT-LargeGPT
[1]
[1] Based on published parameter counts of leading training models. [2] https://openai.com/blog/ai-and-compute
Required compute doubling every ~3.4 months [2]
3 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Technology Challenges
2004 2006 2008 2010 2012 2014 2016 2018 2020
65nm45nm
32nm22nm
14nm10/7nm
90nm
4 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Technology Challenges
Also…
◢Non-recurring costs per SoC
◢ Engineering design
◢ Verification
◢ Mask sets
Cost Per Yielded mm2 for a 250mm2 Die
-
1.0
2.0
3.0
4.0
5.0
6.0
45nm 32nm 28nm 20nm 14nm 10nm 7nm 5nm
Norm
aliz
ed
cost p
er
yie
lded
mm
2
5 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Monolithic Die Manufacturing
Wafer
6 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Monolithic Die Manufacturing
7 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Monolithic Die Manufacturing
Yielded Processors
8 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
High-level Chiplets Concept
Wafer
9 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
High-level Chiplets Concept
10 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
High-level Chiplets Concept
More Yielded Processors
11 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Chiplet Cost
Silicon cost is non-linear
with die area
12 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Chiplet Overheads
◢ Inter-chiplet communication interfaces
◢Per-die functionality
13 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Chiplet Overheads
◢Architectural design effort, partitioning
> X
◢ Inter-chiplet communication interfaces
◢Per-die functionality
14 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Case Study:
AMD EPYC™ Processors
15 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
1st-gen AMD EPYC™ Product Targets
◢Up to 32 “Zen” CPU cores
◢Eight DDR4 memory channels
◢128 lanes PCIe® gen4 I/O
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
DD
RD
DR
DD
RD
DR
DD
RD
DR
DD
RD
DR
I/O I/O I/O I/O
I/O I/O I/O I/O
Monolithic 32-core Chip
777mm2 total area in 14nm
16 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
1st-gen AMD EPYC™ MCM Organization
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
DDRDDR
DDR
DDRDDR
DDRDDRDDR
I/O
I/O
I/O
I/O
I/O
I/O
I/O
I/O
◢213mm2 per chiplet (14nm)
◢4x → 852mm2 total
+10% silicon area
0.59x cost
17 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
1st-gen 14nm chiplet
Moving to 7nm Technology
2x device density
+25% performance
or
50% power at 1x perf.
64-core possibility!
18 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
1st-gen 14nm chiplet
Moving to 7nm Technology
2x device density
Co
res, C
ach
eI/O
, D
DR
19 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Moving to 7nm Technology
2x device density
Co
res, C
ach
eI/O
, D
DR
Cost Per Yielded mm2 for a 250mm2 Die
-
1.0
2.0
3.0
4.0
5.0
6.0
45nm 32nm 28nm 20nm 14nm 10nm 7nm 5nm
Norm
aliz
ed
cost p
er
yie
lded
mm
2
20 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Technology-optimized Chiplet Organization
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
Four “Zen”
Cores+L3
DDRDDR
DDR
DDRDDR
DDRDDRDDR
I/O
I/O
I/O
I/O
I/O
I/O
I/O
I/O
21 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Technology-optimized Chiplet Organization
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
DD
RD
DR
DD
RD
DR
DD
RD
DR
DD
RD
DR
I/O
I/O
I/O
I/O
I/O
I/O
I/O
I/O Move cores and
cache to 7nm
Ke
ep
in
ma
ture
12
nm
pro
ce
ss
86% of die area benefits
22 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Technology-optimized Chiplet Organization
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
DD
RD
DR
DD
RD
DR
DD
RD
DR
DD
RD
DR
I/O
I/O
I/O
I/O
I/O
I/O
I/O
I/O
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
Four “Zen 2”
Cores+L3
23 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Packaging Challenges
◢Chiplets can help address silicon challenges,
but can also introduce other new challenges
to overcome
◢Example: package routing already fully
utilized
◢ Package layers in 1st-gen processor used for
power delivery to CCD
CC
DC
CD
CC
DC
CD
CC
DC
CD
CC
DC
CD
IOD
24 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Solution: Silicon-Package Co-design
◢Metal resources below
chiplet previously
consumed by power
distribution
◢Coordinated re-design of
both L3 VDDM LDOs and
package metal/routing
resources enabled solution
◢Solution achieved VDDM
IR drop within 10mV
◢Take-away: advanced
chiplet design requires an
increasingly vertical and
multi-domain co-design
approach
Packag
eR
DL
25 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
2nd-gen AMD EPYC™ Benefits
0.0
0.5
1.0
1.5
2.0
64 Cores 48 Cores 32 Cores 24 Cores 16 Cores
Norm
aliz
ed D
ie C
ost
Great cost benefit vs. monolithic
Makes 64 cores possible
Two tape-outs for full stack
Linear cost with core count
Full memory and IO
7nm CCD + 12nm IOD Hypothetical Monolithic 7nm
26 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Additional Advantages of Technology-optimized Chiplet Organization
Rem
ote
Requ
est
Local Request
Remote Request
90ns memory latency
141ns memory latency
Average latency 128ns
(25% local, 75% remote)
27 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Additional Advantages of Technology-optimized Chiplet Organization
I/O PHY I/O PHY
DD
R
DD
R
DD
R
DD
R
DD
R
DD
R
DD
R
DD
R
I/O PHY I/O PHY
IS
PCIe
PCIe
IS
PCIe
PCIe
I/O PHY I/O PHY
I/O PHY I/O PHY
MC
MC
MC
MC
MC
MC
MC
MC
Data Fabric switch: 2 FCLK
(best-case, low load)
Repeater: 1 FCLK
1
2
3
4
1 ~94ns
2 ~97ns
3 ~104ns
4 ~114ns
@ FCLK = 1.46 GHz
Improvement in Worst-Best case latency
from 51ns to 20ns
Despite mandatory hop, local
latency only increases 4ns (90ns→94ns)
28 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Case Study:
AMD Ryzen™ Processors
29 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight29 An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Leveraging Technology Across Markets
CC
DC
CD
CC
DC
CD
CC
DC
CD
CC
DC
CD
DD
RD
DR
DD
RD
DR I/O I/O
I/O I/O
DD
RD
DR
DD
RD
DR
I/O I/OI/O I/O
IOD
I/O I/O
DD
RD
DR
CC
D
CC
D
CCD Reuse
Direct IOD IP leverage
Up to 16-core desktop
30 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight30 An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Chiplet Benefits for AMD Ryzen™ Processors
I/O I/O
DD
RD
DR
CC
D
CC
D
0.0
0.5
1.0
1.5
2.0
2.5
16 Cores 8 Cores
No
rma
lized
Die
Co
st
Chiplet (CCD+cIOD) Hypothetical Monolithic 7nm
31 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight31 An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Further Silicon Leverage
CC
D
CC
D
DD
RD
DR
I/O I/O
PCIe
SATA
USB
I/O I/O
DD
RD
DR
Silicon Reuse
Address Platform Need
Harvest Silicon
32 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Silicon Reuse + Package Customization
DDRDDR
I/O
I/O
DDRDDR
I/O
I/O
DDRDDR
I/O
I/O
DDRDDR
I/O
I/O
I/OI/O
I/OI/O
DDR DDR
DDRDDR
I/O
I/O
I/O
I/O
DDR
DDR
33 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Summary
Server
IOD cIOD
CCD
CCD
CCD
CCD
See our ISCA 2021 paper for
additional details and insights
34 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Thanks to the multiple AMD teams
across the globe
Their dedication and hard work are
what truly breathes life into all of
our products
35 | An Overview of Chiplet Technology for the AMD EPYC™ and Ryzen™ Processor Families | IEEE Industry Spotlight | August 24th, 2021 | IEEE SVC Industry Spotlight
Disclaimer
The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions, and
typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but
not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product
differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. Any computer system has
risks of security vulnerabilities that cannot be completely prevented or mitigated. AMD assumes no obligation to update or otherwise correct
or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content
hereof without obligation of AMD to notify any person of such revisions or changes.
THIS INFORMATION IS PROVIDED “AS IS.” AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE
CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS, OR OMISSIONS THAT MAY
APPEAR IN THIS INFORMATION. AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY, OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR
ANY RELIANCE, DIRECT, INDIRECT, SPECIAL, OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY
INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
© 2021 Advanced Micro Devices, Inc. All rights reserved.
AMD, the AMD Arrow logo, EPYC, Ryzen, Threadripper, Infinity Fabric, and combinations thereof are trademarks of Advanced Micro
Devices, Inc. Other product names used in this publication are for identification purposes only and may be trademarks of their respective
companies.