The Role of the FPGA in 400GbE Technology Development
Post on 26-Jan-2022
1 Views
Preview:
Transcript
© 2013 Ethernet Alliance 1
Gordon Brebner Distinguished Engineer Xilinx, Inc.
The Role of the FPGA in 400GbE Technology Development
2 © 2013 Ethernet Alliance
Disclaimer
The views we are expressing in this presentation are our own personal views and
should not be considered the views or positions of the
Ethernet Alliance.
3 © 2013 Ethernet Alliance
400GbE PCS/MAC
Expect first: 16 PCS lanes, each at 25.78125 Gb/s Glueless interface to optics Possible re-use of the 802.3ba PCS Other options possible for PCS, maybe native FEC
Later: 8 lanes, each at 51.56G Or 4 lanes with 2 bits/symbol at 56Gbaud (e.g. PAM4)
Packet size 64 bytes to 9600 bytes
Use 100GbE building blocks where possible
4 © 2013 Ethernet Alliance
Silicon technology
Technology nodes (silicon feature size) 130nm, 65nm, 40nm, 28/32nm, 20/22nm, 14/16nm
Application-Specific Integrated Circuit (ASIC) Fixed chip Increasingly expensive: need high volumes Best suited to post-standardization Ethernet
Field Programmable Gate Array (FPGA) Programmable logic chip Suitable for prototyping and medium volumes Best choice for pre-standardization Ethernet
5 © 2013 Ethernet Alliance
400GbE line/system bridge
500G
Interlaken
40 x 12.5G or
48 x 10G SERDES
Bridge logic
400GbE
PMA/PCS
CDFP or
4xCFP4
Optical
16 x 25G SERDES 400GbE
MAC
Wide parallel data path between blocks
ASIC or FPGA chip
Line side System side
6 © 2013 Ethernet Alliance
MAC rate = Width x Clock
400 Gb/s and 1 Tb/s Ethernet MAC options
MAC rate Silicon node Technology Data path width Clock frequency
100 Gb/s 45, 40nm ASIC 160 bits 644 MHz
100 Gb/s 45, 40nm FPGA 512 bits 195 MHz
400 Gb/s 28, 20nm ASIC 400 bits 1 GHz
400 Gb/s 28, 20nm FPGA 1024 bits 1536 bits
400 MHz 267 MHz
1 Tb/s 20, 14nm ASIC 1024 bits 1 GHz
1 Tb/s 20, 14nm FPGA 2048 bits 2560 bits
488 MHz 400 MHz
7 © 2013 Ethernet Alliance
Multiple Packets/Word
Up to 512-bit, only one packet completed Just need to deal with EOP then SOP in word
Beyond 512-bit, multiple packets completed Need to add parallel packet processing Must deal with varying EOP and SOP positions
Bus width Max packets Max EOPs
512 2 1
1024 3 2
1536 4 3
512 * n n+1 n
8 © 2013 Ethernet Alliance
400GbE CRC Example
All Ethernet packets carry Cyclic Redundancy Code (CRC) for error detection Computed using CRC-32 polynomial Critical function within Ethernet MAC
Requirements Computed at line rate Deal with multiple packets in wide data path Economical with silicon resources
9 © 2013 Ethernet Alliance
400GbE CRC Prototype
Xilinx Labs research project Modular: built out of 512-bit 100G units Computes multiple CRCs per data path word Targeting 28nm FPGA (Xilinx Virtex-7 series)
N-bit data path partitioned into 512-bit sections
512-bit unit CRC results combined to get final CRC results
10 © 2013 Ethernet Alliance
400GbE CRC Prototype
Results:
1024-bit width is feasible for 400GbE Other widths:
Less challenging clock frequencies Demonstrate scalability beyond 400GbE
Data bus word size 1024-bit 1536-bit 2048-bit
Max clock frequency (MHz) 400 381 326
Maximum line rate (Gb/s) 409 585 668
Latency (ns) 17.5 18.4 21.5
FPGA resources (slices) 2,888 4,410 5,719
11 © 2013 Ethernet Alliance
Pre-standard 400GbE
PCS/MAC layer implementation on FPGA: 64B to 9600B packets Priority pause handling, two queues in egress
direction with individual flow control Link fault handling Full statistics Optional 1588 timestamping 16 x CEI-28G-VSR interface
Unknown: actual final nature of PCS
Work under way - ready in 2014
12 © 2013 Ethernet Alliance
400GbE bridge on FPGA
Current (28nm) generation of FPGAs Two-FPGA implementation
Option (a) Directions separated Option (b) Functions partitioned
13 © 2013 Ethernet Alliance
400GbE bridge on FPGA
Future (20nm) generation of FPGAs – 2014 Single-FPGA implementation
Estimated resources: 644K logic cells (without FEC), 835K (with FEC)
14 © 2013 Ethernet Alliance
4x100GbE Aggregation
Xilinx research prototype Includes 32x8 128-bit crossbar switch and scheduler Aggregates 4x100G to 400G Targeting 28nm FPGA
Results:
BUNDLE 0
BUNDLE 2
BUNDLE 1
BUNDLE 3
BUNDLE 4 BUNDLE 5
BUNDLE 6 BUNDLE 7
BUNDLE 8 BUNDLE 9
BUNDLE 11
BUNDLE 13
BUNDLE 10
BUNDLE 12
BUNDLE 15
BUNDLE 17
BUNDLE 19
BUNDLE 21
BUNDLE 23
BUNDLE 14
BUNDLE 20
BUNDLE 18
BUNDLE 16
BUNDLE 22
BUNDLE 24
BUNDLE 26
BUNDLE 30
BUNDLE 28
BUNDLE 25
BUNDLE 27
BUNDLE 29
BUNDLE 31
BRAM
BRAM
BRAM
BRAM
BRAM
BRAM
BRAM
BRAM
BRAM
BRAM
BRAM
BRAM
BRAM
BRAM
BRAM
BRAM
CMAC
3
CMAC
2
CMAC
4
CMAC
1
Data bus word size 1024-bit
Max clock frequency (MHz) 450
Line rate (Gb/s) 460
Latency (# of clock cycles) 10
FPGA resources (logic cells) 20,851
15 © 2013 Ethernet Alliance
Conclusions
Can anticipate 400GbE PCS/MAC standard
Ever-increasing rates mean ever-wider internal data path width in electronics Leading to multiple packets per data word
Demonstrated modular Ethernet CRC block based on 100GbE units Silicon resource scales linearly with line rate
Possible to prototype pre-standard 400GbE PCS/MAC using today’s FPGA technology
top related