Energy Optimization of Interconnection Networks Using ...§Improving energy efficiency while increasing data rate is difficult with traditional means §Silicon photonics based links

Energy Optimization of Interconnection NetworksUsing Silicon Photonics

Romesh Nandwana

Cisco Systems, Allentown, PA

© 2018 Cisco and/or its affiliates. All rights reserved. 2

§ Market Trends

§ Design challenges for next gen Data-Center Interconnects

§ Silicon Photonics based transceivers

§ Conclusion

Outline

© 2018 Cisco and/or its affiliates. All rights reserved. 3

By 2021, Global Internet Growth and Trends

Source: Cisco VNI Global IP Traffic Forecast, 2016–2021

Key Digital Transformers

2016

2021

3.3 Billion

4.6 Billion

17.1 Billion

27.1 Billion

27.5 Mbps

53.0 Mbps

73% of Traffic

82% of Traffic

More Devices & Connections

Faster Broadband Speeds

More Video Viewing

More Internet Users

© 2018 Cisco and/or its affiliates. All rights reserved. 4

Global Mobile Data Traffic Growth / Top-LineAverage Traffic per User

Average Traffic per Connection2016

902 MB per month

2021

4,226 MB per month

2016 2021

1,456 MB per month 8,423 MB per month

Source: Cisco VNI Global Mobile Data Traffic Forecast, 2016–2021

0

10

20

30

40

50

60

2016 2017 2018 2019 2020 2021

35

7

Exab

ytes

per M

onth

24

1711

49

47% CAGR 2016–2021

7x increase from 2016 to 2021

Global Mobile Data Traffic Growth

© 2018 Cisco and/or its affiliates. All rights reserved. 5

§ Cisco Global Cloud Index àProjecting global IP traffic to increase 3X à2016 to 2021

§ $ per Gb/s in Dell O’ro chart is the aggregated cost of port bandwidth in the data switch market

Traffic Growth & Cost Trends

0

5

10

15

20

25

2016 2017 2018 2019 2020 2021

6 .8

17 .1

Zetta

byte

s pe

r Yea

r

14.111.6

9.1

20.625% CAGR 2016–2021

Global Data Center IP-Traffic Growth

0

4

8

12

16

20

0

20

40

60

80

100

2016 2017 2018 2019 2020 2021 2022

$pe

rGb/s

Ports

(mill

ions

)

400GE 100GE50GE 40GE25GE 10GE$perGb/s

DellO'ro EthernetSwitchDataCenterForecastJan'18

Data Center Ethernet Switch Market

© 2018 Cisco and/or its affiliates. All rights reserved. 6

Global Data Center Traffic by Destination, 2021A

B

C

Data Center to User14.9% Within Data Center

71.5%Data Center to

Data Center13.6%

Storage, production and development data, authentication

Within Data Center (71.5%)

Data Center to Data Center (13.6%)

Data Center to User (14.9%)

Replication, CDN, intercloud links

Web, email, internal VoD, WebEx…Source: Cisco VNI Global Cloud Index, 2016–2021

Most Data Center Events/Content Stays Within the Data Center

© 2018 Cisco and/or its affiliates. All rights reserved. 7

Current Trends in Wireline Link Design

Increase in data-rate at the expense of power-efficiency degradation

Daly, IEEE Solid-State Circuits Magazine 10.1 (2018): 30-46.

© 2018 Cisco and/or its affiliates. All rights reserved. 8

Backplane Channels in Data-Center EnvironmentInsertion Loss

Crosstalk

~60dB Loss @ 28GHz

NRZ

© 2018 Cisco and/or its affiliates. All rights reserved. 9

What’s Next ?

Increase data-rates à complex modulation schemes are needed

© 2018 Cisco and/or its affiliates. All rights reserved. 10

§ PAM-4 bandwidth ~ NRZ bandwidth/2 à 2X timing margin

§ SNR loss = 20 log (1/3) ~ 9.5 dB§ Forward Error Correction (FEC) to compensate for additional loss

§ Transimpedance Amplifier (TIA) has to be linear

Backplane Channels in Data-Center EnvironmentInsertion Loss

Crosstalk

~31dB Loss @ 14GHz

~60dB Loss @ 28GHz

NRZ

PAM-4

© 2018 Cisco and/or its affiliates. All rights reserved. 11

§ Accelerating traffic growth demands higher port speed + high switch & router BW

§ Higher SERDES IO & port speeds are being adopted at faster rates to enable efficiencies

§ Pluggable optics transition to higher speeds & higher densities

Market Transitions Increasing PaceEvolution of Module Faceplate BW Density

© 2018 Cisco and/or its affiliates. All rights reserved. 12

§ IP traffic growth → Driving need for more & higher capacity data centers

§ IP traffic growth → Driving Optics transitions to higher & higher speeds§ 10G → 40G → 100G → 400G → 1Tb

§ To support the tsunami in traffic growth, highest volume optics deployment will be in the Data Center → Also the most cost sensitive !

§ Power management in Data Center is critical → Silicon Photonics

§ Single Mode fiber deployment in Data Centers is the defacto standard → Silicon Photonics

§ Higher speed ports are driving lower bandwidth costs per Gb/s

§ Volume of ports will drive scale & cost

Data Center Trends

© 2018 Cisco and/or its affiliates. All rights reserved. 13

L Lower level of integration

L Poor signal integrity

L Low yield

L Small wafer/high cost

L Thermal sensitivity

L Hermetic sealing

III-V Vs Silicon Photonics LinksRX

CDR

TXCDR

ROSA

TOSA

Pwr

μCtrl

ElectricalICSi-PhotonicIC

Pwr

μCtrl

J High density integration

J Excellent signal integrity

J CMOS yield/ wafer level testing

J Large wafer/low cost

J Wide thermal operating range

J Non-hermetic package

III-V Module Silicon-Photonics Module

© 2018 Cisco and/or its affiliates. All rights reserved. 14

4x25 Gbps NRZ

HOST

HOST SERDES

SerializerMZI Driver

PAM Decoder

ADC-CDR

53 GBd Linear TIA

CMOS Electrical ICOptical IC

53 GBd

Mach Zehnder

Modulator53 GBd

LINE

4x25 Gbps NRZ

Photodiode

PAM Encoder

§ Electrical IC flip-chipped on Optical IC

§ Segmented MZI driver for multilevel data transmission

§ Linear TIA integrated with transceiver SoC

Electro-Optical System in Package

© 2018 Cisco and/or its affiliates. All rights reserved. 15

§ Efficient: Charge at peak of optical intensity

§ VπLπ ~ 2 V.mm

§ Low voltage operation 1.0 V (CMOS)

§ Short MZI modulators ~ 400µm

§ Lumped element operation

The SISCAP ModulatorSISCAP – Silicon Insulator Silicon CAPacitor

Optical Intensity

+ + + + + + + + - - - - - - - - - -

P+

N+

Poly-Si

SOIGate Oxide

© 2018 Cisco and/or its affiliates. All rights reserved. 16

§ Temperature performance of modulator important for silicon photonic applications

§ Typical results shown for 1550nm SISCAP ( from 10 sites across wafer)

SISCAP Temperature Behavior

Measured phase behavior Measured attenuation behavior

o T=30C+ T=90C

o T=30C+ T=90C

Optical Intensity

+ + + + + + + + - - - - - - - - - -

P+

N+

Poly-Si

SOIGate Oxide

© 2018 Cisco and/or its affiliates. All rights reserved. 17

§ CMOS (limiting driver) directly drives MZI segments

§ Eliminates the need for an Electrical DAC + Linear Driver

§ Good Potential for PAM and QAM applications

SISCAP Segmented MZI ModulatorLSB MSB

B0 B1

Digital CMOS Drivers Optical Phase

Optical Power bias point

Phase Modulation by segments

MZI Response

© 2018 Cisco and/or its affiliates. All rights reserved. 18

§ Performance to 28 GBd/s and beyond

§ Operates over 1310nm and 1550nm wavelengths

Silicon Photonics Transmitter Performance

*M.Mazzinietal,CiscosystemsInc,OFC2015

**B.Milivojevicetal,CiscoSystemsInc,OFC2016

Direct-Detection Coherent

© 2018 Cisco and/or its affiliates. All rights reserved. 19

Receiver Test Fixtures

Electrical Optical

PCB

Electrical Interposer

Electrical IC

PCB

Electrical IC

Optical IC

Fiber

© 2018 Cisco and/or its affiliates. All rights reserved. 20

TDECQ = 1.22 dB

RLM = 0.982

TDECQ = 1.78 dB

RLM = 0.956

TIA Electrical IC

Optical IC

PAM-4 Pattern Generator[106.25 Gb/s PRBS13]

Sampling Scope With

5-Tap EqualizerOpticalElectricalLaser

SourceLiNbO3

ModulatorOptical

PAM-4 Measurement setup

TDECQ = N/A

RLM = 0.956

*K.Lakshmikumaretal,CiscosystemsInc,CICC2018

© 2018 Cisco and/or its affiliates. All rights reserved. 21

FEC Limit = 4.8E-4

Symbol rate = 53.125GBd

RX Sensitivity: Symbol Error Rate vs. Input OMA*K.Lakshmikumaretal,CiscosystemsInc,CICC2018

© 2018 Cisco and/or its affiliates. All rights reserved. 22

§ IP traffic growth → Driving transitions to higher & higher speeds

§ Improving energy efficiency while increasing data rate is difficult with traditional means

§ Silicon photonics based links provide an excellent alternative to III-V as well as backplane copper to improve performance even further§ Low cost§ Wide temperature range§ Improved signal integrity

§ Low voltage CMOS MZI drivers, and linear TIA provides an energy efficient solution for data center interconnects

Conclusion

Thank you.

© 2018 Cisco and/or its affiliates. All rights reserved. 24

Questions?

© 2018 Cisco and/or its affiliates. All rights reserved. 25

Direct Detection Vs Coherent