Espresso - SDN for Public Internet Sam K. Aldrin On behalf of Google Technical Infrastructure and Google Cloud Platform IETF, July 19th, 2017
Espresso - SDN for Public InternetSam K. AldrinOn behalf of Google Technical Infrastructure and Google Cloud Platform
IETF, July 19th, 2017
Google Global Cache edge nodes
FASTER (US, JP, TW) 2016
Unity (US, JP) 2010SJC (JP, HK, SG) 2013
Points of presence >100
Network fiber
Google NetworkMore than a collection of data centers
#
#
Future regions and number of zones
Current regions and number of zones
3
3
2
3
3 3
3
3
24
3
3
2
Frankfurt
Singapore
S Carolina
N Virginia
Belgium
London
TaiwanMumbai
Sydney
OregonIowa
São Paulo
Finland
Tokyo
Montreal
California
Netherlands
3
3
33
Google Cloud RegionsAdding 11 new regions
Ubiquitous Cloud...10x Scaling
Datacenter
Next-gen disaggregation of storage, memory and compute
Campus & MetroCloud regions and campus expansion driving DC interconnect
WANCloud replication and bandwidth intensive cloud services (e.g., turnkey video, IoT)
10x10x 10x
Step Function Disruptions: Bandwidth, Latency, Availability, Predictability
B4WAN
Interconnect
Andromeda NFV and network
virtualization
JupiterDatacenter Networking
The Pillars of SDN @ Google
B4: [Jain et al, SIGCOMM 13] BwE: [Jain et al, SIGCOMM 15]
B4: Google's Software Defined WAN
10.1.4/24
VNET: 5.4/16
VNET: 192.168.32/24
VNET: 10.1.1/24 Load Balancing
DoS
ACLs
VPN
NFVInternal Network
Andromeda
ToR
Google Infrastructure Services
10.1.1/24
ToR
10.1.2/24
ToR
10.1.3/24
ToR
Watchtower
Saturn
Firehose 1.1
Google Datacenter Network InnovationAnd hardware scale that we could not buy
8
Time
Capa
city
Firehose 1.0
Jupiter
4 Post
1.3Pb/s clusters in 2013
B4WAN
Interconnect
Andromeda NFV and network
virtualization
JupiterDatacenter Networking
The Pillars of SDN @ Google
PublicInternet?
B4WAN
Interconnect
Andromeda NFV and network
virtualization
JupiterDatacenter Networking
The Pillars of SDN @ Google
Espresso SDN for public
Internet
Espresso in Context
B4
Jupiter Data CenterGoogle
Espresso in Context
B4
B2
Peering Metro
Jupiter Data CenterGoogle
Espresso in Context
B4Espresso
B2
Internet
Peering Metro
User
Jupiter Data CenterGoogle
Cloud 1.0Espresso
SDNPeering
RouterCentric
Protocols
Espresso: Before and After
Local viewConnectivity firstCoarse fault recovery
Per-metro and global viewApplication signalsReal-time optimization
Espresso Architecture Overview
Label-switched Fabric
BGP speaker
External Peer
Espresso Metro
Peering Fabric
eBGP Peering
Espresso Architecture Overview
Label-switched Fabric
HostHostHostHostHost
Host
Packet Processor
BGP speaker
External PeereBGP Peering
Espresso Metro
Labeled packets specify egress
HostHostHostHostHost
Peering Fabric
Espresso Architecture Overview
Label-switched Fabric
HostHostHostHostHost
Host
Packet Processor
LocalControl
Global Controller
BGP speaker
External PeereBGP Peering
Espresso Metro
Application Signals
Labeled packets specify egress
HostHostHostHostHost
Peering Fabric
• First things first: an insecure infrastructure is an unavailable infrastructure• Stability is more important than efficiency• Network management is critical• Configuration is hard• Automation matters but can be counter to availability
“Evolve or Die: High-Availability Design Principles Drawn from Google’s Network Infrastructure.” SIGCOMM 2016.
Availability is Paramount
The next wave of computing• Serverless compute in Cloud 3.0• IoT• Tightly coupled, general purpose
distributed computing
It’s time to put it all together• Agile Scale• Jitter• Isolation• Performance is great, but only
meaningful with availability, manageability, and velocity
Next Decade Challenges in Networking
Thank You!Thank You!