Copyright © 2018 - Barefoot Networks – All rights reserved
Software Transforming the Network
John DeMay
October18th, 2018
Copyright © 2018 - Barefoot Networks – All rights reserved
Software Defined Network (SDN)
Packet
Forwarding Packet
Forwarding
Packet
Forwarding
Packet
Forwarding
Packet
Forwarding
Control
Control
Control
Control
Control
Centralized Network State
Control Plane
Control
Program
Control
Program
Control
Program
Copyright © 2018 - Barefoot Networks – All rights reserved
Software Defined Network (SDN)
?Stagnation &
No innovation
Packet
Forwarding Packet
Forwarding
Packet
Forwarding
Packet
Forwarding
Packet
Forwarding
Control Plane
Control
Program
Control
Program
Control
Program
Centralized Network State ONOS, ODL, Ryu
CORD, NSX, ONAP
“Software is
eating the
network"
A network is a means to forward packets from one
place to another, and modify them along the way.
If we cannot control how packets are forwarded
and processed, we are not really in control.
Copyright © 2018 - Barefoot Networks – All rights reserved
What if you want to….
• Add a new proprietary protocol to your network.
• Remove protocols you don’t need, to simplify your network. So they don’t
waste resources in switches.
• Add private packet headers to your packets, to carry measurement data
through the network.
• Move middle box functions, like L4-LB, SPGW, DNS, NAT, DDoS detection
into the switches. Reduce the number of boxes.
• Add your “beautiful new ideas” into your network. Without telling a switch
chip vendor. Write a program, keep it secret, make your products better
than your competitors.
Copyright © 2018 - Barefoot Networks – All rights reserved
Switch OS
Driver
OSPF BGP etc.
Copyright © 2018 - Barefoot Networks – All rights reserved
Switch OS
Driver
OSPF BGP etc.UPF
Copyright © 2018 - Barefoot Networks – All rights reserved
Network
Equipment
Vendor
Network
Owner
ASIC
Team
Software
TeamFeature
Years
Copyright © 2018 - Barefoot Networks – All rights reserved
When you need a new feature…
1. You can’t just upgrade the software
2. New forwarding features take years to develop
3. Eventually, when the upgraded silicon is available, it
either:
◦ No longer solves your problem, or
◦ You need to build a new product
Copyright © 2018 - Barefoot Networks – All rights reserved
Outline
1. Why programmability is happening now
2. How programmability is being used
Subtract features: Reducing complexity
Add proprietary features: Invent, differentiate, own
Silicon independence: Breaking a lock-in
Telemetry and measurement
Copyright © 2018 - Barefoot Networks – All rights reserved
Domain Specific Processors
CPU
Computers
Java
Compiler
GPU
Graphics
OpenCL
Compiler
DSP
Signal
Processing
Matlab
Compiler
Machine
Learning
?
TPU
TensorFlow
Compiler
Networking
?
Language
Compiler>>>
Copyright © 2018 - Barefoot Networks – All rights reserved
Domain Specific Processors
CPU
Computers
Java
Compiler
GPU
Graphics
OpenCL
Compiler
DSP
Signal
Processing
Matlab
Compiler
Machine
Learning
?
TPU
TensorFlow
Compiler
PISA
Networking
P4
Compiler>>>
Copyright © 2018 - Barefoot Networks – All rights reserved
Conventional wisdom in networking
“Programmable switches are 10-
100x slower than fixed-function
switches. They cost more and
consume more power.”
Copyright © 2018 - Barefoot Networks – All rights reserved
Pipelined Architecture
13
Copyright © 2018 - Barefoot Networks – All rights reserved
PISA: Protocol Independent Switch Architecture
14
Match+Action
Stage
Memory ALU
Programmable
Parser Programmable Match-Action Pipeline
Copyright © 2018 - Barefoot Networks – All rights reserved
PISA: Protocol Independent Switch Architecture
Copyright © 2018 - Barefoot Networks – All rights reserved
Example P4 Program
16
Memory ALU
Programmable
Parser Programmable Match-Action Pipeline
header_type ethernet_t { … }
header_type l2_metadata_t { … }
header ethernet_t ethernet;
header vlan_tag_t vlan_tag[2];
metadata l2_metadata_t l2_meta;
Header and Data DeclarationsParser Program
parser parse_ethernet {
extract(ethernet);
return switch(ethernet.ethertype) {
0x8100 : parse_vlan_tag;
0x0800 : parse_ipv4;
0x8847 : parse_mpls;
default: ingress;
}
Tables and Control Flowtable port_table { … }
control ingress {
apply(port_table);
if (l2_meta.vlan_tags == 0) {
process_assign_vlan();
}
}
Copyright © 2018 - Barefoot Networks – All rights reserved
New features can be deployed to networks in
hours instead of years
Copyright © 2018 - Barefoot Networks – All rights reserved
P4 Community – Growing Momentum
~1500 developers
~ 5000 commits
~1500 followers
~ 800 forks
~ 100 Industry and Academia Members
~ 4 Working Groups
~ 4 Bi-weekly face-to-face meetings
~ 8 Mailing Lists
~ 200 contributors
~ 30 Repositories
~ 12 teams
~ Multiple targets
Independent Consortium
Free to join
Apache 2.0 License
Copyright © 2018 - Barefoot Networks – All rights reserved
P4 Working Groups & Open Community initiatives
19
.org
APIWG
AppWG
Charter
• Data-Plane Telemetry (e.g. INT)
• Security: Heavy-hitter Detection
• Services Offload: (e.g. Layer-4 LB)
• In-Network Cache for distributed services
• In-Network Consensus protocol
Initial Accomplishments
• 30+ Active Customers: OEM and Technology
vendors
• Open Sourced INT and Telemetry Report Specs
LanguageDesign
WG
Architect.WG
IETF 100 – Barefoot delivers first ever hardware-based (Tofino) In-situ OAM implementation:
Barefoot Networks Demonstrates In-situ Operations, Administration and Management (IOAM)
Showcasing the Power of Programmable Forwarding Plane Technology
OVS Orbit PODcast on P4 INT (B. Pfaff, C. Kim): https://ovsorbit.org/#e46
How INT works, upcoming OVS support for INT and SDN vendors involvement
https://p4.org/https://p4.org/https://ovsorbit.org/#e46
Copyright © 2018 - Barefoot Networks – All rights reserved
Outline
1. Why programmability is happening now
2. How programmability is being used
Subtract features: Reducing complexity
Add proprietary features: Invent, differentiate, own
Telemetry and measurement
Copyright © 2018 - Barefoot Networks – All rights reserved
How programmability is being used
Reducing complexity1
Copyright © 2018 - Barefoot Networks – All rights reserved
Compiler
Reducing complexity
Programmable Switch
Driver
Switch OSswitch.p4
IPv4 and IPv6 routing
- Unicast Routing
- Routed Ports & SVI
- VRF
- Unicast RPF
- Strict and Loose
- Multicast
- PIM-SM/DM & PIM-Bidir
Ethernet switching
- VLAN Flooding
- MAC Learning & Aging
- STP state
- VLAN Translation
Load balancing
- LAG
- ECMP & WCMP
- Resilient Hashing
- Flowlet Switching
Fast Failover
– LAG & ECMP
Tunneling
- IPv4 and IPv6 Routing & Switching
- IP-in-IP (6in4, 4in4)
- VXLAN, NVGRE, GENEVE & GRE
- Segment Routing, ILA
MPLS
- LER and LSR
- IPv4/v6 routing (L3VPN)
- L2 switching (EoMPLS, VPLS)
- MPLS over UDP/GRE
ACL
- MAC ACL, IPv4/v6 ACL, RACL
- QoS ACL, System ACL, PBR
- Port Range lookups in ACLs
QOS
- QoS Classification & marking
- Drop profiles/WRED
- RoCE v2 & FCoE
- CoPP (Control plane policing)
NAT and L4 Load Balancing
Security Features
- Storm Control, IP Source Guard
Monitoring & Telemetry
- Ingress Mirroring and Egress Mirroring
- Negative Mirroring
- Sflow
- INT
Counters
- Route Table Entry Counters
- VLAN/Bridge Domain Counters
- Port/Interface Counters
Protocol Offload
- BFD, OAM
Multi-chip Fabric Support
- Forwarding, QOS
Copyright © 2018 - Barefoot Networks – All rights reserved
Compiler
Driver
Switch OSMy
switch.p4
Programmable Switch
Lower Latency
Lower Power
Reducing complexity
Copyright © 2018 - Barefoot Networks – All rights reserved
How programmability is being used
Adding custom features2
Copyright © 2018 - Barefoot Networks – All rights reserved
Custom features
25
• Support for new encapsulations
• Continue to parse inner headers
within encapsulations for
forwarding
• Aggregate and segment different
types of traffic to accelerate
processing
Copyright © 2018 - Barefoot Networks – All rights reserved
Custom features
26
• Offloading of sync/heartbeat
messages at unprecedented
rates
• Decrease detection time for
failures
• Decrease convergence times
• Increase accuracy
Copyright © 2018 - Barefoot Networks – All rights reserved
Example new features using P4
1. New encapsulations and tunnels
2. New ways to accelerate data processing
3. New approaches to routing
4. New approaches to congestion control
Copyright © 2018 - Barefoot Networks – All rights reserved
Example new applications using P4
Example #1: Layer-4 Load Balancer
Copyright © 2018 - Barefoot Networks – All rights reserved
Today: Hardware Layer 4 Load-balancing
Hardware Load Balancers
Application Servers
Clients
10s of Gbps
Copyright © 2018 - Barefoot Networks – All rights reserved
Today: Software Layer 4 Load-balancing
Software Load Balancers Application Servers
Clients
Copyright © 2018 - Barefoot Networks – All rights reserved
P4 Example: Layer 4 Load-balancing
Software Load Balancers Application Servers
Clients
P4 Program
Copyright © 2018 - Barefoot Networks – All rights reserved
Example new applications using P4
Example 1: Layer-4 Load Balancer- Reduce cost and power - replace 200 servers with one Tofino switch- >500x improvement in bandwidth
- Operates at 6.5Tb/s vs. 10Gb/s
- >1000x improvement in latency- All packets load-balanced in sub-ms vs. multi-ms
- Up to 10 million http flows
PublicationSilkRoad: Making Stateful Layer-4 Load Balancing Fast and Cheap Using Switching ASICs.Rui Miao et al. ACM Sigcomm 2017
Copyright © 2018 - Barefoot Networks – All rights reserved
Example new applications using P4
Example #2: Fast Firewall and Audit Trail
Copyright © 2018 - Barefoot Networks – All rights reserved
Today: Hardware Firewall
Hardware Firewalls
Application Servers
Clients
Copyright © 2018 - Barefoot Networks – All rights reserved
P4 Example: Fast Firewall and Audit Trail
Application Servers
Clients
Hardware Firewalls
Copyright © 2018 - Barefoot Networks – All rights reserved
P4 Example: Fast Firewall and Audit Trail
Application Servers
Clients
P4 Program
1 Add new firewall entry on demand
2 Keep audit record for every entry
Copyright © 2018 - Barefoot Networks – All rights reserved
Example new applications using P4
Example #2: Fast Firewall and Audit Trail- Add over 1M new firewall entries per second
- Create audit record for every new flow and firewall entry
- Firewall now inline, at 6.5Tb/s and
Copyright © 2018 - Barefoot Networks – All rights reserved
Example new applications using P4
Example #3: Fast key-value store cache
Copyright © 2018 - Barefoot Networks – All rights reserved
Today: Large Key-Value Store
Key-value Store Servers1/N of key-value store on each server
Clients
1 N
Problem: Small number of HOT entriesCongested servers and large tail latency
Total throughput reduced to R * 2/N
Read rate, R
Copyright © 2018 - Barefoot Networks – All rights reserved
Solution: In-network cache of HOT entries
Key-value Store Servers1/N of key-value store on each server
Clients
1 NSolution: Store HOT entries in cacheHOT entries looked up in
Copyright © 2018 - Barefoot Networks – All rights reserved
Example new applications using P4
Example #3: Fast key-value store cache- In-network cache for 100 servers
- 1-2 billion read/write operations per second
- Hot entries retrieved in
Copyright © 2018 - Barefoot Networks – All rights reserved
How programmability is being used
Network telemetry3
Copyright © 2018 - Barefoot Networks – All rights reserved
“Which path did my packet take?”1“I visited Switch 1 @780ns,
Switch 9 @1.34µs, Switch 12
@2.42µs”
“Which rules did my packet follow?”2
“In Switch 1, I followed rules 75 and
250. In Switch 9, I followed rules 3
and 80. ”
# Rule
1
2
3
…
75 192.168.0/24
…
Copyright © 2018 - Barefoot Networks – All rights reserved
“How long did my packet queue at each switch?”3 “Delay: 100ns, 200ns, 19740ns”
Time
Queue
“Who did my packet share the queue with?”4
Copyright © 2018 - Barefoot Networks – All rights reserved
“How long did my packet queue at each switch?”3 “Delay: 100ns, 200ns, 19740ns”
Time
Queue
“Who did my packet share the queue with?”4
Aggressor flow!
Copyright © 2018 - Barefoot Networks – All rights reserved
The network should answer these questions
1. “Which path did my packet take?”
2. “Which rules did my packet follow?”
3. “How long did it queue at each switch?”
4. “Who did it share the queues with?”
Inband Network Telemetry (INT) can answer all four
questions for the first time. PISA + P4 + INT.p4 runs at
full line rate. Without generating any additional packets!
1
2
3
4
Copyright © 2018 - Barefoot Networks – All rights reserved
Log, Analyze
Replay
INT: Inband Network Telemetry
Add: SwitchID, Arrival Time,
Queue Delay, Matched Rules, …
Original Packet
Visualize
Copyright © 2018 - Barefoot Networks – All rights reserved
Benefits
• Lower MTTR: Reduce MTTR by 1000x from 10-100s of minutes to
seconds identify failures more quickly auto-remediate
• More accurate measurements: Get ns-level measurements vs. ms
granularity 1,000,000x◦ Use this to understand micro-transient events
• Better Visibility: 100,000,000x see all traffic at Bpps vs. just 10s
of pps (sflow type)◦ Identifying all gray failures immediately
• Improved performance & efficiency - Use this data to auto-
provision, auto-load balanced workloads
49
Copyright © 2018 - Barefoot Networks – All rights reserved
In summary
1. Programmable switch chips are here to stay: From now
on, programmable switch chips will be as fast as fixed ones.
Same cost, lower power.
2. Peace of mind: Software developers will always choose a
programmable device over a fixed-function device.
3. Innovation: Beautiful new ideas will be owned by the
programmer, not the chip designer.
4. Accelerate: In-network P4 applications will accelerate
compute services
5. Cambrian explosion: Future networks will be designed
very differently. Unique features in every network.
Copyright © 2018 - Barefoot Networks – All rights reserved
Thank you