RED HAT ENTERPRISE LINUX 8 NETWORKING The accelerator for bare metal, virtual, containers, and hybrid clouds Sushil Kulkarni Engineering Manager May 2019 Anita Tragler Technical Product Manager
RED HAT ENTERPRISE LINUX 8 NETWORKINGThe accelerator for bare metal, virtual, containers, and hybrid clouds
Sushil KulkarniEngineering Manager
May 2019
Anita TraglerTechnical Product Manager
APPLIANCE VIRTUAL MACHINE CONTAINER
ACCELERATING APPLICATIONSOne Network to Connect them All
VIRTUAL PRIVATE CLOUD PUBLIC CLOUDBARE METAL
APPLIANCE VIRTUAL MACHINE CONTAINER
RED HAT® ENTERPRISE LINUX® 8 NETWORKING
SERVICES TOOLS
Red Hat Enterprise Linux 8NETWORKING SERVICES
Updated TCP/IP stack● Increased performance● With BBR congestion control
Performance monitoring and network control● eBPF for networking, tracing, firewalls, and filtering
Offloads
● IPsec VPN, TC
NetworkManager 1.14● Default CLI and API to configure services● Reduced footprint
Red Hat Enterprise Linux 8NETWORKING SERVICES
Firewalld 0.6.3● Enabled by default● Nftables backend● Efficient and better performance
IPVLAN● Scalable networking for containers
Ansible Roles
● Seamless network provisioning across RHEL releases● Provision at scale
DPDK for public cloud● Enables fast networking on clouds● Portability on hybrid clouds
USECASE 1 ACCELERATING WEB-SCALE APPLICATIONS
WITH HIGH PERFORMANCE NETWORKING
PLATFORM
ACCELERATING WEB-SCALE APPLICATIONS
Online banking and e-commerce create millions of transactions per secondMaximize TCP setup rate; connections per second (CPS)
Video streaming apps need high bandwidth Maximize TCP bandwidth or Goodput (Gbps)
Chat/VoIP and financial trading have strict latency requirementsReduce TCP round-trip time or HTTP response time
Manage DDoS Attacks Handle high rate TCP SYN flood (pps)
Red Hat Enterprise Linux 8.0
APP
Chat/VoIP Online banking Video streamingE-commerce Live TV Stock trading
Messaging Video conference Online gaming
NETWORK
TCP PERFORMANCE RESULTSRed Hat Enterprise Linux 8.0
RED HAT ENTERPRISE LINUX SERVER KERNEL VERSION MAX TCP CPS MAX SYN FLOOD PPS
Red Hat Enterprise Linux 8.0 4.18.0-80.el8.x86_64 566.40 Kcps (14%⇧) 7.21 Mpps (89%⇧)
Red Hat Enterprise Linux 7.6 3.10.0-924.el7.x86_64 496.09 Kcps 3.82 Mpps
Red Hat Enterprise Linux 7.2 3.10.0-327.el7.x86_64 464.84 Kcps 3.86 Mpps
Red Hat Enterprise Linux 7.1 3.10.0-229.el7.x86_64 417.97 Kcps 0.89 Mpps
Intel Broadwell DELL server : rhel serverIntel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz, core 16, HT enabled, processor 32Disable power management - intel_pstate=disable
Performance numbers for TCP and UDP
USECASE 2 HIGH PERFORMANCE VIDEO STREAMING
ON MOBILE NETWORKS WITH TCP BBR CONGESTION CONTROL
ACCELERATING VIDEO STREAMING
Maximize video Bandwidth and minimize Latency
Spurious packet loss due to poor signal or handoff between Wi-Fi and LTE
High latency (50-120ms) due to excessive buffering (bufferbloat) at ISP or carrier network switches
Limited bandwidth: Wi-Fi and 4G/LTE speeds < 300Mbps
On flaky mobile networks (Wi-Fi and Cellular)
High Speed 10-100 Gbps,Low latency 30%
Content Delivery Network
ISPnetwork
Carriernetwork 4G/LTE
Wi-Fi
Video server
Streamingvideo app
Low Speed < 300MbpsHigh latency 70%
High speed 10Gbps, Low Latency 20ms,
TCP BBR — HOW DOES IT WORK?
Very High speed 40Gbps, Very Low Latency 10ms
Content Delivery Network
ISPnetwork
Carriernetwork 4G/LTE
Wi-Fi
Streamingvideo app
Speed 100Mbps, Latency 70ms, 1% packet loss
TCP ROUND-TRIP TIME RTT = 200ms
Video serverBBR enabled
Ack received
Max data transmit rate(bottleneck bandwidth = 100Mbps)
Bottleneck Bandwidth and Round Trip Time
ACCELERATING VIDEO ON MOBILE NETWORKSTCP max bandwidth—CUBIC throughput—3.3 Mbps; BBR throughput—9.0 Gbps
Fault injectionnetem/tc
RTT = 100msBottleneck BW = 10Gbps
Packet loss 1% Streamingvideo app
Video serverBBR enabled
RT latency = 100ms, speed 10Gbps, 1% packet loss
TCP ROUND-TRIP TIME RTT = 100ms
Max data transmitted (Bottleneck bandwidth = 10Gbps)
Ack received
USECASE 3 EFFICIENT DDOS MITIGATION FOR
WEB SERVERS WITH EBPF XDP
EBPF XPRESS DATAPATH AND TRAFFIC CONTROL
Efficient packet processing with minimum overhead
● eBPF—allows userspace applications to attach programs at different hooks in the kernel
● XDP, TC—hooks for packet processing● Allow packet processing at the earliest point in
the kernel; driver (XDP), TCP/IP (TC)● XDP actions: drop, forward, receive● Data is shared with the application via maps
eBPF XDP, eBPF TC are tech preview for Red Hat Enterprise Linux 8.0
USER
KERNEL
BPF program load and analysis Application
BPF maps (meta data)
TCP/IP stackfirewalls, switching
routing, classification
Network Interface CardNIC NIC
TC bpfclassification
DRIVER
NIC
XDP bpfDrop Forward Receive
EFFICIENT DDOS MITIGATIONeBPF eXpress Datapath and Traffic Control
Cloud admin: Traffic analysis and filteringNetwork services: Load balancer, DDoS mitigation, firewalls, overlay managementApplication security: L4-L7 filtering, cgroup filtering
USER
KERNEL
DDoS BPF program load and analysis
Web Server
BPF maps packet count
TCP/IP stack
Network Interface Card
TC bpfparse pkt
type
DRIVER
XDP bpfDrop Receive
Attacker Web client
firewall nftables packet drop
XDP and TC Tech Preview in RHEL 8.0● Privileged or root access needed● XDP Ingres only; TC both Egress/Ingress● XDP mode: Native, Offload, Generic● TC mode: Default, Offload● FOSDEM 2019 - XDP building blocks● libbpf - Sample XDP/TC tools
EFFICIENT DDOS MITIGATIONeBPF XDP
Avoids pushing packet data from kernel to userspace and back to kernel for packet processing
Real-time updates and modifications to the firewall rules; replace eBPF program on the fly
DDoS attack on a 10G link—with iptables CPU pegged and dropping packets
After XDP_DROP intervention, 50% reduction in CPU usage
Attacker Web Client
Attack description
ebpftools
xdp_kern.c C source
(LLVM) xdp_kern.o
eBPF bytecode
DB Distributed
SERVER
NIC driver
XDP bpf
Drop DDoS attack packets
Performance NumberseBPF XDP
USECASE 4FAST AND ENCRYPTED MULTI-CLOUD
ONLINE BANKING WITH IPSEC CRYPTO OFFLOAD
WHY CRYPTO OFFLOAD?New EU GDPR regulations require securing all financial and personal data
Cryptography goals
Securely send data from one site to another in a reasonably short amount of time
Encryption and decryption should be cheap and fast
Secure encryption needs strong ciphers and keys; compliance with NIST
With new HTTP/3 (QUIC) standard, all web and video traffic will be encrypted
With distributed applications, there is a need for secure Multi-cloud interconnect
PUBLIC CLOUD PRIVATE CLOUD
FAST AND ENCRYPTED MULTICLOUD BANKING
Financial database
server
Cloudgateway
Cloudgateway
Distributed Banking Application with Web Server in
Public cloud and Database (DB) in Private cloud
● Secure encrypted data transfer using IPsec
tunnel from Cloud Gateway to DB server
● Scale to hundreds of connections; large
number of IPsec tunnels
● High Bandwidth for IPsec tunnels for data
aggregation and replication for multiple sites
● High setup rate for certificates and SA key
updates
IPsec TunnelsBanking
Banking
PRIVATE CLOUD PUBLIC CLOUDPUBLIC CLOUD PRIVATE CLOUD
IPSEC CRYPTO OFFLOADRed Hat Enterprise Linux 8.0
CRYPTO ALGORITHMAES-GCM RFC 4106 (symmetric keys) faster encrypt/decrypt, lower CPU utilization
Inline acceleration of ESP
Tunnel or Transport
TCP/UDP performance for all security levels
No IKE offload INLINE OFFLOAD
Application
NIC(EncryptDecrypt)
Crypto engine
Raw data
FAST AND ENCRYPTED MULTI-CLOUD BANKINGTCP bandwidth with and without IPsec crypto offload
IPsec Mode # OF TUNNELS
NO-OFFLOAD: TCP
OFFLOAD:
transport 1 5.12 Gbps 14.2 Gbps
tunnel 1 4.94 Gbps 14.7 Gbps
transport 2 10.04 Gbps 29.0 Gbps
tunnel 2 9.24 Gbps 28.7 Gbps
transport 3 15.03 Gbps 36.6 Gbps
tunnel 3 13.76 Gbps 36.1 Gbps
Financial database
server
Coudgateway
Cloudgateway
Banking
Banking
IPsec Tunnels
PUBLIC CLOUD PRIVATE CLOUD
USECASE 5PTP FOR NFV EDGE WITH 5G RADIO ACCESS
NETWORK (RAN)
vRAN/cRAN EDGE FOR 4G/LTE AND 5G
5G cRAN
Distributed Unit
Edge Compute
Ethernet or fiber Fronthaul
Core DataCenter4G/ LTE EPC or
5G NG-Core
Backhaul
vBBU
DURU
4G/LTE vRAN
CU
GrandMaster ClockT-GM
Radio Unit
DU Slave Clock
CU Slave Clock
Boundary Clock
TRANSPORT NETWORK: < 1 µs (sub-microsecond)
Fronthaul Midhaul Backhaul
Centralized Unit
IEEE 1588 Precision Time Protocol (PTP) Timing Accuracy
Baseband UnitRU
USER
PTP TELECOM PROFILES
VM
CU/DU Slave Clock
NIC1-2
RT-KVM
NIC2-3
RHEL-RT CU/DU slave clock
linuxptp 2.0 update
● Unicast messaging and ● Best Master Clock Alternate (BMCA)
ITU G.8275 telecom profile ● Ethernet multicast and IPv4 Unicast● Ordinary Clock, Boundary Clock
Two-way messagesAnnounce (8/sec), Sync (16/sec), Follow-up, Delay_Req (16/sec), Delay_Resp, Signaling GrandMaster
Clock
System real-time clock
chronyd/dev/ptpx
linuxptp:ptp4l linuxptp:phc2sys
CONTAINER
SERVER
RHEL 8 NetworkingOne Network to Connect them All
Drivers, eBPF XDP, TC &
Crypto Offload
DHCP, DNS, firewalls, PTP
Timing
Services
TCP/IP, TCP BBR, IPVLAN,
DPDK
Hardware
Manageability
Performance
Ansible roles, firewalld,
NetworkManager
Please rate the session on the App…Feedback welcome!