Data Plane Acceleration Open Source...Multi-architecture: x86, IBM, Freescale, EZChip(Tilera) support Supports KVM, VMWare and HyperV plus Linux Containers • DPDK has a large application

Data Plane Acceleration Open Source Margaret T. Chiosi

AT&T Labs Distinguished Network Architect

Open Platform for NFV – OPNFV President (Linux Foundation )

Bob Monkman, ARM

Keith Wiles, Intel

Uri Elzur, Intel

© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other AT&T marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. The information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change

2007 2008 2009 2010 2011 2012 2013 2014

2

CUSTOMER DEMAND


Faster

Highly Secure

More Flexible

More Reliable

Easier to Scale

Cheaper Agile

RETHINKING THE NETWORK


3

4 © 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other AT&T marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. The information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change

VIRTUALIZE AND CONTROL

Business Internal & External

Voice

Strategic Data

Mobility Video

Hardware Platform

AT&T Integrated Cloud Platform

Software Platform

All Legacy non-D2.0 WAN, Access, Cloud, Sites, etc.

5 5 5

Infrastructure • Infrastructure Controllers • IaaS • Compute, Storage,

Network • Hypervisor (KVM, ESXi)

Network • Virtual Network Functions

(e.g. vEdge, vCE) • SDN/Network Controllers

(Packet, Optical, Overlay)

Application • Reusable platform capabilities • Modular virtual functions &

apps, XaaS • Application Controllers • Common Data Repositories • Cloud Software Elasticity

Legacy Base Infrastructure

AT&T Finished Products

Scope of Domain 2.0 N

ot

D2

.0

ECOMP • Overall AIC control,

orchestration, mgt, policy • SD&C, MSO, A&AI, DCAE • Control loop pattern • Overall platform & app

support components

Do

mai

n 2

.0

Next Gen WAN • Packet Optical White Boxes

Physical Infrastructure Assets • Compute & Storage • Site commons (power, cooling,

racks, etc.)

Next Gen Access • Wireline Broadband White Boxes

(GPON, G.Fast) • Wireless Broadband White Boxes

(xRan, etc.)

Next Gen LAN/Underlay • Local Network and Fabric • Topology scaling

Programmable platform with API based automation, analytics & security throughout

Common / Reusable in Support of AT&T Finished Products & All Other Workloads

------------------------------------------------------------------------- Loose Coupling ---------------------------------------------------------------------------

Includes All Workloads


VPN Connectivity for Virtual Private Clouds – Target

6


Virtual Private Data Centers

VPN-1

VPN-2

CBB

RR

Customer-1 Sites

Customer-2 Sites

VRF VRF

IRB I-PE/ I-CE

Service Orchestrator

IRSCP

Subnets Subnets

Subnets Subnets

Mo

bile

an

d B

road

ban

d

Inte

rnet

Acc

ess

VM

vSwitch/Rtr

Network Orchestrator

VM

VM VM

Domain 2.0 – 3 Pillars, 1 platform, 1 vision


7

New Paradigms for AT&T and Telecommunications

NFV

SD

N

ECO

MP

AT&T integrated Cloud

Customers

Cycle Time

Composition

Efficiency

SDN NFV

eCOMP

People Process

Culture

SDN NFV

eCOMP

Business Programs Voice, Strategic Data, Mobility, Video

AIC Request Interface

AT&T Integrated Cloud High-level Architecture

AT&T Technical Centers (IT & Mobility Centers, U-verse, Central Offices)

Shared Server, Storage and Network

Virtual Machines

Virtualization Abstraction (Openstack)

Network and Application Orchestration

AIC

AP

I In

terf

ace

Laye

r

Platform Business and Network Applications, Services

or Functions. Examples: • OS Image • DB • Web Services • vRouter • SDN • DNS • FW • Network on Demand / NoD

Policies and analytics defined for each of the Applications/Virtual Network Functions run within a common control loop for policy, orchestration and management.

Commodity, white box vendor agnostic equipment and software.

Handles the request(s). This includes workflow assemble of AIC services and resources, configuration of end services unique to the request as needed and sends completion confirmation back.

Handles the provisioning and assembly of cloud services and lower level resources.

Notes: 1. APIs are used between layers. 2. APIs are used between services and components within a layers 3. This architecture includes Opensource, vendor and AT&T code.

Controls and manages the AIC infrastructure resources.

Containers Bare metal servers


8

AT&T Integrated Cloud Distribution Choices


Virtualization of Home and Enterprise Networks

Legend VNF Hardware resources

Hardware resource pool

BS BS

CO

Virtualization of CDNs

Virtualization of Base Stations (vBS)

Virtualization of Fixed Access

Virtualization of Mobile Core/ IMS

HW HW HW HW

CSCF SGW CSCF MME

DNS DHCP SGW SGW Firewall

HW HW HW HW

MME CSCF LB DHCP

HW HW HW HW

CSCF PGQ App

Server App

Server App

Server

HW HW

RGW NAT STB

HW HW

CDN 1 CDN 2

HW HW

vBS LTE

vBS 3G

vBS 2G

vBS WiMax

HW HW

C-plane ADSL

C-Plane VDSL

C-Plane ITU-T/G

HW

Big Deployments / Low Distribution (<50 sites)

• Centralized Functions

• Large scale facilities, e.g., compute centric

National

Medium Deployments / Distribution (300-600 sites)

• Regional/metro functions

• Hub locations, proximity centric

Regional/Metro

Small/Medium Deployments / Large Distribution (5000+ Sites)

• End Office, LATA, and MOW functions

• Access hand-off points, highly proximity centric

End Office

Small / Very Small Deployments, Extreme Distribution (100K to 100M)

• Remote terminal, campus/venue

• Customer premises CPE, Customer devices

Remote

Trend to distribute functions towards the edge to reduce cost of transport, latency, concentration/disaster risk

Trend to centralize functions into the network to reduce Access & CPE TCO

© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other AT&T marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. The information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change 9

10 © 2014 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property. AT&T Proprietary (Internal Use Only) Not for use or disclosure outside the AT&T companies except under written agreement.

10

Transformation: Converging to Single, Common Infrastructure

H

H

100’s+ to 1000’s


Broad range of locations:

• Data Center

• Central Offices

• Huts

• Customer Premises

Therefore broad range of servers based on different chips – Intel, ARM, AMD, Power….

Broad range of applications – Networking, Web, IT, Video, Voice, …

Networking requires high throughput techniques

Expediency in solving the throughput challenges

Platform Requirement


11

Application Requirement in Virtualized World


Intel, ARM, Power, MIPS & Other Platforms

Linux

Hypervisor

V N F

V N F

V N F

Need one framework to support VNFs running on different platforms • Today there are two leading proposals

with similarities, but also fundamental differences

• Requires collaboration to converge and support multiple use cases

OVS/R

V N F

ODP ODP

NIC SoC

DPDK DPDK

Bring Data Plane to NFV Applications via DPDK

13

• Open-source (BSD license) community project 5+ years, current version is 2.1 – http://dpdk.org All code is completely Open Source from 1.2 release (Sept 2012 01.org) Multi-architecture: x86, IBM, Freescale, EZChip(Tilera) support Supports KVM, VMWare and HyperV plus Linux Containers

• DPDK has a large application install base and included in Linux Distro’s CentOS, Ubuntu, Red Hat, …(Fedora)

Included in frameworks like VirtIO/Vhost and OpenvSwitch Integrated into many projects: OVS, mTCP, Pktgen, xdpd, NetVM, MICA, ….

• Encompasses legacy platforms and newer acceleration platforms Common API to shield the VNF developer from the hardware differences No compromise in performance for DPDK or ODP/SoC devices Supports a common binary for NFV platforms

• Scalable solution to meet different NFV use cases VM to VM traffic as an example via OVS/DPDK

• Hardware acceleration complementing software for consistent VNF services


13

http://dpdk.org

SUPPORTING ODP IN THE DPDK FRAMEWORK


DPDK – Architecture

DPDK-AE (Acceleration Enhancements)

DPDK – API

Software

Hardware

SoC SDK

ODP PMD

external memory manager 3rd Party

VNF Application

Crypto Device + others

Simple model for SOC integration

Ethernet Device

ODP PMD: Poll Mode driver model for ODP SoC devices Provides a clean integration of SoC/ODP via a PMD in DPDK

• Hardware abstraction in DPDK is at the PMD layer

• DPDK-API: A generic API extended to support SoCs

– DPDK provides a two layer device model to support many devices at the same time/binary

– Need to enhance DPDK with some SoC specific needs

– DPDK runs on all platforms Intel, ARM, MIPS, Power, …

– Large application supported features Hash, Rings, lpm, …

• ODP-PMD: Poll Mode Driver model for ODP

– Retains ODP’s current code base

• Unifies DPDK and ODP to create one solution for NFV

FreeScale, EZchip,

Broadcom, AMD

Open Data Plane API • General data plane API to enable networking applications to

accelerate across most any silicon on any architecture

• Open support for ARM, x86, MIPS, Power 7, Other Architectures

• Structured to enable future innovation

• Lightweight abstraction preserves performance without prescribing lower –level processing structure

• Unique, Direct Access and Management of HW accelerators

• Supports optional schedulers in an event driven model to provision easy management and dynamic traffic load balancing

• Vendor run-time environments sit underneath to preserve vendor optimizations

ODP embraces and extends existing, vendor-optimized RTEs, enabling an efficient, truly cross-platform standardized data plane programming interface

ODP is more than a packet

processing framework- It is a

Data Plane Application-level

Programming Interface


15

Community of ODP Applications

ODP Application

ARM MIPS or

ARM

ODP API Layer

QorIQ – DPAA (Power or ARM)

Application can run in simple Polled Mode (DPDK Mapping) or Advanced Event Driven Model for maximum Multicore Scalability

SDK N Cavium

SDK Freescale

SDK DPDK

X86

ODP - Application View ODP demonstrated to run at negligible overhead on several Platform SDKs Intel x86/DPDK a work in progress. Currently achieve <2% overhead Developers can always to choose to use underlying interfaces- at expense of portability


16

Applications in a Container World


Intel, ARM, Power, MIPS & Other Platforms

Linux

V N F

V N F

V N F

Need one framework to support applications running on different container systems. As both DPDK and ODP run on the same platforms which interface is to be used?

NIC

V N F

ODP ODP

SoC

DPDK DPDK

OVS/R

DPDK ODP

Different vendor applications running on same version of Linux or even same Linux vendor – a lot of tweaking based on a vendor implementation

Security Concerns due to sharing same host OS

Running in Root – own name space solves this?

Container Challenges in the Networking World


vSwitch Challenges… and Opportunities

Many demands as adoption

picks up, need to support …

Increased E W traffic

Network Appliances place new demands

Higher core density =>

higher VM VM traffic

L3 based multi-tenant segmentation

Security, Isolation, SLA

Monitoring, visibility

Many implementations with varying

features, performance and API

Some requirements for High performance

VMs and Network Appliances

High performance VM (40Gb / 100G NICs)

Adequate throughput, low CPU utilization

Low latency, Controlled latency (jitter)

As-is Open vSwitch small packet performance may fall behind NFV applications needs!


NIC-OVS-NIC Performance and Features • Performance has many aspects. Clearly NFV/SDN requires top performance to meet workload

requirements and meet economical feasibility and infrastructure efficiency goals

• Performance numbers and gaps are presented below as reference

• Single flow / Single core 64 byte UDP raw datapath switching performance with pktgen

• Latency

• OvS features are required too e.g. NSH, DPI, QoS, DPDK,

• Yet, some features are much slower to make it into OvS while other (e.g. Geneve, OVN) make it to main stream while not more stable or more advanced in standards…

• How does the community get to a more open process for new features? Performance improvements?

STANDARD-OVS DPDK-OVS LINUX-BRIDGE

Gbits / sec 1.159 9.9 1.04

Mpps 1.72 14.85 1.55

Source: http://openvswitch.org/support/ovscon2014/ OVS performance measurements and analysis (Madhu Challa, Noiro Networks)

OVS DPDK-OVS LINUX-BRIDGE NIC-NIC VM-OVS-OVS-VM

TCP 46 33 43 27 72.5

UDP 51 32 44 26.2 66.4


20

http://openvswitch.org/support/ovscon2014/

http://openvswitch.org/support/ovscon2014/

AT&T is moving quickly to an SDN enabled AT&T Integrated Cloud

Performance of the AIC platform (DPDK, ODP, OVS, OVR) is critical to get to 75% by 2020

DPDK-ODP convergence is important for the deployment of 1000s of AT&T Integrated Clouds

Summary


21

© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other AT&T marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. The information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change 22

Data Plane Acceleration Open Source...Multi-architecture: x86, IBM, Freescale, EZChip(Tilera) support Supports KVM, VMWare and HyperV plus Linux Containers • DPDK has a large application

Documents