Top Banner
DPDK on Azure Past, Present and Future Stephen Hemminger DPDK Summit Userspace - Dublin- 2018
19

DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

Apr 19, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

DPDK on AzurePast, Present and Future

Stephen Hemminger

DPDK Summit Userspace - Dublin- 2018

Page 2: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

TL;DR

● DPDK on Azure – Today– DPDK support is in production– Lastest Enterprise distributions support it– Many partners are updating their applications

Page 3: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

Topics

● Why is Azure different?● How does it work?● Need to Know● What should I do?

Page 4: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

The problem

High performance

&

Security

Page 5: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

Azure SmartNIC (FPGA)

● HW implementation of SDN– Programmable Generic Flow

Tables– QoS– Overlay Network

Host

CPU

NICASIC

FPGA

SmartNIC

ToR

Page 6: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

Azure Networking Internals

Page 7: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

Linux AN

NDISVMBus

User

Kernel

TCP/IP

Application

ConnectX3 VF

Mlx4NetVSC

Page 8: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

NDISVMBus

User

Kernel

TCP/IP

SR-IOV VF

NetVSC

TAPPMD

Fail Safe PMD

TAP

Mlx4 PMD

RDMA

MLX4Provider

DPDKApplication

TCredirect

DPDK AN

Page 9: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

NDISVMBus

User

Kernel

SR-IOV VF

HV_UIO

Netvsc PMD Mlx4 PMD

RDMA

MLX4Provider

DPDKApplication

DPDK AN

Page 10: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

Netvsc PMD

● Why bother?– works with or without SR-IOV– Less overhead– Smaller– Simpler

Page 11: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

UIO driver

● Pass VMBus to Userspace– Like igb_uio

● Handle control channel– Subchannel creation

TransmitRing

ReceiveRing

MonitorPages

InterruptPage

TransmitBuffer

ReceiveArea

Page 12: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

Failsafe vs Netvsc PMD

● Failsafe– Stable

● Kernel 4.2● DPDK 18.02

– Supported● Ubuntu, SLES, RHEL, CentOS

– Bifuricated model– Requires SR-IOV

● Netvsc PMD– Experimental

● Kernel 4.18● DPDK 18.11

– Device Assignment– Vswitch or SR-IOV

Page 13: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

DPDK on Azure Roadmap

2017● Azure AN preview● Ubuntu 18.04+● DPDK 17.11

2018● Azure AN default● Ubuntu, SLES, RHEL, Debian,

CoreOS, Oracle, …● DPDK 18.02● Netvsc PMD

– 18.08 early– 18.11 AN

Page 14: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

Performance

● Accelerated Networking– SR-IOV 40G – Direct Device Assignment 10G

● Fallback– TAP (failsafe)– Netvsc PMD

NOT Azure Testpmd transmit only, single queue

Windows server 2016, Ubuntu 18.04, Xeion E5-1620

K Packet/sec0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

SR-IOVDDATapNetvsc

Page 15: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

Backports

● Older versions of DPDK?– Netvsc PMD need bus (17.11)– Failsafe etc needs vdev (17.11)– vdev_netvsc needs devargs

(18.02)

Page 16: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

OS support● Linux

– Netvsc needs uio_hv_generic● 4.16 single queue● 4.18 multi queue

– TAP needs eBPF/flower● Classifier 4.2● RSS 4.9

● FreeBSD?– TAP issues– VMBus UIO

Page 17: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

Future Possibilities

● Performance– Ring copy avoidance– Optimize descriptor

● Offloads– Flow– RSS parameters

● FreeBSD?● ARM?

Page 18: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

MellanoxMatan AzradRaslan DarawshehMoti HamovoskyEyal MizrachiThomas MonjalonOphir MunkOlga Shern

6WindGaetan RivetAdrien MazarguilOlivier MatzPascal MazonThomas Monjalon

MicrosoftJosh PoulsonRay CuiMadhan SivakumarGabriel SilvaSimon XiaoHaiyang Zhang

IntelWei DeiDeclan DohertyGage EadsRadu NicolauBruce RichardsonJianfeng TanKeith WilesFerruh YigitQi Zhang

Hyper-V and Azure Credits

BrocadeChas Williams Eric KinzieJan BlunckNachiketa Prachanda Sven-Thorsten Dietrich Wen Chiu

Page 19: DPDK on Azure · Ubuntu, SLES, RHEL, CentOS ... – Experimental Kernel 4.18 DPDK 18.11 – Device Assignment – Vswitch or SR-IOV. DPDK on Azure Roadmap 2017

Questions?Stephen Hemminger

[email protected]