Mellanox and Nuage Networks SDN for High Performance

Copyright 2013 Alcatel-Lucent. All rights reserved.CONFIDENTIAL - SOLELY FOR AUTHORIZED PERSONS HAVING A NEED TO KNOW

PROPRIETARY – USE PURSUANT TO COMPANY INSTRUCTIONNuage Networks

Mellanox andNuage SDNforHighPerformance

Christoph Andreas Torlinsky – Technical Business Developer EMEA [email protected] London City UK – December 1st 2016

©2016Nokia.Allrightsreserved.Nuage NetworksisaNokiaventure.CONFIDENTIAL- SOLELYFORAUTHORIZEDPERSONSHAVINGANEEDTOKNOW

PROPRIETARY– USEPURSUANTTOCOMPANYINSTRUCTION

Nuage SDNVRSPoweredbyMellanox VXLANOffload

Wire speed performance

SDN flexibility, agility and security

Highest infrastructure efficiency

Accelerated OpenVswitch



Assessing Stateless VXLAN Offload NICs

1. Performance:§ Throughputandlatencychanges§ CPUutilizationoverhead§ End-to-endpacketlatency§ SingleOVSBridgewithSDN(inthiscaseNuage)

2. Availability:§ Active-standbyredundancysupport§ Latencyonsinglelinkfailure

12/8/163

FactorstoconsiderwhendeployingVXLANandOpenVswitch forOpenStack



DatapathOptimizationwithSDNandacceleratedOVSDifferentiationtoNeutronwithNuage VRS

br-intint-br-ext

VM1TenantA

VM2TenantA

VM3TenantB

eth0eth0eth0

qbraqvba

vneta

qvoa

qbrbqvb

b

vnetb

qvob

qbrcqvbc

vnetc

qvoc

TAP Device

veth pair

Linux Bridge

Open vSwitch

VM3TenantB

eth0

qbrdqvb

d

vnetdPHY Port

qvod

br-extphy-br-ext

Internal Router Namespace

qr-f qr-g

IP IP IP IP

IP IP

qr-fqrouter-yInternal Router Namespace

qr-h qr-jIP IP

qr-n qrouter-z

Floating IPNamespace

qfloat-x qf-nqr-m

qf-xbr-tun

int-br-tun1

int-br-tun1

FlowTableentry

FlowTableentry

DVR AGENT

(Enhanced L3 Agent)

PrivateNetwork

eth1

PublicNetwork

eth0

Ext-IP

alubr0VRS

(Single OVS bridge)

o SingleOVSBridgeo IsFlow-Basedo PerformsFirewalling,

Switching,Routing,NAT,…

o ProcessesARP,DHCPLOCALLY

o NoDedicatedNetworkNodeforo non-DVRcase:

Routing,DNAT,SNAT,DHCP

o DVRcase:SNAT,DHCP



Nuage Networks VSP VXLAN Performance Testbed - KVM Compute Node

KVM

1GbE

VM VM

KVM

1GbE

VM VM

KVM

1GbENIC

br0

Switch Data/MgmtPlane(192.168.50.0/24)

Public(10.21.0.0/24).153

.32 .31 .40

.152 .151 .150

V2(RHEL7.1) V1(RHEL7.1) Mgmt(CentOS 7.1)

VXLAN alubr0alubr0

NICVRSVRS

NIC

VSD VSC



Nuage Networks VSP VXLAN Performance Testbed Setup

KVM

VM VM

KVM

VM VM

Data/MgmtPlane(192.168.50.0/24)

.31 .32V1(RHEL7.1) V2(RHEL7.1)

VXLAN alubr0alubr0

NICVRSVRS

NICCPU

Throughput(Gbps)

LoadGeneration(tx):netperf -H<overlayIP>-tTCP_STREAM-- -m64koriperf -c<overlayIP>–P12

CPU

Listeners(rx):netserveroriperf -s



NIC Configuration

CheckNICsettingsusingethtool -k§ MostNICsreportthesameinfo§ NotallNICsupportthefeaturesEnable:checksum,GRO,GSO,UDP

TunnelSegmentationDisable:LRO,Tx Cache

[root@vrs1~]#ethtool -keth1Featuresforeth1:rx-checksumming:ontx-checksumming:on

tx-checksum-ipv4:ontx-checksum-ip-generic:off[fixed]…

scatter-gather:ontx-scatter-gather:ontx-scatter-gather-fraglist:off[fixed]

tcp-segmentation-offload:ontx-tcp-segmentation:ontx-tcp-ecn-segmentation:off[fixed]

…generic-segmentation-offload:ongeneric-receive-offload:onlarge-receive-offload:off…receive-hashing:on…tx-udp_tnl-segmentation:on…tx-nocache-copy:offloopback:offrx-fcs:offrx-all:offtx-vlan-stag-hw-insert:offrx-vlan-stag-hw-parse:on

12/8/167



VXLAN Offloading for 1 VM Pair Throughput

[root@localhost Runs]#../utils/dp1Long.shdemo|grep SUM[SUM]0.0- 3.0sec3.42GBytes 9.80Gbits/sec[SUM]3.0- 6.0sec3.33GBytes 9.53Gbits/sec[SUM]6.0- 9.0sec3.67GBytes 10.5Gbits/secßOFFLOADSDISABLED…[SUM]33.0-36.0sec3.64GBytes 10.4Gbits/sec[SUM]36.0-39.0sec3.26GBytes 9.32Gbits/sec[SUM]39.0-42.0sec3.74GBytes 10.7Gbits/sec[SUM]42.0-45.0sec5.94GBytes 17.0Gbits/secßOFFLOADSENABLEDD[SUM]45.0-48.0sec10.2GBytes 29.2Gbits/sec[SUM]48.0-51.0sec10.1GBytes 28.9Gbits/sec[SUM]51.0-54.0sec9.95GBytes 28.5Gbits/sec

12/8/168



Nuage Networks VSP VXLAN 1 VM Pair Performance

KVM

VM

KVM

VM


VXLAN alubr0alubr0

NICVRSVRS

NICPPS

Throughput (Gbps)Offloads Off: 10.7Offloads On: 29.9

VM:CLIENTTX:.19Mpps RX:.48Mpps

PPS

HV:CLIENTTX:.91Mpps RX:.480Mpps

HV:SERVERTX:.48Mpps RX:.91Mpps

VM:SERVERTX:.48Mpps RX:.93Mpps



Nuage Networks VSP VXLAN 1 VM Pair Performance

KVM

VM

KVM

VM


VXLAN alubr0alubr0

NICVRSVRS

NICPPS

Throughput (Gbps)Offloads Off: 10.7Offloads On: 29.9

VM:CLIENTTX:.19Mpps RX:.48MppsTX:.13Mpps RX:.48Mpps

PPS

HV:CLIENTTX:.91Mpps RX:.480MppsTX:.13Mpps RX:.48Mpps

HV:SERVERTX:.48Mpps RX:.91MppsTX:.20Mpps RX:2.50Mpps

VM:SERVERTX:.48Mpps RX:.93MppsTX:.20Mpps RX:2.50Mpps



VXLAN Throughput Nuage VSP on ConnectX-4 Lx (using Netperf)

12/8/1611

Mellanox VXLANOffloadsEnableLineSpeedThroughput

§ 2-5xhigherthroughput

§ Greatestbenefitforlightloadsindicating

§ Latencyreduction

§ ConnectX-3Prohasverysimilarresults



VXLAN CPU Savings Nuage VSP on ConnectX-4 Lx (using Netperf)

12/8/1612

Mellanox VXLANoffloadsfree CPUandsystemresources

§ 30-60%improvementinCPUefficiencyfromVXLANoffloading

§ A1%changeinefficiencyrepresents40%systemsbenefitat40Gpbsthroughput

§ CPUUsage=LoadGeneration+Networking



Redundant ConnectX NIC VXLAN Offload PerformanceLinuxActive-BackupBondPerformance

• RedundantNICconnectionsdonotimpactVXLANoffloadthroughput

• VXLANoffloadingreducesCPUusageby25%forheavilyloadedbondedNICconfigurations

• Nodetectableperformancedegradationduringsingleconnectionfailure

12/8/1613



Summary and What’s Next?

§ Today:NuageNetworksSDNonMellanox ConnectX-4worksverywell:§ StatelessVXLANoffloadsenablelinespeedat40Gbps§ 2-5xhigherthroughput§ 30-60%higherCPUefficiency§ Similarresultsforredundantactive-backupbonds

§ Tomorrow:§ Overlaynetworksbecomeincreasinglytransparent§ Moveto25/50/100GbEwillshowsimilaroffloadbenefits§ DPDKbypassingkernelcontextswitching§ Stateful offloadswillfurtherenhanceVXLANdeployments(SDNintegratedOVS

ontheNIC)

12/8/1614

©2016Nokia.Allrightsreserved.NuageNetworksisaNokiaventure.CONFIDENTIAL- SOLELYFORAUTHORIZEDPERSONSHAVINGANEEDTOKNOW

PROPRIETARY– USEPURSUANTTOCOMPANYINSTRUCTION 12/8/1615

NuageNetworksDataCenter

VSD VSC

lab01.nuagex.io

PrivateDataCenter

TestdriveVSPforfreeatNuageX at:http://nuagex.ioDeployonbaremetal andtestit!

©2016Nokia.Allrightsreserved.NuageNetworksisaNokiaventure.CONFIDENTIAL- SOLELYFORAUTHORIZEDPERSONSHAVINGANEEDTOKNOW


1612/8/16

ThankYouLondon!

@nuagenetworks

Mellanox and Nuage Networks SDN for High Performance

Technology