DPDK Summit China 2017 · Unable to effectively aggregate the overlay packets in tunnel capsulation and IP ... Virtual Layer 2 Switch (SW) Switch (SW) Micro Segment (MS) Security

DPDK Summit China 2017

Practice of Network Monitoring and Security Technologiesin Cloud Data Center

2

Kai, WangYunShan Networks

Data center is evolving to be cloud basedand software defined

The monitoring and security problemsin SD-CDC

The logical topologies become more and more complex

Difficult to quickly find and locate the network problems in the tenant business

The collection of network data is inefficient

Netflow/sFlow/IPFIX: Sampling, per-packet interrupt & netlink upcall

Limited variety of supported fields for collected flows

The analysis of overlay traffic is insufficient

Unable to do flexible & find-grain traffic collection on demand

Unable to distinguish duplicated traffic from multiple tenants

Unable to effectively aggregate the overlay packets in tunnel capsulation and IP fragments

The physical boundaries of network security disappear

Zero trust for the nodes in internal network

The monitoring solution

、

Hypervisor

vSwitch

Physical resource pool Virtual resource pool

TAP

Physical network layer

Resource layerExporter

Man

agem

ent TAP

Traffic

Traffic

Traffic

、CloudAnalyzer

Splitting

Mirror

、、

Analyzer x86 Cluster

Switch

Analyzer

Controller

Monitoring Fabric

、

The security solution

、

Hypervisor

vSwitch

Physical resource pool Virtual resource pool

Physical network layer

Resource layer

Man

agem

ent

Traffic

Traffic

Traffic

、Security Protection、

Cloud Fabric

Security x86 Cluster

Controller

SwitchTraffic

Traffic

Traffic

Our solution: hypervisor based DFI (Deep Flow Inspection)

Probe utilizing OvS in Hypervisor

Overlay traffic collection

Kernel module + Userspace agent + OvS action

Cons: invasive deployment

Stability Problems: crash, soft lockup

Influence to tenant business

Our solution: VM based DFI

Deployed in VM

Mirror overlay traffic to VM

Performance bottleneck vswitchd ovsdb

openvswith.ko (datapath)OvS

Kernel

Userspace

VM VM VMę

DFI

agent

dfi. ko

Exporter VM

vswitchd ovsdb

openvswith.ko OvS

vswitchd ovsdb

openvswith.ko (datapath)OvS DFI

agent

dfi.ko

Kernel

Userspace

VM VM VM VMę

Technology evolution forvirtualized networks monitoring

Our current solution: DPDK based Utilizing OvS-DPDK

Fully exploit the compute resource of VM

Extend functions based on OvS-DPDK conntrack ACL

Flow generation

Packet header extraction and compression

DPI

NPB

SDN More efficient, flexible, benefit for debug Used for physical networks monitoring as well

Kernel

Userspace

vswitchd

dpif-netdev (datapath)pkt_dedup, pkt_slicing, pkt_mask, pkt_timestamping, flow_gen, flow_slicing,

flow_pkt_hdr_extract, mod_qinq/vlan, vxlan_encap/decap, dpi, ...

ovsdb

Exporter VM

uio_pci_generic

OvS-DPDK

Technology evolution forvirtualized networks monitoring

NIC Multi-queue & Symmetric RSS VM template

Parallelize conntrack processing Make it scalable

Optimize the datapath classifier (dpcls) algorithm Tuple Space Search (TSS) HyperSplit algorithm

Intel vTune Amplifier Lock, Polling & Interrupt

Open vSwitchKernel

NIC

Guest OS

Virtio

DPDK

Network APP

Guest OS

Virtio

Com

puterN

ode

VM Exporter VM

Guest OS

Virtio

VM

Further optimization for exporter

Cluster-based analyzer

Use Storm to do real-time analysis

DDoS/Port Scan

Abnormal connections/transactions, Abnormal login

ARP/MAC/IP Spoof

Loop detection

Use Spark to do off-line analysis

Security analysis model

Use ElasiticSearch/Kibana to do search and visualization

Customized statistics in different dimensions

Trace back of historical events

Third-party analysis tool

E.g. SQUIL, SQL injection detection

Analysis & Visualization

Use the monitoring results to generate security policies Exporter

Overview the security problems & risks in cloud networks

Analyzer Locate the problematic nodes

or areas

Controller Prevent/Protect these nodes

or areas via SDN

Exporter

Analyzer

ControllerFlow-based Data

More and more complex networks

Underlay& Overlay

Big-scale Support

High-perf & Parallel

Big Data

Machine

Learning

Real-tim

e &

Off-line

Automated Policy

Operational Decision

AI

Virtualized

No Border

Business D

riven

From monitoring to security control

Use VNF to do security detection/prevention Based on VXLAN

Pros Elastic and flexible

Cons Inefficient and low-performance, hard to

cover the large-scale east-west traffic VXLAN encap/decap load

Poor scalability of security service chain

vSwitch and VNF performance bottlenecks

VM1 VM3

VM4 VM5

Security Service Chain Orchestration

Controller

vSwitch

VM1 VM2 VM3 VM4 VM5

Compute Node

Service Chain 1

VXLAN Networking

IPS Pool FW Pool

vSW/VTEP vSW/VTEPvFW vIPS

vSW/VTEP vSW/VTEPvFW vIPSService Chain 2

Security service chain and problems

Use VLAN instead of VXLAN to introduce traffic to assigned security nodes Offload VXLAN encap/decap to ToR switch,

save more CPU for SSE processing table=0,priority=202,dl_vlan=2000,ip,actions=output:20

table=0,priority=102,in_port=10,dl_vlan=0xffff,ip,actions=mod_vlan_vid:2000,resubmit(,0)

Virtual Layer 2

Switch (SW) Switch (SW)

Micro Segment (MS) Security Service Element (SSE)

Micro Segment (MS)

Micro Segment (MS) Security Service Element (SSE)

Security Service Element (SSE)

Micro Segment (MS)

Micro Segment (MS)

Micro Segment (MS)

Security Service Element (SSE)

Security Service Element (SSE)

Security Service Element (SSE)

……

…

……

…

……

……

Traffic TractionRules

Traffic TractionRules

…

Compute Pool

vSW

vSW

vSW

vSW

vSW

vSW

… …

VM VM VM VM VM VM VM…

VM VM VM VM VM VM VM…

Security Service ChainSecurity Service Chain

SSE SSE SSE SSE SSE SSE SSE…

VXLAN VXLAN VXLAN VXLAN

VLAN

VLAN

Underlay

Overlay

VM VM VM VM VM VM VM…

SSE SSE SSE SSE SSE SSE SSE…

SSE SSE SSE SSE SSE SSE SSE…

Security Pool

…

MS-2

MS-1

VM VM VM

VM VM VM

SSE-1

SSE-2

SSE-3

SSE-N…

…

…

vSW Traffic Traction Policies

…

…

Performance optimization

Single VNF/SSC has limited performance Use SDN policies based trade-off to

dispatch traffic to multiple chains Based on pseudo node Linearly increase the performance

E.g. priority=401,table=0,dl_vlan=1000,ip,tcp,

tp_src=0/0x0001,tp_dst=0/0x0001,actions=mod_vlan_vid:2000,resubmit(,0)

priority=401,table=0,dl_vlan=1000,ip,tcp,tp_src=1/0x0001,tp_dst=1/0x0001,actions=mod_vlan_vid:2000,resubmit(,0)

priority=401,table=0,dl_vlan=1000,ip,tcp,tp_src=0/0x0001,tp_dst=1/0x0001,actions=mod_vlan_vid:3000,resubmit(,0)

priority=401,table=0,dl_vlan=1000,ip,tcp,tp_src=1/0x0001,tp_dst=0/0x0001,actions=mod_vlan_vid:3000,resubmit(,0)

VM2-4

VM1-1

SSE2-1

SSE1-2

SSE2-3

SSE2-4

vSW ACL Policies

SSE1-1

SSE2-2

SSE1-3

SSE1-4

vSW Trade-off Policies

Performance optimization

Use OvS-DPDK to accelerate the networking in security resource pool

Use DPDK to accelerate SSE TOPSEC

DPDK vhost-user-clientOpen vSwitch + DPDK

NIC

SSE

Network APP

Guest OS

Virtio

SecurityN

ode DPDK PMD

SSE

Network APP

Guest OS

Virtio

DPDK vhost-user-clientOpen vSwitch + DPDK

DPDK

SSE

Guest OS

Virtio

SecurityN

ode DPDK PMD

NetworkAPP

DPDK

SSE

Guest OS

Virtio

NetworkAPP

vswitchd

datapath

NIC

VM

Guest OS

Virtio

VM

Guest OS

Virtio

Open vSwitch

NICNIC NIC NIC

Com

puterNode

Performance optimization

SQL injection attack detection

Kibanavisualization

DDoS situational awareness

Custom developmentLB+vFW+vIPS

x86 KVM Cluster, OvS-DPDKSecurity Cloud

SLB Cluster

OpenStack

SDN Switch

SDN Switch

vFW vIPS VNF……

ControllerTraffic traction via route

ISP

Core Router

control

HA&

LB Securityanalysis and protection

Security cloud

DPDK China Summit 2017 Shanghai，

Thanks!!

欢迎关注DPDK开源社区