Software Defined Networking COMS 6998- 8 , Fall 2013

Software Defined NetworkingCOMS 6998-8, Fall 2013

Guest Speaker: Seyed Kaveh Fayazbakhsh Stony Brook University11/12/2013: SDN and Middleboxes

Need for Network Evolution

2

New devices

New applications

Evolving threats Policy

constraintsPerformance, Security

3

Type of appliance NumberFirewalls 166NIDS 127Media gateways 110Load balancers 67Proxies 66VPN gateways 45WAN Optimizers 44Voice gateways 11Total Middleboxes 636Total routers ~900

Network Evolution today: Middleboxes!

Data from a large enterprise: >80K users across tens of sites

Just network security$10 billion

(Sherry et al, SIGCOMM’ 12)

Software Defined Networking (COMS 6998-8) 4

There are many middleboxes!Survey across 57 enterprise networks (Sherry et al, SIGCOMM’ 12)

10/22/13


Things to keep in mind about middleboxes• A middlebox is any traffic processing device except for routers and

switches.

• Why do we need them?– Security– Performance

• Deployments of middlebox functionalities:– Embedded in switches and routers (e.g., packet filtering)– Specialized devices with hardware support of SSL acceleration, DPI, etc.– Virtual vs. Physical Appliances– Local (i.e., in-site) vs. Remote (i.e., in-the-cloud) deployments

• They can break end-to-end semantics (e.g., load balancing)10/22/13

Controller PlatformSwitch API

Controller

Switches

App

Runtime

SDN Stack

Control Flow, Data Structures, etc.

6

Applications

Where do middleboxes logically fit in?


Outline• Recent trends in middlebox design/deployment

– Middlebox Consolidation (CoMb)– Extensible Software Middlebox Architecture (xOMB)

• Middleboxes/SDN Integration– Elastic Execution (Split/Merge)– Policy enforcement (SIMPLE)– Handling dynamic traffic modification (FlowTags)

10/22/13





10/22/13

9

Design and Implementation of aConsolidated Middlebox

ArchitectureVyas Sekar Sylvia Ratnasamy Michael Reiter Norbert Egi

Guangyu Shi

Ack: Vyas Sekar

10

Specialized boxes

Narrowinterfaces

“Point”solutions!

Increases capital expenses & sprawl Increases operating expensesLimits extensibility and flexibility

Management ManagementManagement

Key “pain points”

Network-wide Controller

Key idea: Consolidation

2. Consolidate Management

1. ConsolidatePlatform

Two levels corresponding to two sources of inefficiency

11

Consolidation at Platform-Level

12

Proxy Firewall IDS/IPS AppFilterToday: Independent, specialized boxes

Decouple Hardware and Software

Consolidation reduces CapEx

13

Multiplexing benefit = Sum_of_MaxUtilizations/ Max_of_TotalUtilization

Consolidation Enables Extensibility

14

Session Management

Protocol Parsers

VPN Web Mail IDS Proxy

Firewall

Contribution of reusable modules: 30 – 80 %

Management consolidation enables flexible resource allocation

15

N1 N3

N2P: N1 N3

Process (0.4 P)

Today: All processing at logical “ingress”

Overload!

Network-wide distribution reduces load imbalance

Process (0.3 P)

Process (0.3 P)Process (P)


CoMb System Overview

16

General-purpose hardware

Logically centralized e.g., NOX, 4D

Existing work: simple, homogeneous routing-like workload

Middleboxes: complex, heterogeneous, new opportunities


Processingresponsibilities

CoMb Management Layer

Policy Constraints

ResourceRequirements

Routing, Traffic

Goal: Balance load across network.Leverage multiplexing, reuse, distribution

17

Capturing Reuse with HyperApps

18

IDS Proxy

common

Footprint onresource

HTTPUDP

HTTPNFS

2 311

Memory

CPU

HTTP = IDS & Proxy

43Memory

UDP = IDS

NFS = Proxy

13

41

CPU

CPU

Memory

Memory

CPU

HyperApp: find the union of apps to run

Need per-packet policy, reuse dependencies!

HTTP

HTTP:1+2 unit of CPU1+3 units of mem

Modeling Processing Coverage

19

HTTPN1 N3

N1 N2 N3

What fraction of traffic of class HTTP from N1 N3 should each node process?

IDS < Proxy0.4

HTTP: Run IDS -> Proxy

IDS < Proxy0.3

IDS < Proxy0.3

Network-wide Optimization

20

Minimize Maximum Load, Subject to

Processing coverage for each class of traffic Fraction of processed traffic adds up to 1

Load on each node Sum over HyperApp responsibilities per-path

A simple, tractable linear programVery close (< 0.1%) to theoretical optimal

No explicitDependencyPolicy


CoMb System Overview

21

General-purpose hardware

Logically centralized e.g., NOX, 4D

Existing work: simple, homogeneous routing-like workload

Middleboxes: complex, heterogeneous, new opportunities

CoMb Platform

22

NIC

Policy Shim (Pshim)

Core1 Core4

IDS

…

… Proxy

Traffic

Applications

Policy Enforcer

Classification: HTTP

IDS -> Proxy

Challenges: PerformanceParallelizeIsolation

Challenges: LightweightParallelize

Challenges: No contentionFast classification

Parallelizing Application Instances

23

M1 M2

PShim

App-per-core

Core3

M3

PShim

Core1 Core2

HyperApp-per-core

M2 M3

PShim

M1 M2

PShim

Core1 Core2

- Inter-core communication- More work for PShim+ No in-core context switch

+ Keeps structures core-local+ Better for reuse- But incurs context-switch- Need replicas

✔

HyperApp-per-core is better or comparableContention does not seem to matter!

CoMb Platform Design

24

HyperApp1

HyperApp2

HyperApp4

HyperApp3

HyperApp3

PShim PShim PShimPShim PShim

M1 M4 M1 M4M2 M3

Q1 Q3Q2 Q4 Q5

M1 M5

Core 1 Core 3Core 2

NIC hardware

Contention-free network I/O

Core-local processing

Parallel, core-local

Workload balancing

Benefits: Reduction in Maximum Load

25

Consolidation reduces maximum load by 2.5-25X

MaxLoadToday /MaxLoadConsolidated

Benefits: Reduction in Provisioning Cost

26

Consolidation reduces provisioning cost 1.8-2.5X

ProvisioningToday /ProvisioningConsolidated





10/22/13

xOMB: Extensible Open Middleboxes with Commodity Servers

James Anderson, Ryan Braud, Rishi Kapoor, George Porter and Amin Vahdat

28Ack: Rishi Kapoor

29

xOMB

• CoMb focused on consolidated deployment of middleboxes and simplifying network deployment.

• xOMB is a framework for programmable middleboxes using commodity servers.

• Provides low-level functionality necessary for high performance processing

• User defined functionality on top of basic xOMB blocks

30

xOMB Framework

Client Connections

Hardware SwitchLAN F(N) G(N)

F(N) G(N)

F(N) G(N)

Controller

Backend Servers

30

31

xOMB Architecture

Socket I/O

Control Plane

ConnectionManager

Message Reorder Buffer

Client TCP

BufferManager

BackendTCP

xOMB Modules orUser Defined

Pipeline

BasicFunctionality





10/22/13

Split / MergeSystem Support for Elastic Execution in Virtual Middleboxes

Shriram RAJAGOPALAN IBM Research & UBCDan WILLIAMS IBM ResearchHani JAMJOOM IBM Research

Andrew WARFIELD UBC

Ack: Shriram Rajagopalan

Elastic Apps Need Elastic Middleboxes

Elastic AppTier

Flows SDN

IDS/IPS Firewall

Clients

Virtual Middleboxes

LB VPN Accelerator

Alleviating Hotspots

M1 M2

Solution: When M1 is overloaded, provision more middlebox(es) to serve new flows.

Virtual Middleboxes

Elastic AppTier

SDN

Scaling Inefficiencies Lead to Poor Utilization.

M1 M2 M3 M4Virtual Middleboxes

Elastic AppTier

SDN

Alleviating Hotspots

FlowM1 Table

Split/Merge Insight

Virtual Middleboxes

Elastic AppTier

SDN

Intuition: Dynamic partitioning of flow states among replicas enables elastic execution.

FlowTable

FlowTable

Split/Merge Insight

Virtual Middleboxes

Elastic AppTier

SDN

State Inside a MiddleboxOutputFlows

Caches Otherprocesses … Internal to a replica

Threshold Non-criticalcounters statistics

Middlebox VM

Flow TableKey Value

5-tuple [Flow State]

May be shared among replicas (coherent)

Partitionable among replicas

Split/Merge: A State-Centric Approach to Elasticity

VMVirtual NetworkInterface

InternalCoherent

Partitionable(Flow States)

VMVirtual NetworkInterface

SplitReplica 1 Replica 2VM VM

InternalCoherent


Replica 3VM1 2 3


Virtual NetworkInterface!

SplitReplica 1VM

InternalVMCoherent


Replica 2 Replica 3VM VMUnchanged 1 2 3 Coherency is

Interfaces! maintained

MergeReplica 1 Replica 2+3VM VM1 2


Implementation

VM

InternalCoherent


Replica 1 Replica 2

VM

12

VMM VMM

Traffic to Middlebox

Flow 1Flow 2

FreeFlow

Replica 1 Replica 2

�  Need to manage VM VM

application state

12

VMM VMM


Flow 1Flow 2

FreeFlow

� Need to manageapplication state

�  Need to ensureflows are routed tothe correct replica

Rep

lica 1

1

VMM

Flow 1FreeFlow Module

Controller

Replica 2

VM

2

VMM

Flow 2


VM

FreeFlow

� Need to manageapplication state

�  Need to ensureflows are routed tothe correct replica

�  Need to decidewhen to split ormerge a replica

1

VMM

Flow 1

Orchestrator

FreeFlow ModuleController

2

VMM

Flow 2


Rep

lica 1

Replica 2

VMVM

FreeFlow

Forwarding Flows Correctly using OpenFlow

1.1.1.1 (de:ad:be:ef:ca:fe)

Flow Table

OpenFlow Table

<a> port 1

 port 2

<c> port 2

…

OpenFlow Table

<a> port 1 port 1…

port 1 Virtual Switch!

OpenFlow Table

<a>

…

Middlebox Replica 1!

1.1.1.1 (de:ad:be:ef:ca:fe)Flow Table

OpenFlow Switch! port 2 port 1

<c> port 1

…

Virtual Switch! port 1



<c>

…


<c>

Open Flow SW

Open Flow SW

Open Flow SW

MBox Replica 1

MBox Replica 2

Flow Migration


Migrating from replica 2 to replica 1

OpenFlow Table

<a> port 1

 port 2

<c> port 2

…

OpenFlow Table


OpenFlow Table

Flow Table

<a>

…



 port 1

<c> port 1

…




<c>

…


Open Flow SW

Open Flow SW

Open Flow SW

MBox Replica 1

MBox Replica 2


port 2


1. Suspend flow and buffer packets

OpenFlow Table

<a> port 1

 controller

<c> port 2

…

OpenFlow Table


OpenFlow Table

Flow Table

<a>

…



<c> port 1

…!




<c>!

…


Flow Migration

<c>

Open Flow SW

Open Flow SW

Open Flow SW

MBox Replica 1

MBox Replica 2


port 2


Flow Table2. Move flow state to target

OpenFlow Table

 controller

<c> port 2

…

OpenFlow Table



OpenFlow Table!

<a>



…


1.1.1.1 (de:ad:be:ef:ca:fe)Flow Table!

<c> port 1

…

<c>

Virtual Switch! port 1! …


Flow Migration

<c

Open Flow SW

Open Flow SW

Open Flow SW

MBox Replica 1

MBox Replica 2

port 2

<a> port 1


Flow Table3. Release buffer and resume flow

OpenFlow Table

<a> port 1

 port 1

<c> port 2

…

OpenFlow Table

<a> port 1 port 1 port 1

…


OpenFlow Table

<a>



…



OpenFlow Switch! port 2 <c> port 1

…

Virtual Switch! port 1 ..


Flow Migration

<c>

Open Flow SW

Open Flow SW

Open Flow SW

MBox Replica 1

MBox Replica 2

Managing Coherent State

create_shared(key, size, cb) delete_shared(key)

state get_shared(key, flags) // synch | pull |localput_shared(key, flags) // synch | push |local

Strong Consistencycreate_shared(“foo”, 4, NULL)while (1)

process_packet() p_foo = get_shared(“foo”, synch)

Distributed lockfor every update val = (*p_foo)++

put_shared(“foo”, synch)

if (val > threshold) bar()

Middlebox applications rarely need strong consistency!


Eventual Consistencycreate_shared(“foo”, 4, merge_fn)while (1)

process_packet() p_foo = get_shared(“foo”, local)

Hi frequencylocal updates

Periodic globalupdates

val = (*p_foo)++ put_shared(“foo”, local)

if (val > threshold) bar()put_shared(“foo”, push)


Eliminating Hotspots by Shedding Load

Eliminating Hotspots by Shedding Load





10/22/13

SIMPLE-fying Middlebox Policy Enforcement Using SDN

Zafar Ayyub QaziCheng-Chun Tu Luis ChiangVyas Sekar

Rui MiaoMinlan Yu

61

Can SDN simplify middlebox management?Centralized Controller

“Flow” FwdAction… …


OpenFlow

Proxy IDS

Necessity + Opportunity: Incorporate functions markets views as important

Scope: Enforce middlebox-specific steering policies

Firewall IDS ProxyWeb

62

What makes this problem challenging?Centralized Controller



OpenFlow

Proxy IDS

Middleboxes introduce new dimensions beyond L2/L3 tasks.

Achieve this with unmodified middleboxes and existing SDN APIs



Our Work: SIMPLE

LegacyMiddleboxes

OpenFlow capable

Flow Action… …

Flow Action… …

63

Policy enforcement layer for middlebox-specific “traffic steering”

64

Challenge: Policy Composition

S1 S2

Firewall Proxy IDS

Firewall IDS Proxy*Policy Chain:

Oops! Forward Pkt to IDS or Dst?

Dst

“Loops” Traditional flow rules may not suffice!

65

Challenge: Resource Constraints

S1

S2S4

S3

ProxyFirewall

IDS1 = 50%

IDS2 = 50%

Space for traffic split?

Can we set up “feasible” forwarding rules?

66

S1Proxy

S2User 1

User 2

Proxy may modify flows

Are forwarding rules at S2 correct?

Challenge: Dynamic Modifications

Firewall

User1: Proxy FirewallUser2: Proxy

67

Rule Generator

Resource Manager Modifications Handler

SIMPLE System Overview

LegacyMiddleboxes

OpenFlow capable

Flow Action… …

Flow Action… …


68

Composition Tag Processing StateFirewall IDS Proxy

*Policy Chain:

S1 S2

Firewall Proxy IDS

DstORIGINAL Post-Firewall

Post-IDSPost-Proxy

Fwd to Dst

Insight: Distinguish different instances of the same packet

69

Rule Generator



LegacyMiddleboxes

OpenFlow capable

Flow Action… …

Flow Action… …


70

Resource Constraints Joint Optimization

Resource Manager

Topology & Traffic

Switch TCAM

MiddleboxCapacity + Footprints

Policy Spec

Optimal & Feasible load balancing

Theoretically hard! Not obvious if some configuration is feasible!

71

Offline + Online Decomposition

Offline Stage Online Step

Deals with Switch constraints Deals with only load balancing

Resource Manager

Network Topology

Switch TCAM

Policy Spec

TrafficMatrix

Mbox Capacity + Footprints

72

FW IDS ProxyWeb

Rule Generator



LegacyMiddleboxes

OpenFlow capable

Flow Action… …

Flow Action… …

73

Modifications Infer flow correlations

Correlate flows

Install rules

S1Proxy

S2User 1

User 2 Firewall

User1: Proxy FirewallUser2: Proxy

Payload Similarity

74

FW IDS ProxyWeb

Rule Generator (Policy Composition)

Resource Manager(Resource Constraint)

Modifications Handler(Dynamic modifications)

SIMPLE Implementation

OpenFlow 1.0Flow Tag/

TunnelAction

… …

Flow Tag/Tunnel

Action

… …

POX extensions

CPLEX

75

Evaluation and Methodology• What benefits SIMPLE offers? load balancing? • How scalable is the SIMPLE optimizer?• How close is the SIMPLE optimizer to the optimal?• How accurate is the dynamic inference?• Methodology

– Small-scale real test bed experiments (Emulab) – Evaluation over Mininet (with up to 60 nodes)– Large-scale trace driven simulations (for convergence times)

76

Benefits: Load balancing

4-7X better load balancing and near optimal

Optimal





10/22/13

78

FlowTags: Enforcing Network-Wide Policies

in the Presence of Dynamic Middlebox Actions

Seyed K. Fayazbakhsh Vyas Sekar

Jeff MogulMinlan Yu

Network OS

Data Plane

Control Apps

“Flow” Action

… …

Physical View

Logical view: Specify policy goalsAdmin

Middleboxes complicate policy enforcement in SDN

79

Policy routingAccess controlDiagnosticsForensics

Dynamic traffic-dependentmodifications!e.g., NATs, proxies

80

Example: Policy Routing

S1 S2

NAT

Internet

H2

H1

IDSH1: NAT Firewall H2: NAT IDS

Firewall

How do we setup correct forwarding rules?

81

Example: Dynamic Dependence

S1 S2

Proxy

Internet

H2

H1

Web ACL: Block H2 xyz

Get xyz.com

Get xyz.com

Cached response

Response

Cached responses may violate policy

Cached response

Strawman Solutions• Careful placement? (i.e., manual)

– May not always be feasible

• Consolidating middleboxes? (e.g., CoMb)– Just “punting” the problem

• Inferring flow mappings? (e.g., SIMPLE)– Hard to reason about accuracy + high overhead

82

Key missing piece: Lack of “visibility” into middlebox context

FlowTags: High-level Idea• Middleboxes “help” with the lack of visibility

• Add FlowTags to packets to bridge gaps– NAT gives IP mappings; Proxy gives cache hit/miss

• Middleboxes “produce” + “consume” FlowTags

• Switches only “consume” FlowTags

83

84

FlowTags Architecture Overview

ControllerExisting Interfacese.g., OpenFlow

SDN enabledSwitches

FlowTable

Control Apps

FlowTags API

FlowTagsEnhanced

Middleboxes

FlowTagsConfig

e.g., NAT exposes mappingsProxy gives hit/miss stateIDS uses tags to disambiguate

“decouple”

Policy Implementation via FlowTags

S1 S2Proxy

Internet

H2

H1

Input Tag OutProxy 2 H1Proxy 1,3,4 S2

Tag Src Action4 H2 Block

H1, MISS 1H1, HIT 2H2, MISS 3H2, HIT 4

Input Tag OutS1 1,3,4 ACLACL 4 S1ACL 1,3 Internet

ACL

Policy: Block H2 xyz

85

TagsFlowTable TagsActionTable

86

FlowTags Proof-of-Concept• Using Squid (> over 100,000 lines of code)

• About 30 lines of code to add FlowTags support– Manually identify code chokepoints

• Validated use-cases with examples

Software Defined Networking COMS 6998- 8 , Fall 2013

Documents

management consolidation

middleboxesa middlebox

networkwide distribution

totalutilization consolidation

traffic processing device

outlinerecent trends

enterprise networks

load imbalanceprocess