© 2008 AT&T Intellectual Property. All rights reserved. March 27, 2009 Page 1 Clouds and Networks: Technology and Network Infrastructure Evolution Charles.

© 2008 AT&T Intellectual Property. All rights reserved. March 27, 2009Page 1

Clouds and Networks:

Technology and Network Infrastructure Evolution

Charles (Chuck) Kalmanek

V.P. – Networking and Services Research

AT&T Labs


Network Infrastructure Evolution

Networks evolve in response to the changing nature of the traffic and advances in component and system technology.

Changes in traffic are driven by advances in computing technology and applications.


Router Scalability

– Growth in network loado Data plane: traffic load

o Control plane: number of Internet prefixes, OSPF topology, LDP labels, etc.

– Currently require fork-lift router upgradeso Significant CapEx costs

o Additional costs: manpower hours for certification, upgrade management tools, reduction in network reliability

– Architectural reorganizations can improve network manageabilityo Especially at the customer / aggregation / edge


Evolving the Traditional ISP Architecture

• How to handle failures and planned maintenance at network edge?– Treat AR’s as a resource pool,

similar to blade servers

– Migrate to a spare router, similar to taking a blade out of the load balancer rotation

• How to handle control plane scale issues at network edge?– Install new routers and migrate

to them

– Feature incompatibilities, differences in configs add complexity

AR

CE CE CE CE

BR BR

PoP

Access Router

Backbone Router

XC

AR


1. Extract customer configuration from initial router

2. Install customer configuration on to target router

3. Reconfigure transport (layer 1) connectivity

4. Wait for network to converge

5. Verify service

1. Extract customer configuration from initial router

2. Install customer configuration on to target router

3. Reconfigure transport (layer 1) connectivity

4. Wait for network to converge

5. Verify service

BGPBGP

RouterFarm in Action*(Network Migration)

Transport Network

ISP Backbone

* Agrawal, Bailey, et al., RouterFarm: Towards a Dynamic Manageable Network Edge, INM’06


Application 2Replace a router

with N physical ones

Extending Access Router Life?– Use multiple physical routers to form a logical, Composable Router*

– A Composable Router appears as a single routing entity to rest of networko Distribute control and data plane load among physical routers

o When load increases, add more routers to Composable Router

o Reduce upgrade frequency and associated costs

Application 1Form a virtual access routerfrom multiple access routers

* Ee, Breslau, Ramakrishnan, REAP: Router Extensibility via Address-based Partitioning


Basic Idea– Start with a bunch of existing routers

o These form an array, we call each of them an array router

– Need to distribute control and data plane load within the arrayo Let each array router be responsible for a subset of address space

o Divide address space into blocks, assign to array routers

Array

•Reduces per-router prefixes, labels

•Reduces forwarding load per router

Block 1

Block 2

Block 3

Block 4

Block 5

IPv4 space


Basic Idea (data plane)

– Splitter serves as distribution and aggregation point for arrayo Local packet routing is static, on per-interface basis

– Distribute incoming data packets based on dest IP

– Aggregate outgoing data packets based on static mappingo E.g., 802.1q VLAN tags

– Simple, large scale splitter /switch is needed

Array

Splitter


Basic Idea (control plane)

– Control packet distribution, aggregation handled by meta-routero Splitter classifies and passes only control packets to meta-router

o Splitter - meta-router link need not be of high-capacity

o Meta-router distributes control packets, e.g. based on prefix advertised

o Again, incoming interface at splitter determines interface leading to array router (which also maintains state for that address block)

Array

Splitter

Composable Router

Meta-router

© 2008 AT&T Intellectual Property. All rights reserved. March 27, 2009Page 10Page 10

Road to Compute-based Infrastructure

Source: Intel®Source: Intel®(with permission) (with permission)

2000 2012+

10X10X

SINGLE ThreadSINGLE ThreadWith Multi-coreWith Multi-core

MULTI-COREMULTI-CORE

2006

FORECASTFORECAST

Perf

orm

ance

Perf

orm

ance

3X3X

You AreYou AreHereHere

Normalized Performance vs. Initial Intel® Pentium® 4 Processor

Asymmetric/Symmetric Multiprocessing And

Virtualization Supported


Network Distributed Computing

• Distributed Computing + Network => Services– Network application services

– Cloud / utility computing

– Software as a service

• Central office => data center– Data center switching

– Virtualized computing

– Storage

– Security

• Example: video content delivery gateway


Content Delivery Gateway

IPTV

MBMSDVB-H DSLAM

RANRNC

TVPC

BroadbandWireless

DSLAM

RG

STB

DSL

WiMax

ContentDeliveryGateway

CO CO CO

WiFI

BroadcastMedia Library

NationalContent

SHENDC

ApplicationServers

HSSCSCFBGCFCDGCMGC

IMSCore

ISCOAM&P

Metro Network

NTE

MediaGateway

WebServicesGateway

Metro Network Metro Network

VHO VHO

IPTV-AS

Cellphone

SecureMedia

Distribution

IP MPLS Transport Network



WebServices

Internet

PSTN

BroadcastAdsVODEPG

LocalContent

ContentSource

Gateway

VHO

SecureMedia

Distribution

RAN BAN HSAN

dlna

DSL

PON


Video Content Delivery Gateway: Functionality

TranscodingTranscodingTranscodingTranscoding

Digital Rights ManagementDigital Rights Management

Content SecurityContent Security

Image ProcessingImage ProcessingImage ProcessingImage Processing

Ad OverlayAd OverlayAd OverlayAd Overlay

Key ManagementKey Management

Dynamic Ad SplicingDynamic Ad SplicingDynamic Ad SplicingDynamic Ad Splicing

High Performance Computing TechnologyHigh Performance Computing TechnologyHigh Performance Computing TechnologyHigh Performance Computing Technology

ManagementControl

Zone Ad TargetingZone Ad TargetingZone Ad TargetingZone Ad Targeting

Unicast / BroadcastUnicast / BroadcastUnicast / BroadcastUnicast / Broadcast

Hardware/Software AccelerationHardware/Software AccelerationHardware/Software AccelerationHardware/Software Acceleration

IMS Network ManagementIMS Network ManagementIMS Network ManagementIMS Network Management

IMS Session ControlIMS Session ControlIMS Session ControlIMS Session Control

ContentTransformation

DigitalRights

Management

AddressableAdvertising

PlatformTechnologies

PacketManagement

Packet Management (Transport / Service Control)Packet Management (Transport / Service Control)Packet Management (Transport / Service Control)Packet Management (Transport / Service Control)

NEBs High Availability Bladed PlatformNEBs High Availability Bladed PlatformNEBs High Availability Bladed PlatformNEBs High Availability Bladed Platform

Network Monitoring / SecurityNetwork Monitoring / SecurityNetwork Monitoring / SecurityNetwork Monitoring / Security


Cloud Computing

• Lease computation and storage resources on demand

• Highly dynamic resource provisioning– Add new servers within minutes

– Easy to replicate virtual resources

• Only pay for what you use

• Several emerging services – Amazon EC2, IBM Blue Cloud, Google

App Engine, Microsoft Azure, AT&T Synaptic Hosting, etc.

Cloud Platform


What is Missing?

• Control over network management– Can’t request specific IP addresses– Can’t put VMs on own private network

• Control of Network Resources– Bandwidth, traffic isolation, etc

• Lack of network security and isolation– VMs have IP on public internet– Customer must manage security on VM

itself

at&ttop secret

Verizonpay roll


CloudNet: Bringing VPNs to the Cloud*

• Use VPNs to separate customer resources

• Customer’s VMs are only reachable from her other VPN end points

• More flexible control of how IP addresses are assigned

• Physical network is transparent to customer

VPLS

* Collaboration bet. U. Mass (P. Shenoy, T. Wood) & AT&T Labs (J. van der Merwe, K. K. Ramakrishnan)


System Components

• Cloud Manager– Create VMs

– Resource Allocation

– Controls up to CEs

• Network Manager– VPN management

– Access controls

– Controls PEs

• May be separate business entities

Cloud 1 Cloud 9

NetworkManager

CloudManager

…CE CE


VPN Management

• All endpoints need to “match”

• Making changes to all endpoints is a pain!

• Use IRSCP– Centralized VPN manager

– Looks like route reflector

– Speaks BGP to PEs

• Rewrites VPN route targets

IRSCP

IRSCP Rules:

VPN 1 = + +

VPN 2 = +

Takes about 5-8 seconds


Shadownet

• Provides infrastructure for CloudNet

• Uses Juniper router support for logical routers– Subdivide a physical router

• Instantiates arbitrary networks based on topology description

• Simplifies and automates router configuration– Tracks links, used interfaces, VLAN ids, etc

Site 1 Site 2


CloudNet Prototype

Logical View

Physical Instantiation

PE

CE

VM

VM

CE

VM

VM

PECEVM

VM

Customer W Cloud E

PE

Customer S

PE

Cloud N

PECEVMVM

PECECE

VMVM

VMVM

PE CECE

VMVM

VM

VMPE

VM

VMCE


Summary

• Evolution in computing technologies continues to change the nature of network infrastructure

• Router scalability– Architectural reorganization at network edge holds promise

• Network distributed computing – Distinction between central offices and data centers is

breaking down

– Integration of cloud computing and VPNs provides isolation and security to enterprise customers


Thank you!

Questions???

© 2008 AT&T Intellectual Property. All rights reserved. March 27, 2009 Page 1 Clouds and Networks: Technology and Network Infrastructure Evolution Charles.

Documents

att intellectual property

network loaddata plane

router scalabilitygrowth

router extensibility

spare router

data plane load

traffic loadcontrol

multiple physical routers