End-to-End GMPLS Signaling in CHEETAH Project Xiangfei Zhu xzhu@cs.virginia.edu 5/5/2005 Master’s Project Presentation.

End-to-End GMPLS Signaling in CHEETAH Project

Xiangfei Zhu

xzhu@cs.virginia.edu

5/5/2005

Master’s Project Presentation

Outline

CHEETAH project overview GMPLS signaling overview End host software for GMPLS signaling External GMPLS signaling Engine for Cisco

15454 Network setup and experiments Conclusion and future work

CHEETAH Project

CHEETAH: Circuit-switched High-speed End-to-End Transport ArcHitecture[1]

Supported by NSF Experimental Infrastructure Networks (EIN) Project

Goal: enable dynamically controlled rate-guaranteed connections

between end hosts Application:

eScience community, especially, Terascale Supernova Initiative (TSI ) High speed file transfer Remote visualization

CHEETAH Network OverviewNC

NCSU

Controlcard

OC192card

GbE/10GbEcard

X1ORNL

SN16000

OC192card

Controlcard

GbEcard

NCSU/MCNC/NLR

SN16000

OC192card

Controlcard

Atlanta (NLR/SOX)

SN16000

OC192card

OC-192

OC-192

GbE/10GbEcard

…Centaur Lab

Optical Network Signaling

GMPLS: Work in progress at IETF RFC2205 – RSVP for IP network RFC3209 – RSVP-TE for MPLS RFC3471 & RFC3473 – RSVP-TE for GMPLS RFC3946 – SONET and SDH

Optical Internetworking Forum (OIF) UNI, I-NNI, E-NNI

International Telecommunications Union (ITU) G.8080 (G.ASON)

Vendor Support to GMPLS

Some vendors provide varying-level support for GMPLS in their products eg: CIENA CoreDirector, Sycamore SN16000,

etc. Successful multi-vendor GMPLS

interoperability demo at ISOCORE The implementation of GMPLS signaling by

different vendors are basically compatible

University GMPLS Code

KOM RSVP Engine – Technische Universitat Darmstadt [7]

partial support of RFC 2205, 2210 & 3209 Dragon RSVP-TE code – MAX/ISI[4]

partial support of RFC 3471, 3473 & 3946 Use of the university GMPLS code

GMPLS client on end hosts External GMPLS engine

My contributions

Implement a GMPLS software for end host – “Bandwidth manager” Integrating GMPLS signaling with

Admission control OCS (Optical Connectivity Service)

Integrate with 15454 control software CHEETAH network setup and interoperability

test with Sycamore SN16000

End host Context

Routing decision: Decide use CHEETAH circuit or Internet to transfer the file base on the Internet congestion status and file size

FRTP: Fixed-Rate Transport Protocol designed for circuit-switched network[6]

Bandwidth Manager: Dynamic provision of the circuit

InternetPC

NIC I

PC

FRTP FRTP

Bandwidth Manager

FTP

TCP/IP TCP/IP

FTPRouting decision

Routing decision

NIC II NIC IICHEETAH

Network

NIC I

[3]

Bandwidth Manager

Bandwidth Manager Architecture

Bwmgr

SocketInterface

DNS

Control-planeAddress, Linktype, ...

Configure File

CAC

RSVPD

bwrequestor

Routing Decision

Application

End Host

RSVP_API

RSVP ClientProcess Pool

RSVPReceiver

OCS

bwadmin

FRTP

bwlib

Update ARP/IP table

Bandwidth Manager Daemon

Read the configuration from a configuration file.

Initiate circuit setup. Accept the bandwidth request Check if destination is in

CHEETAH network (OCS) Bandwidth management (CAC) Create RSVP session and send

out PATH message Update ARP/IP table

Accept the circuit setup requests Listen to PATH message If it is a new session

Perform CAC Create a new session and send back RESV Update ARP/IP table

Bwmgr

SocketInterface

DNS

Control-planeAddress, Linktype, ...

Configure File

CAC

RSVPD

bwrequestor

Routing Decision

Application

End Host

RSVP ClientProcess Pool

RSVPReceiver

OCS

bwadmin

FRTP

bwlib

Update ARP/IP table

RSVP_API

PATH

Bandwidth Manager User Interfaces

bwadmin - provide an interfacefor system administrators The configuration set by

administrator is stored in a configuration file.

Commands supported: bwadmin show bwadmin set control-plane-ip IPADDR bwadmin add telink …. (TElink information)

bwadmin del telink ID bwrequestor - provide command for end users to

request a bandwidth. bwrequestor DESTINATION-DOMAIN-NAME BANDWIDTH

Bwmgr

SocketInterface

DNS

Control-planeAddress, Linktype, ...

Configure File

CAC

RSVPD

bwrequestor

Routing Decision

Application

End Host

RSVP_API

RSVP ClientProcess Pool

RSVPReceiver

OCS

bwadmin

FRTP

bwlib

Update ARP/IP table

Bandwidth Manager Configuration File The configuration file includes:

The control-plane address of the node TE-link information:

TE-link ID Bandwidth and type (Ethernet/SONET) Interface types (numbered or unnumbered) of the two

interfaces (local and remote) IP (numbered interface) / IFID (unnumbered) of each

interface A sample configuration file

control-plane-IP = 128.143.137.155# TE-Links# TELinkID bandwidth(unit: Mbit) link type (0-Ethernet, 1-SONET) local interface type (0-unnumbered, 1-numbered) local interface IP/ID remote interface type (0-unnumbered, 1-numbered) remote interface IP/IDTELink1 1000 0 0 1 0 1TELink2 500 0 1 192.168.2.1 1 192.168.2.2

My contributions

Implement a GMPLS software for end host – “Bandwidth manager” Integrating GMPLS signaling with

Admission control OCS (Optical Connectivity Service)

Integrate with 15454 control software CHEETAH network setup and interoperability

test with Sycamore SN16000

External GMPLS Engine for Equipment without GMPLS Capability

Dragon’s VLSR (Virtual Label Switching Router)[4] as an external GMPLS engine. RSVP-TE message parsing and construction is done Fabric programming module for some Ethernet switches through

SNMP Adopt VLSR for Cisco 15454

Monfox TL1 Library Allows for an external program to provision circuits by issuing TL1

commands to 15454 Difficulty: Library in Java while the Dragon code is in C++

Figured out how to integrate Java code with C++ through CNI (Cygnus Native Interface) (by Lingling Cui)

Integrate Dragon’s RSVP-TE software with 15454 control software

External GMPLS Engine for CISCO 15454

15454Controller(Monfox

DynamicTL1)

Control Plane

PC

Cisco-15454

RSVPD

TL1

VLSR

Control Plane

Data Plane Data Plane

PATH

RESV

My contributions

Implement a GMPLS software for end host – “Bandwidth manager” Integrating GMPLS signaling with

Admission control OCS (Optical Connectivity Service)

Integrate with 15454 control software CHEETAH network setup and interoperability

test with Sycamore SN16000

July 26, 2004 First interoperability at Sycamore

SN16000 #1

SN16000 #2 SN16000 #3

RSVP-TE Client RSVP-TE Client

Sycamorelocal-area network

User planeSONET Link

Control planeSignaling Link

Dynamically set upSONET Circuit

July 26, 2004 First interoperability at Sycamore

SN16000 #1

SN16000 #2 SN16000 #3

RSVP-TE Client RSVP-TE Client

Sycamorelocal-area network

User planeSONET Link

Control planeSignaling Link

Dynamically set upSONET Circuit

PATH message

RESV message

Did find some bugs in Dragon’s code but bugs were fixed on-the-fly

Nov 6-12, 2004 Supercomputing 04’

Demo 1: A web application integrated with: dynamic circuit setup and release (RSVP-TE)

bandwidth manager on end host external RSVP-TE control engine for Cisco 15454

MSPP a transport protocol designed for dedicated

circuits (FRTP – Fixed Rate Transport Protocol) Demo 2: Signaling interoperability with

Sycamore SN16000

Mar 23, 2005CHEETAH NC Deployment

NC

NCSU

Controlcard

OC192card

GbE/10GbEcard

X1ORNL

SN16000

OC192card

Controlcard

GbEcard

NCSU/MCNC/NLR

SN16000

OC192card

Controlcard

Atlanta (NLR/SOX)

SN16000

OC192card

OC-192

OC-192

GbE/10GbEcard

…Centaur LabOC192

card

Controlcard

GbEcard

NCSU/MCNC/NLR

SN16000

OC192card

SN16000

Atlanta (NLR/SOX)

May 28, 2005 CHEETAH Atlanta Deployment

NC

NCSU

Controlcard

OC192card

GbE/10GbEcard

X1ORNL

SN16000

OC192card

Controlcard

GbEcard

NCSU/MCNC/NLR

SN16000

OC192card

Controlcard

Atlanta (NLR/SOX)

SN16000

OC192card

OC-192

OC-192

GbE/10GbEcard

…Centaur Lab

Controlcard

OC192card

GbE/10GbEcard

Experiments with Sycamore SN16000– SONET-to-SONET

OC192 OC192 GbE

3435

1-6-1-1

1-6-17-1

1-7-1-1

1-7-17-1

Route ID/Switch IP/Eth Address:

128.109.34.18

Eth1

128.109.34.18

1-8-33-1

SN16000-NC End Host 2End Host 1

Internet

SONET TE-Link

Fiber

Provisioned Cross Connection

Dynamic Cross Connection

RSVP-TE Signaling

Performance

For repeated circuit setup and release, average circuit setup time is 159ms.

However, the first circuit setup is usually around 270ms. The extra time is suspected to be consumed by

ARP and probably routing table lookup

Experiment with Cisco 15454

15454CONTROLLER

(MonfoxDynamicTL1)Control Plane

PC

CISCO-15454

RSVPD

TL1

VLSR

Data PlaneData Plane

PC I

NIC I

NIC II

PC II

NIC I

NIC II

Performance

Performance of external GMPLS engine for MSPP[5] Time for crossconnection setup:

STS-1: 17.833 ± 0.184 ms STS-3: 18.000 ± 0.081 ms

Time for crossconnection delete: STS-1: 16.400 ± 0.175 ms STS-3: 16.300 ± 0.145 ms

Conclusion

It is feasible to extend dedicated circuits to end hosts by running RSVP-TE software on end hosts

It is feasible to add GMPLS signaling capability to devices without build-in GMPLS capability

The standards are mature and vendor implementation is good

Future Work

Development part Complete the implementation Hand out to scientists to use

Research part Bandwidth scheduling of circuit-switched network Immediate call vs. scheduled call Distributed bandwidth scheduling

Thank you!

Reference[1] http://cheetah.cs.virginia.edu/[2] CHEETAH overview, John H. Moore, Xuan Zheng, Malathi

Veeraraghavan, http://cheetah.cs.virginia.edu/networks/Cheetah%20Overview.jpg

[3] CHEETAH network, Malathi Veeraraghavan, Nagi Rao, July 7, 2004

[4] http://dragon.east.isi.edu/[5] External Switch Control Software, Lingling Cui, CHEETAH project

year 1 demo, September 01, 2004[6] X. Zheng, A. P. Mudambi, and M. Veeraraghavan,

FRTP: Fixed Rate Transport Protocol -- A modified version of SABUL for end-to-end circuits, Pathnets2004 on Broadnet2004, Sept. 2004, San Jose, CA

[7] KOM RSVP Engine, http://www.kom.e-technik.tu-darmstadt.de/rsvp/[8] Monfox DynamicTL1 SDK, http://www.monfox.com/dtl1_sdk.html

Acronym CHEETAH – Circuit-switched High-speed End-to-End Transport

ArcHitecture RSVP – Resource Reservation Protocol RSVP-TE – RSVP – Traffic Engineering GMPLS – Generalized Multiple Protocol Label Switching SONET – Synchronous Optical NETwork SDH – Synchronous Digital Hierarchy IETF – Internet Engineering Task Force RFC – Requests for Comments UNI – User-Network Interface I-NNI – Internal-Network-Network Interface E-NNI – External-Network-Network Interface