Enabling New Applications with Optical Circuit-Switched Networks Xuan Zheng April 27, 2004

Enabling New Applications with Optical

Circuit-Switched Networks

Xuan Zheng

April 27, 2004

2

Outline Background and problem statement Proposed RESCUE service Application I: High-speed optical Dial-Up

Internet access service using RESCUE circuits

Application II: end-to-end RESCUE circuits to improve file transfer delays

Implementation of application II Summary

3

Background Current optical network architectures

Ethernetswitch/

IP router

Enterprise building

Ethernethosts

Access service providernode

Metro opticalaccess network

Internet serviceprovider router

Internet - Packet Switched backbone network(IP routers interconnecting various networks)

Metro opticalcore network

Leased lines

Inter-switchcircuitsWide-area

optical network

Current optical network applications Leased access circuits for enterprise users High-speed inter-switch/inter-router circuits

4

Gaps between User Needs and Current Network Solutions

Access link bottleneck problem Date rates of access links are still slow. Access links are often heavily utilized.

TCP limitations TCP is not suited for High-Delay-Bandwidth-Product

(HDBP) networks because of its congestion control scheme.

Hard to create end-to-end connections to provide QoS for interactive real-time applications

Current Internet is connectionless.

5

Prior work In packet-switched networks

Packet-switched ring (RPR) is proposed for access links Increasing the circuit rate does not help a lot if the packet

loss rate remains high. TCP enhancements are proposed to achieve high end-to-

end TCP throughputs HighSpeed TCP, Scalable TCP, FAST TCP, etc. Did not touch the shared nature of Internet; no end-to-end

QoS guarantee. QoS in IP based networks

IntServ, DiffServ, TCP switching, etc. Implemented at IP routers instead of end hosts. Not scalable, especially when traffic is large.

6

Prior work In circuit-switched networks

Traditionally, bandwidth-on-demand is primarily focused on inter-switch/inter-router circuits in service provider networks.

Fast restoration and rapid provisioning Centralized resource management with human

interventions Latest efforts on bandwidth-on-demand

UCLP in Canarie network, ESnet, etc. Provide user-controlled end-to-end optical circuit

provisioning Still centralized approach Applications are limited to the elephant data transfer and

other eScience applications in a small community Too costly Does not scale for commodity service

7

Problem Statement Design new network architectures

exploiting advances in optical switching technologies to bridge the gaps between user needs and network limitations. High-speed circuit switches Dynamic distributed control with

signaling/routing protocols

8Other Enterprises

Optical circuit-switched networkEthernetswitch/IP router

Ethernethosts

RESCUE circuit

NIC 2 NIC 1

Enterprise building

To ISP's router

To ISP's router oranother signaling-capable

network switch

MSPP EthernetInterface

From other endhosts

SONETInterface

Primary Internetleased access circuit

Application +RESCUE software

OS

Proposed Architecture: Reconfigurable Ethernet/SONET Circuits for End Users (RESCUE)

SETUP

SETUP

SUCCES

S

SUCCESS

Second leased line

Second NIC

Software upgrade

9

RESCUE: An “Add-on” Service to Primary Internet Access

Two paths between two entities: the primary TCP/IP path and an Ethernet/SONET circuit.

Packet-switched Internet

Packet-switched Internet

End host I

End host II

Optical Circuit-switched Network

Optical Circuit-switched Network

“Parallel-hybrid” architecture vs. traditional “sequential-hybrid” architecture

10

RESCUE: Applications

High-speed optical Dial-Up Internet access service

End-to-end file transfers

Gap #1

Gap #2

11

Dial-Up server(signaling

+configuration

software)

Internet serviceprovider

SONETMSPP

Ethernetswitch/IP router

Ethernethosts

User space

EthernetInterface

RESCUE circuitfor Dial-Up service


Ethernetswitch/IP

router

From otherend hosts

ARP tableMap MAC addresses

to newly setupRESCUE circuit

Routing tableMap IP address to

newly setupRESCUE circuit

NIC 2 NIC 1

SONETMSPP

Enterprise building

Optical circuit-switchedaccess network


OS

Application I: Dial-Up Internet Access Service using RESCUE Circuits

7min delay Transfer 0.01P50ms,T 100Mbps,r

100MB f

primarypropprimary

19sec delay Transfer 0.00001P50ms,T Mbps,4r

10sec delay Transfer 0.00001P50ms,T 100Mbps,r

dialuppropdialup

dialuppropdialup

5

12

Application II: End-to-end RESCUE Circuits to Improve File Transfer Delays

Internet - Packet Switches(IP routers interconnecting

various networks)

Optical circuit-switchednetworks

SONETMSPP


Kernalspace

Ethernethosts

User space

EthernetInterface

From otherend hosts

NIC 2 NIC 1

Enterprise building

SONETMSPP


Kernalspace

Ethernethosts

User space

EthernetInterface

From otherend hosts

NIC 2NIC 1

Enterprise building


RESCUE circuit for End-To-End file transfer

service


OS


OS

Use new transport protocols other than TCP on end-to-end RESCUE circuits

hours 15.3 and days 4 delay Transfer 0.0001P50ms,T 1Gbps,r

1TB f

primarypropprimary

hours 2.28000secTf/r delay Transfer 50msT 1Gbps,r proprescueproprescue 2/

13

Application II: Analytical Basis for the Routing Decision - Delay Analysis

circuit the of rate data the :

dtransferre being file the of size the :

circuit RESCUE the on file the transfer to time the :

link. access primary the using file the transfer to time mean the :

setup, circuit failed a of delay setup-call mean the :

setup, circuit successful a of delay setup-call mean the :

network,

switched-circuit optical the on yprobabilit blocking-call the :

(1)

rescue

prop

ctransfer

tcp

fail

setup

b

tcpfailbtransfersetupbrescue

r

f

T

r

fT

TE

TE

TE

P

TETEPTTEPTE

2

][

][

][

])[][()][)(1(][

14


:get we ,to equal be to ingapproximat By ][][ setupfail TETE

setup circuit attempt if

(3) path TCP/IP the to directly resort if

transfertcpb

setup

transfertcpb

setup

TTEP

TE

TTEP

TE

][1

][

][1

][

setup circuit attempt if

(2) path TCP/IP the to directly resort if

][][

][][

tcpresuce

tcpresuce

TETE

TETE

with (1) from Compare ])[][ tcpresuce TETE

15


2000. March

Israel, Aviv,-Tel 1751,-1742 pp. 3, vol. Infocom, IEEE of proceeding

”Latency, TCP Modeling" Anderson, T. and Savage, S. Cardwell, N. [2]

2001. February 46,-31 pp. 9, vol. ,Networking on nTransactio IEEE/ACM

”,Validation Empirical its and Model Simple A :Throughput TCP

Modeling" Kurose, J. and Towsley, D. Firoiu, V. Padhye, J. [1]

(4) ) ,P,TF(r]E[T]E[T]E[T]E[T]E[T losspropprimarydelaycalosssstcp

router. IP sISP' the and

host end Up-Dial the between delay npropagatio trip-round :

switch, each at incurred delay processing-call the :

path, circuit Up-Dial the on switches of number the :

queue, M/D/1 an with processor call the on load traffic the :

queue, M/D/1 an with link signaling the on load traffic the :

rate, link signaling the :

setup, call in used messages signaling of size cumulative the :

(5)

dialupprop

sp

sp

sig

s

sig

dialupprop

sp

spsp

sig

sig

s

sigsetup

T

T

k

r

m

TkTkr

mTE

))1(2

1()1())1(2

1()(

16

Application II: Analytical Basis for the Routing Decision -Delay Analysis

Tprop = 0.1ms Tprop = 50ms

MbpsrBmkMbpsrr ssigspsigprimaryrescue 10100207.0100 , , , ,

17


ms.TMbpsrr propprimaryrescue 10100 and when sizes file Crossover

For example:

Crossover file size=180KB

Pb=0.3 + Ploss=0.01

18

...

...N N

maccess

mcore

maccess

maccess

maccess

...

Local trafficLong distance traffic

),( coreb

core Pv

),( accessb

access Pv

Application II: Analytical Basis for the Routing Decision - Utilization Analysis

ion.approximat load reduced

point-fixed the using by calculated is

nutilizatio circuit aggregate : 2)

rate circuit the :

size file crossover the:

ondistributi Pareto of parameter scale the :

ondistributi Pareto of parameter shape the :

ondistributi Pareto with size file average fractional the][

where ,

nutilizatio circuit-per : 1)

nutilizatio network Total

),u(u

u

r

χ

k

αα

αχχX|XE

r

XXETE

TETE

TEu

u

uuu

corea

accessa

a

c

ctransfer

transfersetup

transferc

c

ca

1000

06.1

:1

]|[][

][][

][

Symmetric three-link network model

19

Application II: Analytical Basis for the Routing Decision - Utilization Analysis

accesscoredistlongprop

localprop mmmsTmsTfN 10,50,1.0,8.0,100 and

Access link utilization uaccess Core link utilization ucore

93%

84%

20

Analytical Basis for the Routing Decision

In low propagation-delay environments Delay-based decision Crossover file size depends upon the link rates

and the loading conditions on the two paths In high propagation-delay environments

Utilization-based decision A lower bound is needed for crossover file size

21

Implementation of Application II End-host RESCUE software

A high-speed transport protocol module for end-to-end file-transfer applications,

A routing decision module, A signaling module.

Application

Signaling

TCP NIC I

NIC II

High speedtransportprotocol

RESCUE software

Routing decision

Database

Primary TCP/IP path

End-to-end RESCUE circuit

22

High-speed Transport Protocol: Design Rationale

Flow control: rate-based scheme to achieve high circuit utilization.

Implementation is not trivial. Error control: selective-Automatic-Repeat-reQuest

(selective-ARQ) scheme to achieve a high efficiency. Negative Acknowledgements (NAK) because of the

guaranteed in-sequence delivery of data blocks on dedicated circuits.

Positive Acknowledgements (ACK) are still needed to update sender’s retransmission buffers.

Dual communication paths Use primary TCP/IP path to transport reverse-path control

messages. Our transport solution: Fixed Rate Transport

Protocol (FRTP).

23

High-speed Transport Protocol: FRTP Specification

The model of FRTP connections

Control process

Data transferprocess

Control process

Data transferprocessData channel over

RESCUE circuit

Control channel overprimary TCP/IP path

The sender The receiver

24

High-speed Transport Protocol: An Implementation of FRTP protocol

FRTP is implemented as an application-level process using a combination of UDP and TCP.

FRTP receiver

Initiation

Establish TCPcontrol channel

Listening

Establish TCPcontrol channel TCP channel

FRTP sender

FRTP parameterexchange

Copy one block ofdata into

retransmission buffer

TCP channel

Initiation

FRTP parameterexchange

* Check andprocess feedbackfrom the receiver

The loss list isempty?

Pick up a lostpacket

Encapsulate a newDATA packet

Wait one inter-packet time

** Send feedbackto the sender if

necessary

Transmit aDATA packet

ReceiveDATA packet

If an errordetected?

Update the loss list and the nextexpected sequence number

Send ERR packetto the sender

Move one block ofdata out of

resequencing bufferTCP channel

UDP channel

Yes

No

Network-IO thread

Disk-IO thread

Network-IO thread

Disk-IO thread

Retransmission buffer

The loss list

Resequencing buffer

The loss list

No

Yes

25


Experimental environment: Connections: Two Dell Precision 650 workstations

connected via a Dell PowerConnect Gigabit Ethernet switch.

Hardware configurations: A 2.4-GHz Intel CPU connected to a 533-MHz front-

side bus (34Gbps CPU bandwidth), An E7505 chipset with 512MB of DDR 266MHz

memory (17Gbps memory bandwidth), An 80GB ATA/100 7200 RPM EIDE disk drive with 2MB

cache (400Mbps average access rate measured by Bonnie [66]), and,

A 64bit/100MHz PCIx bus for the GbE NIC (6.4Gbps network bandwidth).

The operating systems: RedHat Linux 9 with version 2.4.20-30.9 kernel.

26


Experimental results with default settings 256KB UDP buffer size, 1500Bytes DATA packet size, 40MB

FRTP buffer size, and 8MB block size for disk I/O operations.

FRTP throughput FRTP packet-loss rate

27


Impact of UDP buffer size 500Mbps sending rate, 1500Bytes DATA packet size, 40MB FRTP

buffer size, and 8MB block size for disk I/O operations.


28


Impact of FRTP DATA packet size 500Mbps sending rate, 256K UDP buffer size, 40MB FRTP buffer

size, and 8MB block size for disk I/O operations.


29

Routing Decision Module Design

Dest IP Ploss Pb Tprop r rc

192.168.0.2 0.01 10% 30ms 100Mbps 100Mbps

... ... ... ... ... ...

192.168.0.8 0.001 10% 30ms 10Mbps 100Mbps

... ... ... ... ... ...

Table lookup

QUERY(f, dest)

File sizecomparison

Attempt circuit setupif f > fc

Use TCP/IP pathif f < fc

Crossoverfile size

...

...

27MB

600KB

Database

Run-timemodule

Pre-computationmodule

30

Signaling Module Design A RSVP-TE implementation

Sycamoreswitch

Sycamoreswitch

CiscoMSPP

CiscoMSPP

Ethernet switch

Application

Signaling

TCP NIC I

NIC II

Dellworkstation 1

RESCUE software

Routingdecision

Application

Signaling

TCPNIC I

NIC II

RESCUE software

Routingdecision

TL1messages

TL1messages

RSVP_TEmessages

Dellworkstation 2

FRTP FRTP

RSVP_TEmessages

RSVP_TEmessages

Dellworkstation

3

31

Contributions New network architecture

“Parallel-hybrid” instead of traditional “sequential-hybrid” Dedicated end-to-end high-speed connectivity between end hosts Distributed, dynamic end-to-end circuit provisioning instead of

centralized resource management. Objective: a large-scale network providing commodity services

High aggregate network utilization Commodity services: the elephant data transfer as well as small data

transfer High traffic load -> high utilization -> low cost

Call blocking mode with packet-switched back-up paths. High circuit utilization

Superfast provisioning: distributed + hardware signaling High-speed rate-based flow control

Leveraging current conditions of Ethernet and SONET Circuit-switched SONET are widely deployed in wide-area networks. Ethernet dominates local-area networks.

32

Publications from this work

Journal papers: M. Veeraraghavan and X. Zheng, “A Reconfigurable Ethernet/SONET Circuit Based

Metro Network Architecture,” IEEE JSAC on Advances in Metropolitan Optical Networks (Architectures and Control), 2004.

M. Veeraraghavan, X. Zheng, W. Feng, Hojun Lee, E. Chong, and H. Li, “Scheduling and transport for file transfers on high-speed optical circuits,” JOGC on High Performance Networking, 2004.

Conference papers: X. Zheng, M. Veeraraghavan, and H. Lee, “Using Dial-Up Optical Circuits to

Address the Access Link Bottleneck Problem,” Under revision based on reviews from Infocom 2004.

Best Student Paper Award, M. Veeraraghavan, X. Zheng, H. Lee, M. Gardner, and W. Feng, “CHEETAH: Circuit-switched High-speed End-to-End Transport ArcHitecture,” Proceeding of Opticomm 2003, Dallas, TX, Oct. 13-16, 2003.

T. Moors, M. Veeraraghavan, Z. Tao, X. Zheng, R. Badri, Experiences in automating the testing of SS7 Signaling Transfer Points, International Symposium on Software Testing and Analysis (ISSTA), July 22-24, 2002, Via di Ripetta, Rome - Italy.

Magazine paper: M. Veeraraghavan, D. Logothetis, and X. Zheng, “Using dynamic optical

networking for high-speed access,” Optical Networks Magazine, special issue on “Dynamic Optical Networking around the Corner or Light Years Away?”, vol. 4, no. 5, pp. 30-40, Sep. 2003.

Workshop papers: M. Veeraraghavan, H. Lee, and X. Zheng, “File transfers across optical circuit-

switched networks,” PFLDnet 2003, Geneva, Switzerland, Feb. 3-4, 2003.

33

Questions?

Thanks!

Enabling New Applications with Optical Circuit-Switched Networks Xuan Zheng April 27, 2004

Documents

internet access serviceend

endtoend rescue circuits

circuit rate

access linksincreasing

network limitations

canarie network

prior workin circuit

rescue circuitsapplication