Automatic Dynamic Run-time Optical Network Reservations

1

Automatic Dynamic Run-time Optical Network Reservations

John R. Lange

Ananth I. Sundararaj and Peter A. DindaPrescience Lab

Department of Computer Science

Northwestern University

http://plab.cs.northwestern.edu

2

Introduction

• Recently the Grid community has begun turning the network into a standard grid service– Network and compute resources are still very different– Requires in depth understanding of program behavior

or targeted application development

• VRESERVE– Enable unmodified applications to efficiently utilize

reservation based networks without user intervention

3

Overview

• Reservations and Circuit Switching• Optical Networking

– OMNInet and ODIN

• Virtual Machines– Virtuoso

• Overlay Networking– VNET and VTTIF

• Putting it all together– VRESERVE

• Performance Evaluation

4

Reservable Networks

• Generic Reservation API– CreatePath(<srcIP>, <dstIP>, <bw>, <lat>);– TeardownPath(<srcIP>, <dstIP>);

• Somebody or something has to make the API calls

• Circuit switching and network reservations are inherently linked– Establishing a circuit is similar to reserving the

network elements along a path

5

Reservation Issues

• Parameter Values– How does someone determine the required

bandwidth and latency?– Are the endpoints always evident?

• Highly parallel applications in a grid environment• Migratable Virtual Machines

• Time– What is the flow duration?– What is the traffic behavior pattern?

• Applications often have cyclical behaviors (compute and send)

6

Reservation Problems

• Complexity:– Someone has to call the reservation API

• Developer must incorporate API calls into application• User must reserve network on application’s behalf

• Efficiency:– Efficient reservations require understanding

application’s behavior and topology– Networks usually reserved in spatial and temporal

blocks• Networks reserved to connect all hosts• Networks reserved for entire execution time

7

Optical Networks

• Recent advances in switching technology have brought renewed interest to the area

• Offer provisioning of dynamic lightpaths– One of the latest incarnations of circuit switched

networking– Emergence of network reservation grid services– Generating a large deal of interest in the high

performance computing and grid communities• NLR, Canarie, Netherlight• OptIPuter

8

OMNInet and ODIN• OMNInet

– Experimental dynamically configurable optical network– Provisionable wavelengths (lambdas)– All-to-All topology– Connects research centers in Chicago and Northern Illinois

• ODIN– Reservation system for OMNInet

• Developed by iCAIR – International Center for Advanced Internet Research– Mambretti, J., Weinberger, J., Chen, J., Bacon, E., Yeh, F.,

Lillethun, D., Grossman, B., Gu, Y., and Mazzuco, M. The photonic terastream: Enabling next generation applications through intelligent optical networking at iGRID2002.

9

ODIN

• Lightpath Reservation System– Implements path discovery based on user supplied

endpoints– Also beginning to look at scheduling services– Command line client interfacing with a trusted server

• Interface– oclient -c <srcIP> <dstIP> <lambda#> <flags>

• Path Creation

– oclient -t <pathID>• Path Teardown

10

10 GE

10 GE

Lake Shore

Photonic Node

S. Federal

Photonic Node

W Taylor

1 Gbps

1 Gbps

10 GE

Optera5200

10Gb/sTSPR

Photonic Node

PP

8600

10 GE

PP

8600

Optera5200

10Gb/sTSPR

Optera5200

10Gb/sTSPR

10 GE

10 GE

PP

8600

NWUEN-2

NWUEN-3

NWUEN-4

NWUEN-8 NWUEN-9

5200 OFA

5200 OFA

Optera 5200 OFA

5200 OFA

VM

VM

Network PathTaken

Potential Paths

Fiber

KM MI1 35.3 22.02 10.3 6.43 12.4 7.74 7.2 4.55 24.1 15.06 24.1 15.07 24.9 15.58 6.7 4.29 5.3 3.3

NWUEN Link

Span Length

Host

Host

Internet

Internet

http://www.dotresearch.org

11

VirtuosoVirtual Machine marketplace

• Collection of remotely distributed VMs that appear to reside on the same LAN

• Provides many opportunities for optimization– Adaptive overlay networks, VM migration, resource scheduling,

etc…

– A. Sundararaj, A. Gupta, and P. Dinda, Increasing Application Performance In Virtual Environments Through Run-time Inference and Adaptation

• HPDC 2005– A. Sundararaj, M. Sanghi, J. Lange, P. Dinda, An Optimization Problem in

Adaptive Virtual Environments • (MAMA 2005), To Appear

• http://virtuoso.cs.northwestern.edu

12

VMM

VADAPT

VRESERVE VNET Overlay

Reservation Service(ODIN)

VNETVTTIF

Application

Reservable Network(OMNInet)

Internet

Data

Control

13

VNET Virtual Networking for Virtual Machines

• Allows VMs to behave as if they were on the same LAN– All remote VMs tunnel traffic to a central proxy

• Default star topology

• A. Sundararaj, P. Dinda, Towards Virtual Networks for Virtual Machine Grid Computing – USENIX VM 2004

14

VMM

VADAPT

VRESERVE VNET Overlay

Reservation Service(ODIN)

VNETVTTIF

Application

Reservable Network(OMNInet)

Internet

Data

Control

15

VTTIFApplication Topology Inference

• Extracts network topologies from application behavior

• Generates a global traffic matrix defining the global application network topology

• A. Gupta, P. Dinda, Inferring the Topology and Traffic Load of Parallel Programs Running In a Virtual Machine Environment – Workshop on Job Scheduling Policies for Parallel Processing, 2004

16

VADAPTOverlay Network Adaptation

• Modify virtual network to match actual network topology– Creates overlay links between communicating VMs

• Based on global reduction of VTTIF matrices

• A. Sundararaj, A. Gupta, P. Dinda, Dynamic Topology Adaptation In Virtual Networks of Virtual Machines – LCR 2004

• A. Sundararaj, A. Gupta, and P. Dinda, Increasing Application Performance In Virtual Environments Through Run-time Inference and Adaptation – HPDC 2005

17

VMM

VADAPT

VRESERVE VNET Overlay

Reservation Service(ODIN)

VNETVTTIF

Application

Reservable Network(OMNInet)

Internet

Data

Control

18

VRESERVENetwork Reservations for Overlay Networks

• Extension to VADAPT– Allows true adaptation of the network

• Components– Reservation API interface

• ODIN CLI – Routing mechanism

• Address mapping service– Path Discovery

• Depends on Reservation System functionality

• Allows for scheduled and delayed reservations– Greatly increases usability for scheduler based

reservation systems

19

Optical Overlays

• VRESERVE routing is accomplished with VNET overlay links

• Issues– Reservation based resources usually aimed at high

performance– Application unaware of changing network conditions

• TCP performance is typically poor in high performance networks

• Benefits– Application unaware of changing network conditions– Routing is much easier at the overlay level

• Network is capable of reacting to global state changes

20

10 GE

10 GE

Lake Shore

Photonic Node

S. Federal

Photonic Node

W Taylor

1 Gbps

1 Gbps

10 GE

Optera5200

10Gb/sTSPR

Photonic Node

PP

8600

10 GE

PP

8600

Optera5200

10Gb/sTSPR

Optera5200

10Gb/sTSPR

10 GE

10 GE

PP

8600

NWUEN-2

NWUEN-3

NWUEN-4

NWUEN-8 NWUEN-9

5200 OFA

5200 OFA

Optera 5200 OFA

5200 OFA

VM

VM

Fiber (MWUEN-4) length = 5 miles

Host

HostInternet

Internet

VNET

VTTIF + VADAPT

http://www.dotresearch.org

VRESERVE ODIN

21

Evaluation

• This is an existence proof – Possible to automatically reserve network resources

on unmodified application’s behalf• Caveats and Disclaimers

– Scalability is unknown– Final performance measurements are also unknown

• Infrastructure issues prevented complete performance evaluations– Only one functional optical link– Network dismantled during experiments

22

Optical Network Performance

Latency (ms)

Optical .321

Internet .498

23

Initial VNET performance (optical Network)

24

Performance Improvement(Synthetic BSP benchmark)

Demonstrable improvement by using automatic reservations

25

Making VNET faster

• Overlay links switched to UDP– Reliable transport implemented in VM TCP stack– 2x improvement

• Lookup table caching– 3x improvement

• Future:– Memory mappings for packet operations– In kernel forwarding– Specialized VM device drivers

26

VNET Performance (Round 2)

• No results on Optical Network– Infrastructure unavailable

• Experiments conducted between cluster nodes w/ gigabit Ethernet

• VNET performance only– Reservation and adaptation systems unused

• Overheads insignificant

27

VNET Performance(Gigabit switch)

28

Future Work

• Where does this fit into the overall optimization problem?

• How do we define the overall optimization problem?

• Extension to other reservation systems, e.g. GARA

29

Conclusion

• Networks are not generic grid services

• Middleware is required to make reservation networks usable

• VRESERVE– Enable unmodified applications to effectively

utilize reservation based networks without user intervention

30

• Prescience Lab– http://plab.cs.northwestern.edu

• Virtuoso– http://virtuoso.cs.northwestern.edu

• DOT (Distributed Optical Testbed)– http://www.dotresearch.org

31

Optical Network Performance

Throughput (MB/sec)

Optical TTCP 79.48Optical SCP 11.5Internet TTCP 11.2Internet SCP 10.4

Latency (ms)

Optical Network .321

Internet .498

Throughput