System Software OptIPuter: System Software Research Nut Taesombut Computer Science and Engineering, University of California, San Diego January 2007 OptIPuter.

System Software

OptIPuter: System Software Research

Nut Taesombut Computer Science and Engineering, University of California, San Diego

January 2007OptIPuter All-Hands Meeting

System Software

OptIPuter System Software Architecture

Distributed Applications/ Web Services

Telescience

GTP XCP UDT

LambdaStreamCEP RBUDP

Vol-a-Tile

SAGE JuxtaView

Visualization

DVC ConfigurationDVC API

DVC Runtime Library

Data Services

LambdaRAM

Globus

PIN/PDC

DVC Services

DVC Core Services

DVC Job Scheduling

DVCCommunication

Resource Identify/Acquire

NamespaceManagement

Security Management

High SpeedCommunication

Storage Services

GRAM GSI RobuStore

Photonic

InfrastructureHigh-Speed

Transport Protocols

Optical Signaling,

Management

Distributed Virtual Computer (DVC) Middleware

VisualizationApplications

System Software

System Software/Middleware Progress

• Significant Progress in Key Areas!• Integration and Demonstration of Capabilities

– Integrated system software release in March 2006– Persistent visualization pipeline between NCSA and SIO (In Progress)

• Distributed Virtual Computer (DVC)– Simulation study of service models for configurable networks – Simulation study of efficient resource selection algorithms (In Progress)– Simulation study of network information models for configurable networks

• Advanced Transport Protocols– GTP: simulation and analytical study, comparison with other protocols– CEP: investigate block scheduling problems, new dynamic algorithms– LambdaStream: multi-stream management optimized for cluster-to-cluster

streaming

• Performance Modeling– Analysis of visualization applications

• Real-Time– Redesign RT-DVC support middleware, application development experiments

• Security– Study DoS attack, defense against spoofing, group key management

System Software

Cross Team Integration and Demonstration

• OptIPuter Gold Standard System Software Release v.1 [March 2006]– DVC middleware, network protocols (CEP-1.2, UDT-2.2, GTP-0.96) and optical

control planes (PIN-3.0/PDC-2.0)

– Document DVC User Manual and APIs Specification

• Persistent Visualization Pipeline between SIO and NCSA (In Progress)– LOOKING Project: Oceanographic data simulation, visualization and analysis

– Model Simulation (SIO/UCSD) -> Storage (JSOE/UCSD) -> Rendering (NCSA) -> Visualization (SIO/UCSD)

– 5-layer: Application, SAGE, DVC middleware, dynamic network config., high-speed transport (GTP)

Photos: Atul Nayak – IGPP/SIO

System Software

Distributed Virtual Computer (DVC) (Nut Taesombut – UCSD)

• Evaluating Services Models for Configurable Optical Networks [IEEECommMag06]

• Plan for 2007– Evaluating resource selection strategies for LambdaGrids (publication)– Evaluating the impact of network Information models on applications

and service providers (publication)

- DVC achieves highest performance (both lower and deterministic runtime) - DVC produces lowest network configuration cost- INET and AFTP achieve higher resource efficiency and utilization

(INET)

(DVC)

(AFTP)

System Software

OptIPuter High-Performance Transport Protocols

• Bridge the Gap between High Speed Link Technologies and Growing Demands of Advanced Applications– TCP has well-documented performance problems on long-haul networks

• Pursue complementary avenues of investigation – Efficient congestion/flow management, fairness among flows

– High-speed group communication (multipoint-to-point, multipoint-to-multipoint)

Private Lambda Shared, Routed

Network Connection

RBUDP/

-stream

GTP SABUL/

UDT

XCP

Unicast ManagedGroup

EnhancedRouters

StandardRouters

CEP

System Software

Transport Protocol Progress

• Group Transport Protocol (GTP): Ryan Wu – UCSD

– Fair bandwidth sharing and allocation at both sources and sinks (technical report)

– Finalize convergence proofs in asynchronous cases (technical report)– Evaluate and compare GTP with other transport protocols

• Composite Endpoint Protocol (CEP): Eric Weigle – UCSD

– "Peer-to-Peer Error Recovery for Hybrid Satellite-Terrestrial Networks”, IEEE P2P2006

– Enhanced hardware and system support– Investigate block scheduling and application integration problems– Explore new algorithms for one-to-many transfers and on dynamic networks

• LambdaStream: Venkat Vishwanath – UIC – Performance analysis and improvements (using enhanced MAGNET to monitor

10Gbps streams at line rate)– Implement an efficient multi-stream LambdaStream optimized for cluster-to-

cluster streaming

System Software

Storage: RobuStore (Huaxia Xia – UCSD)

• RobuSTore: Use Erasure Codes and Speculative Access to Deliver Robust and High Storage Performance

• Explored “Read” Performance with Various Data Redundancy– Bandwidth of RobuSTore increases rapidly with data redundancy

– Reach the maximum at 5x storage (over 15x performance compared to RRAID-0)

– Get most of the bandwidth at 3x storage

– RobuSTore has the lowest access variation with >2x redundancy

• Explored “Write” Performance with Various Data Redundancy – RobuSTore outperforms other systems:

– E.g. When data redundancy is 300%

– Bandwidth: 25x of RRAID-S/A

– Variation of Latency: 1/13 of RRAID-S/A

System Software

Performance Modeling/Analysis (Valerie Taylor, Xingfu Wu – TAMU)

• Performance Analysis of Visualization Applications– Update TAMU OptIPuter nodes with SAGE2

– Setup the wide-area environment for ongoing large experiment tests across CAVEWave/NLR/LEARN

– Measured optical network performance between TAMU and UIC/SDSC/UCSD over CAVEWave/NLR/LEARN

– RRT between TAMU and UIC is 27.3 ms

– RTT between TAMU and UCSD is 105.2 ms

– Working with LEARN to reduce the RTT

• Publications and Research Activities– “Performance Analysis, Modeling and Prediction of a Parallel Multi-block Lattice

Boltzmann Application Using Prophesy System” [Cluster2006]

– Three research booth posters at SC2006

– Two research talks at SC2006 Gulf Coast Academic Supercomputing Booth

– Invited Talk, OptIPuter Project: Performance Analysis and Monitoring, Texas LEARN Annual Meeting, Nov2006

System Software

Ongoing Large Experiments (Vol-A-Tile) across CAVEWave/NLR/LEARN

Master

GigabitNetwork

VolvistfUI

Client1Volvis

Client3Volvis

Client4Volvis

Client2Volvis

Visualization cluster at TAMUVisualization cluster at TAMU

LEARNLEARN

Optistore

OptIPuter CAVEWave TestbedOptIPuter CAVEWave TestbedOptistore OptistoreOptistore

NLRNLR

System Software

Real-Time Progress(Kane Kim, Stephen Jenks - UCI)

• Prototype Implementation of Real-Time DVC Support Middleware– Significantly redesigned version (4.21) of TMOSM (Time-triggered Messages and Objects

Support Middleware)– Improved modularity, concurrency, portability and timing precision– Support a range of platforms (Linux, WinXP, WinCE, etc.)

• Programming Model– API for RT middleware enables high-level RT programming (TMO) without a new compiler– The notion of Distance-Aware (DA) TMO, an attractive building-block for RT wide-area DC

applications, was created and a study for its realization is under way

• Enhancement the Network Infrastructure of OptIPuter– GPS receivers acquired from German vendor & installed in UCI/UCSD (Calit2 bldgs)– One-way message delay/jitter between UCI and UCSD measured

– Between device drivers: Delay (1630 μsec), Jitter (20 μsec on most trials)– Between application modules: Delay (1900-1980 μsec), Jitter (100 μsec on most trials)

• Application Development Experiments (Remote Control of an electronic car)– 90 miles through (one-way) CAVEWave: UCI -> UCSD – 180 miles through (two-way) CAVEWave UCI -> UCSD -> UCI

• Publication– A paper on TMOSM/Linux to appear in Real-Time System Journal

System Software

Real-Time Application Demo

Wireless

LAN

Videocam

Joystick

Videocam

Irvine

Seoul

Capturing the scenery of the

operator controlling the remote

carPC displaying what the

remote car controller in Seoul sees

Advanced Network

Connection over

Pacific Ocean

TMO-structured Software

System Software

Security Progress(Joe Touch – ISI; Mike Goodrich – UCI)

• IETF Progress Addressing Security (ISI)– BTNS (we created) near completing goals

– TCPM anti-spoof description in last call

– TCPM proposal to fix TCP/MDS (simple authentication)

– TCPM proposal decoupling TCP port names/numbers

• Impact of DoS Attack (ISI, UCI)– “Reducing the Impact of DoS Attacks on Endpoint IP Security,” NPSec2006 (in

conjunction with ICNP2006)

– “Probabilistic Packet Marking for Large-Scale IP Traceback”, To appear in IEEE/ACM Transactions on Networking 2007

• Group Key Management for Grid Computing (UCI)– “Notarized Federated Identity Management for Web Services” 20th IFIP WG

Working Conference on Data and Application Security (DBSec)

System Software

Year 5 Plans

• Integrated System Software Release– ~June 2007 – System Software Rock Gold Roll version 2

• Planned Features & Improvements– Distributed Virtual Computer (DVC)

– Complete and stabilized prototype (software release)

– Evaluation of network information models (publication)

– Evaluation of resource selection algorithms (publication)

– High Speed Protocols

– GTP: study of interaction with other TCP traffics, optimized implementation

– CEP: Improved scalability results, decentralized dynamic block-based CEP, comprehensive testing

– Real-Time

– Middleware support for distance-aware TMOs (software release)

– Remote control of an electronic car between Konkuk Univ. (Korea) and UCI; 145ms round-trip delay (demonstration)

System Software

Year 5 Plans (cont.)

• Planned Features & Improvements (cont.)– Security (ISI/UCI)

– Fast/low-CPU DoS defense layer (software release)

– “Stronger network security using weaker means” (publication)

– “Grid security for pipelined computations” (publication)

– Performance Analysis (Prophesy)

– Test and measure the performance of different layers of OptIPuter testbeds (CAVEWave/NLR/LEARN)

– Analyze the performance of SAGE2, Vol-A-Tile, new JuxtaView and LambdaRAM on larger-scale OptIPuter testbeds (publication)