The OptIPuter— Lambda Coupled Distributed Computing, Peer-to-Peer Storage, and Volume Visualization Dr. Larry Smarr, Director, California Institute for Telecommunications and Information Technology Professor, Computer Science and Engineering, Jacobs School of Engineering, UCSD
20
Embed
Dr. Larry Smarr, Director, California Institute for Telecommunications and Information Technology
The OptIPuter— Lambda Coupled Distributed Computing, Peer-to-Peer Storage, and Volume Visualization. Dr. Larry Smarr, Director, California Institute for Telecommunications and Information Technology Professor, Computer Science and Engineering, Jacobs School of Engineering, UCSD. - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
The OptIPuter—Lambda Coupled Distributed Computing,
Peer-to-Peer Storage, and Volume Visualization
Dr. Larry Smarr,
Director, California Institute for Telecommunications and Information Technology
Professor, Computer Science and Engineering,
Jacobs School of Engineering, UCSD
Two New Cal-(IT)2 Buildings Approved by Legislature This Week!
• Will Create New Laboratory Facilities– Clean Rooms for Nanotech and BioMEMS– Computer Arts Virtual Reality– Wireless and Optical Networking– Interdisciplinary Teams
Bioengineering
UC San Diego
UC Irvine
Future Systems Will “Waste” Bandwidth and Storage, While Conserving Computing
Scientific American, January 2001
S-Curves of Exponential Technology Growth
0%
100%
Research
Experimental/Early Adopters
Production/Mass Market
Time
Technology S-Curve
Gigabit Testbeds
Connections Program
Internet2 Abilene
DWDM
Experimental Networks
Lambda Grids
~1990s 2000 2010
Networking Technology S-Curves
Tec
hn
olog
y P
enet
rati
on
CONTROL
PLANE
Clusters
DynamicallyAllocatedLightpaths
Switch Fabrics
PhysicalMonitoring
Apps Middleware
A LambdaGrid Will Be the Backbone for an e-Science Network
• Metro Area Laboratories Springing Up Worldwide
• Developing GigE and 10GigE Applications and Services
• Testing Optical Switches
• Metro Optical Testbeds-the next GigaPOP?
Source: Joe Mambretti, NU
NSF Defines Three Classes of Networks Beyond the Commodity Internet
• Production Networks (e.g. Internet2) – High-Performance Networks– Reaches All US Researchers– 24 / 7 Reliable
• Experimental Networks– Trials of Cutting-Edge High-Performance Networks – Deliver Advanced Application Needs Unsupported by Production Networks– Robust Enough to Support Application-Dictated Development:
– Software Application Toolkits, – Middleware, – Computing and Networking
• Research Networks– Smaller-Scale Network Prototypes – Enable Basic Scientific and Engineering Network Research– Testing of Component Technologies, Protocols, Network Architectures– Not Expected to Be Persistent– Not Expected to Support Production Applications
www.evl.uic.edu/activity/NSF/index.html
Local and Regional Lambda Experimental Networks Are Achievable and Practical
• Several GigaPOPs Are Building Multi-Lambda Metropolitan Experimental Networks by Lighting up Their Own Dark Fiber – With Hundreds of Lambdas by 2010
• LambdaGrid Software to Dynamically Couple Applications to Metro and Resources Is Possible
• Metro Experimental Networks Will Be Sensitized to: – End Application Needs – Local Connectivity Issues – Current National Production Networks Have Not Begun to Satisfy
• Substantial State and Local Funds Can Be Heavily Leveraged by an NSF Experimental Networks Program – Cross-country Inter-Connection– Persistent Support of Emerging Experimental Networks– First NSF Workshop UIC December 2001– Second NSF Workshop UCI May 2002– Expected NSF RFP by Fall 2002
Cal-(IT)2 is Developing a Metro-ScaleExperimental Optical Network
• Driven by Data-Intensive Applications– Real Time Seismic – Emergency Response– Medical Imaging
• Linked UCSD and SDSU– Dedication March 4, 2002
Linking Control Rooms
Cox, Panoram,SAIC, SGI, IBM,
TeraBurst NetworksSD Telecom Council
UCSD SDSU44 Miles of Cox Fiber
Beginning of Cal-(IT)2 Metro Optical Networking Laboratory
TeraBurst
TeraBurst
Next Step– California Must Have a State-Wide
Experimental Optical Network• The Institutes are Creating a Joint Plan
– Led by Cal-(IT)2 & CITRIS
– Involving QB3 and CNSI
• Leveraging Today’s CENIC Investment– Provides California Internet Connectivity
– K-12 and Universities
• Necessary for Data-Intensive Science– Widely Available to Many Disciplines
• California is Not the Leader Today!– Illinois, Indiana, and Canada Already There
CENIC and CISI May Create a Dark FiberExperimental and Research Network
Figure 2: SoCal Optical Research and Experimental Network
The SoCal Component
Some Scientific Applications Require Experimental Optical Networks
• Challenges– Large Data Challenges in Neuro and Earth Sciences– Each Data Object is 3D and Gigabytes– Data are Generated and Stored in Distributed Archives– Research is Carried Out on Federated Repository
• Requirements– Computing Requirements PC Clusters– Communications Dedicated Lambdas– Data Large Peer-to-Peer Lambda Attached Storage – Visualization Collaborative Volume Algorithms
• Response– Cal-(IT)2 OptIPuter Research Project
NIH is Funding a National-Scale Grid Which is an OptIPuter Application Driver
National Partnership for Advanced Computational Infrastructure
Part of the UCSD CRBS Center for Research on Biological Structure
Biomedical Informatics Research Network
(BIRN)NIH Plans to Expand
to Other Organs and Many Laboratories
The OptIPuter is an Experimental Network Research Project
• Multiple Lambdas Linking Clusters and Storage– Integration with InfiniBand PC Clusters– Rethink TCP/IP Protocols– Peer to Peer Storage– LambdaGrid Software Stack– Interactive Collaborative Volume Visualization
• NSF Large Information Technology Research Proposal– Larry Smarr, PI
– UCSD and UIC Lead Campuses– USC, UCI, SDSU, NW Partnering Campuses– Industrial Partners: IBM, Telcordia/SAIC, Chiaro
Networks, CENIC– San Diego Telecom Council Letter of Support
– Submitted April 4, 2002– Seeking $15M over 5 Years
The OptIPuter Research Team and ItsRegional, National, Int’l Extensions
Vancouver
Seattle
Portland
San Francisco
Los Angeles
San Diego(SDSC)
NCSA
SURFnet CERNCA*net4
AsiaPacific
AsiaPacific
AMPATH
PSC
Atlanta
CA*net4
Source: Tom DeFanti and Maxine Brown, UIC
NYC
TeraGrid DTFnet
CENIC
Pacific LightRail
Chicago
UICNU
USC
UCSD, SDSUUCI
OMNInet Optical Switching TrialNow Underway in Chicago Metro Area
• A Four-Site Network in Chicago -- The First 10GE Service Trial!
• A Test Bed for All-Optical Switching and Advanced High-speed Services
• Partners: SBC, Nortel, iCAIR at Northwestern, UIC, CANARIE, ANL
2x10GE
Northwestern Univ.
OpticalSwitchingPlatform
Passport8600
ApplicationCluster
ApplicationCluster
OpticalSwitchingPlatform
Passport8600
2x10GE
StarLight
OPTera Metro5200
ApplicationCluster
OpticalSwitchingPlatform
Passport8600
2x10GE8x1GE
Univ. Illinois at Chicago
CA*net3/4--Chicago
OpticalSwitchingPlatform
Passport8600
ApplicationCluster
2x10GE8x1GE
8x1GE
8x1GE
Planned Chicago Metro Electronic Switching OptIPuter Laboratory
Int’l GE, 10GE
Nat’l GE, 10GE
Metro GE, 10GE
16x1 GE 16x10 GE
16-Processor McKinley at University of Illinois at Chicago
• Distribute Storage While Optimizing Storewidth:– Distribute Massive Pools of Physical RAM (Network Memory)– Develop Visual TeraMining Techniques to Mine Petabytes of Data – Enable Ultrafast Image Rendering – Create for Optical Storage Area Networks (OSANs)
• Enhance Security Mechanisms:– End-to-End Integrity Check of Data Streams– Access Multiple Locations With Trusted Authentication Mechanisms– Use LambdaGrid Middleware for Authentication, Authorization,
Validation, Encryption and Forensic Analysis of Multiple Systems and Administrative Domains