SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004 NSF Workshop on Cyberinfrastructure for Environmental Observatories Introduction to CI Topics Chaitan Baru, SDSC/NLADR Bertram Ludaescher, UC Davis/SDSC Michael Welge, NCSA/NLADR
51
Embed
NSF Workshop on Cyberinfrastructure for Environmental Observatories Introduction to CI Topics
NSF Workshop on Cyberinfrastructure for Environmental Observatories Introduction to CI Topics. Chaitan Baru, SDSC/NLADR Bertram Ludaescher, UC Davis/SDSC Michael Welge, NCSA/NLADR. Outline. A nexus of CI projects CI project “principles” CI technical focus areas/topics - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
NSF Workshop on Cyberinfrastructure for Environmental Observatories
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
CI Project Principles• Use IT state-of-the-art, and develop advanced IT where needed, to support
the “day-to-day” conduct of science (e-science)• (not just “hero” computations)• Based on a Web/Grid services-based distributed environment
The “two-tier” approach• Use best practices, including commercial tools,• while developing advanced technology in open source, and doing CS research
• An equal partnership • IT works in close conjunction with science, to create CI, i.e., the best practices, data
sharing frameworks, useful and usable capabilities and tools
• Create the “science IT infrastructure”• Online databases with advanced search engines• Robust tools and applications, etc.
• Leverage from other intersecting projects• Much commonality in the technologies, regardless of science disciplines• Constantly work towards eliminating (or, at least, minimizing) the “NIH” syndrome• And, importantly, try not to reinvent what industry already knows how to do…
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
Important Focus Areas / Topics
• Security• Authentication, access control, controls for data publication…
Antelope WSRF ExtensionsCourtesy: Tony Fountain, SDSC and LOOKING project
Object Ring
BufferField
Interface Module
ORB Operations:
Orb ImportOrb ExportProcessingArchiving
field digitizer
field digitizer
field digitizer
Databases
Antelope Executive
Module
WS-Resource
WS-Resource
WS-Resource
WS-Resource
Soap HeaderSoap Body
Proxy Cert
Request
Params
SoapRequest
SOAP/HTTPPortal Data Analyzer
ORBcommander
ORBManager
LookupService
WSRFAuthentication & Authorization
Antelope Web
Services
ServiceInvoker
Proxy RepositoryCerts,username, password, others
Services Repositoryname, definiton, others
ORBMonitor
ServicesSubscriber
Databaseoperator
Event Coordinator
OtherServices
WS-Resource
WS-Resource
WS-Resource
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
CI Organizational Issues
• How to foster development of common infrastructure (based upon science needs/input), across multiple science domains• Not just at hardware level (e.g. supercomputers, high-
speed networks) or OS and system services level• But, at the database, data integration, data mining levels
• How to deal with the continuum of activities from basic CS research to production IT systems
• NLADR – created with above issues in mind• Prototype for a CI organization
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
NLADR—National Lab for Advanced Data Research
• Joint activity between SDSC and NCSA, started October 1, 2004
• Formed based on NSF’s requirement that SDSC and NCSA collaborate on CI activities
• Collaborative R&D activity focused on advanced data technologies• Guided by real applications from science communities• …to assemble expertise and a “knowledge base” of data technologies• And, also develop a broad data architecture framework• …within which to develop, integrate, test, and benchmark data-related
technologies• …in the context of national-scale physical infrastructure
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
NLADR Services Architecture
NLADR Data Management ServicesManagement and archiving of large simulation outputs, streaming data, databases, data collections
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
Different views on State Geological Maps
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
Sedimentary Rocks: BGS Ontology
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
Sedimentary Rocks: GSC Ontology
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004formalized as domain map/ontology
Purkinje cells and Pyramidal cells have dendritesthat have higher-order branches that contain spines.Dendritic spines are ion (calcium) regulating components.Spines have ion binding proteins. Neurotransmissioninvolves ionic activity (release). Ion-binding proteinscontrol ion activity (propagation) in a cell. Ion-regulatingcomponents of cells affect ionic activity (release).
domain expert knowledge
Made usable for the system using Description Logic
Example: Domain Knowledge to “glue” SYNAPSE & NCMIR
Data
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
Ilkay Altintas SDM, ResurgenceKim Baldridge Resurgence, NMI Chad Berkley SEEK Shawn Bowers SEEKTerence Critchlow SDM Tobin Fricke ROADNetJeffrey Grethe BIRNChristopher H. Brooks Ptolemy II Zhengang Cheng SDM Dan Higgins SEEKEfrat Jaeger GEON Matt Jones SEEK Werner Krebs, EOLEdward A. Lee Ptolemy II Kai Lin GEONBertram Ludaescher SDM, SEEK, GEON, BIRN, ROADNetMark Miller EOLSteve Mock NMISteve Neuendorffer Ptolemy II Jing Tao SEEK Mladen Vouk SDM Xiaowen Xin SDM Yang Zhao Ptolemy IIBing Zhu SEEK •••
Ptolemy IIPtolemy II
www.kepler-project.orgwww.kepler-project.org
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
Scientific Workflows as a Melting Pot:Example: The Kepler SWF System
• A grass-roots project• collaboration at the level of developers
• Intra-project links• e.g. in SEEK: AMS SMS EcoGrid
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
Job Management (here: NIMROD)
• Job management infrastructure in place• Results database: under development• Goal: 1000’s of GAMESS jobs (quantum mechanics)
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
Some Recent Actor Additions
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
in KEPLER (w/ editable script)
Source: Dan Higgins, Kepler/SEEKSource: Dan Higgins, Kepler/SEEK
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
Blurring Design (ToDo) and Execution
SAN DIEGO SUPERCOMPUTER CENTER NSF Workshop on CI for Environmental Observatories, Dec 6.7, 2004
Towards Real-time Analysis Pipelines:Towards Real-time Analysis Pipelines:Combining Simulations, Models, and ObservationsCombining Simulations, Models, and Observations
A Briefing On Data Mining to the NSF Planning Meeting Discussion Group on Cyberinfrastructure For Environmental Observatories
• EQL (Event Query Language) implements a compositional semantics for event expressions.• Composite events are
“first order” events.• Monitors can monitor
monitors.
• Clock events are part of the language implementation.• Easy to write queries
with temporal constraints.
EventWorksEvent Router
Streams
Monitor 1 Monitor N
…EQL EQLNew
Events
Monitors are generated by users or programmatically.
D2K : A Framework For Building Data-Driven Apps – Persistent Stream Data Analytics Foundation
Designed for Building and Maintaining Complex Persistent and Stream Designed for Building and Maintaining Complex Persistent and Stream Data-Driven ApplicationsData-Driven Applications
http://alg.ncsa.uiuc.edu
D2K/T2K/I2K: Data, Text, and Image Analysis
http://alg.ncsa.uiuc.edu
Uses novel methods to do real-time stream data analysis.
LOOKING: Stream Data Analytics/Information Visualization scientific “dashboard”
Discovers association and correlation rules in data stream environment.
8 X 8 Processor 4 Pipe, 16 gig Memory each, Prisms coupled with Infiniban for On-demand, Interactive
U of W
NLADR Tier 1 Architecture
…. Data-Drive Science
• Collaboration• Information Gathering (experiments, simulation,
observation – calendar of upcoming activities)• Data Management
• Generation and Publishing of Data (experiments, simulation, or observation)• Persistent Data Stores (Distributed Data Management)• Stream Data Management (Event Management)
• Detection• Mining of new types of data, such as large static data