This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Grid Technologies in Disaster Management & Infrastructure
Introduction• Grid is NOT a mesh in this talk!!!!• The Grid represents “Internet Scale Distributed Computing”
or the “Managed Internet”• Major commercial and research Initiative – IBM HP Oracle
Sun SGI Platform• We describe how one can use familiar system of systems
(Grid of Grids) language for critical infrastructure and emergency response (command and control) Grids
• We give infrastructure examples from flood and earthquake cases with HPC simulations linked to other Grid resources
• Discuss GIS and Collaboration Grids
e-Business e-Science and the Grid e-Business captures an emerging view of corporations as
dynamic virtual organizations linking employees, customers and stakeholders across the world.
e-Science is the similar vision for scientific research with international participation in large accelerators, satellites or distributed gene analyses.
The Grid or CyberInfrastructure integrates the best of the Web, Agents, traditional enterprise software, high performance computing and Peer-to-peer systems to provide the information technology e-infrastructure for e-moreorlessanything.
A deluge of data of unprecedented and inevitable size must be managed and understood.
People, computers, data and instruments must be linked. On demand assignment of HPC resources, experts,
computers, networks and storage resources
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
IMAGING INSTRUMENTS
COMPUTATIONALRESOURCES
LARGE-SCALE DATABASES
DATA ACQUISITION ,ANALYSIS
ADVANCEDVISUALIZATION
e-Defense and e-Crisis Grids support Command and Control and provide Global
Situational Awareness • Link commanders and frontline troops to themselves and to archival and
real-time data; link to what-if simulations
• Dynamic heterogeneous wired and wireless networks
• Security and fault tolerance essential
System of Systems; Grid of Grids• The command and information infrastructure of each ship is a Grid; each
fleet is linked together by a Grid; the President is informed by and informs the national defense Grid
Crisis Management and Response enabled by a Grid linking sensors, disaster managers, and first responders with decision support• Grid supports On-demand HPC Simulations
Define and Build DoD relevant Services – Collaboration, Sensors, GIS, Database etc.
Raw (HPC) Resources
Middleware
Database
PortalServices
SystemServices
SystemServices
SystemServices
Application Service
Libraries
UserServices
“Core”Grid
Typical Grid Architecture
Application Service
Application Service
Re-use
Re-use
ApplicationCustomization
Each service should beable to run independently on separate machines
Service OrientedArchitectures scalebetter than traditionalDistributed objects
A typical Web or Grid Service In principle, services can be in any language (Fortran .. Java ..
Perl .. Python) and the interfaces can be method calls, Java RMI Messages, CGI Web invocations, totally compiled away (inlining)
The simplest implementations involve XML messages (SOAP) and programs written in net friendly languages like Java and Python
PaymentCredit Card
WarehouseShippingcontrol
WSDL interfaces
WSDL interfaces
Security CatalogPortalService
Web Services
Web Services
Composing Systems• We are familiar with a hierarchy
Lines of Code Methods Objects Programs Packages
• Grids extend the software hierarchy to larger distributed systems and link it to a corresponding resource hierarchy
Overlayand ComposeGrids of Grids
Methods Services Functional Grids
CPUs Clusters ComputeResource Grids
MPPs
DatabasesFederatedDatabases
Sensor Sensor Nets
DataResource Grids
Information Grid
Enterprise Grid
Compute Grid
Campus Grid
R2R1
Teacher
Students
Dynamic light-weight Peer-to-peerCollaborative Training Grid
Composing anOverlay Grid
Education Grid
Inservice TeachersPreservice TeachersSchool of Education Teacher Educator
Grids
Informal Education(Museum)
Grid
Student/Parent …Community Grid
Science GridsCI Grids
Typical Science GridService such as ResearchDatabase or simulation
Transformed by Grid Filterto form suitable for education
Learning Managementor LMS Grid
Publisher Grid
Campus orEnterprise
AdministrativeGrid
Education as a Grid of Grids
DigitalLibrary
Grid
Solid Earth Research Virtual Solid Earth Research Virtual Observatory (SERVO)Observatory (SERVO)
NASA funded: JPL, UC Davis, UC Irvine, USC, Brown, IndianaWeb/Grid-services and portlet based Problem Solving Environment (PSE)Couples data with simulation, pattern recognition software, and visualizationEnable investigators to seamlessly merge multiple data sets and models, and
– Education (schools) and Training (of managers and first responders)
– Interdependency analyses between the different infrastructures
• Note need to link to “Cell Phone Grid” for first responders with services that reduce image sizes etc.
What are we trying to do• Center for Disaster and Critical Infrastructure Informatics
– IUPUI, Indiana, Purdue, openGIS Consortium project to build a generic Crisis/CI Grid infrastructure with a FloodGrid as initial prototype
• SERVOGrid (Solid Earth Research Virtual Observatory) building Grid to support Earthquake Forecasting
• GlobalMMCS is an open-source Audio-Video conferencing and collaboration system built totally on Web Services and spanning Access Grid, H323, SIP and WebCAM’s.
• NaradaBrokering is a Grid message system supporting reliable message delivery, software multicast, firewall friendliness, security. (Grid as managed web)
• Hope to partner with NISAC (National Infrastructure Simulation and Analysis Center) on core CIGrids
CI/Crisis Grid of Grids Services in a CI or Crisis Grid fall into three classes 1) Those that special to Critical Infrastructure or
Crises such as biological sensor nets or evacuation planning service
2) Those that are important but can be taken from other Grids such as GIS or collaboration and security
3) Those that come from other Grids and are refactored for crises• The simulation is reduced in size• The public health database interface is simplified
GridResource
Filter CI/Crisis Grid View
of generic Grid Resource
CI/Crisis Grid
Gas Servicesand Filters
Security Workflow MessagingNotification
Metadata Data Access/StorageRegistry
Physical Network
Compute
Sensors Visualization
Collaboration
openGISServices
Flood Servicesand Filters
Flood CIGrid Gas CIGrid… Electricity CIGrid …
Critical Infrastructure CIGrids
openGIS Grid Semantics• Note GIS (Geographical Information System) Grid at heart of all
these Grids• Geography Markup Language (GML) is an XML encoding for the
specification of the geometry and properties of geographic features. GML utilizes the OpenGIS Abstract Specification geometry model which has been harmonized with the ISO geospatial geometry model. – We are building CI specific ontologies in terms of GML to define
• Styled Layer Descriptor (SLD) specifies the format of a map-styling language for portraying the output of Web Map Servers, Web Feature Servers and Web Coverage Servers etc. SLD will enable different communities in the Emergency Response area to develop a set of customized portrayal rules that best fit their mission requirements.– This becomes the specification of portals to different composite Grids
• Sensor Markup Language (SensorML) defines the information model for discovering, querying and controlling Web-resident sensors.
• Observations & Measurements (O&M) defines the information model for observations that are returned from the CrisisGrid sensors.
GIS Grid Services I• Web Feature Service (WFS) supports the query and discovery of
geographic features delivering GML representations of simple geospatial features in response to queries from HTTP clients. WFS can access geographic features including critical infrastructure features, incident locations, and flood-related geographic features including inundation areas, watershed boundaries, and demographic feature.
• Web Coverage Service (WCS) supports the query and discovery of digital geospatial information such as digital elevation models, imagery, orthophotography, weather coverages (such as predicted rainfall, air pressure, wind speed and direction), and any other space-varying flood-related phenomena.
• Web Map Service (WMS) uses a SLD portrayal to generate "pictures" of georeferenced feature or coverage data. WMS will provide a means to portray geographic information independent of the underlying data model (WFS or WCS).
• Coverage Portrayal Service (CPS) defines a standard interface for producing visual pictures from coverage data typically accessed via WCS with a SLD portrayal.
GIS Grid Services II• Web Terrain Service (WTS) augments WMS with advanced
visualization including 3D terrains. • Catalog Service - Web Profile (CS-W) is a catalog service
that will be built on a general Grid metadata service• Sensor Collection Service (SCS) fetches observations from
a sensor or group of sensors and will be integrated with research on Grid sensor services
• Sensor Planning Service (SPS) assists in 'collection feasibility plans' and to process collection requests for a sensor or group of sensors.
• Web Notification Service (WNS) will be replaced by standard Grid notification service
Grid Portals Portals need to provide services for security,
customization, layout, rendering Jetspeed and GridSphere are two well known portals
• Commercial systems like IBM WebSphere similar These Portals are very important as they encourage
“component” model for user interfaces and so this fits service model so every service can be packaged with its (document fragment) user interface
Portlets are good as they support they mixing and matching of services for a particular composed/overlay Grid to allow this to be easily reflected in user interface• New JSR168 portlet Java Standard
NSF Middleware Initiative NMI funding collection of open source Grid portlets
OGCEOGCEConsortium
The OGCE Computing Grid Portal
• Provides Portlets for– Management of user proxy
certificates– Remote file Management via
Grid FTP– News/Message systems
• for collaborations– Grid Event/Logging service– Access to OGSA services – Access to directory services– Specialized Application Factory
access• Distributed Workflow
– Access to Metadata Index tools• User searchable index
– Real Time Collaboration• Audio/Video Conferencing
Download from open source http://www.ogce.org
OGCEOGCEConsortium
Example Capability: File Management
• Grid FTP portlet– Allow User to manage remote file spaces– Uses stored proxy for
authentication– Upload and download files– Third party file transfer
• Request that GridFTP server A send a file to GridFTP server B
• Does not involve traffic through portal server
Portal Server
User
GridFTPServer A GridFTP
Server B
GridFTPService
1 of many Portlets
Jetspeed
Watershed Entry Page: Select a County
FloodGridPortal
Watershed Boundaries in Black
Open source GIS MapServer presented as a simple portlet
Data Download Portal
This is where you can download ArcView compatible data files for your watershed. Some of this gets hooked to the runoff model in our connected crisis grid model.
Flood Grid
DataArchives
DataArchives
RunoffModel
RunoffModel
FlowModel
FlowModel
FlowModel
GIS Grid Services Link Distributed
Data and Applications
SOAP MessagesAnd Events
DataArchives
DataArchives
RunoffModel
RunoffModel
FlowModel
FlowModel
FlowModel
GIS Grid Services Link Distributed
Data and Applications
SOAP MessagesAnd Events
Parallel Simulations
Sensors
GIS Data Archives provide terrain, elevation, and land use data
Grid WorkflowLinks Data, Sensors, Model
SERVOGrid HPC ApplicationsSERVOGrid HPC Applications Codes range from simple “rough estimate” codes to parallel, Codes range from simple “rough estimate” codes to parallel,
high performance applications.high performance applications.• DislocDisloc: handles multiple arbitrarily dipping dislocations (faults) in an : handles multiple arbitrarily dipping dislocations (faults) in an
elastic half-space.elastic half-space.• SimplexSimplex: inverts surface geodetic displacements for fault : inverts surface geodetic displacements for fault
parameters using simulated annealing downhill residual parameters using simulated annealing downhill residual minimization. minimization.
• GeoFESTGeoFEST: Three-dimensional viscoelastic finite element model for : Three-dimensional viscoelastic finite element model for calculating nodal displacements and tractions. Allows for realistic calculating nodal displacements and tractions. Allows for realistic fault geometry and characteristics, material properties, and body fault geometry and characteristics, material properties, and body forces. forces.
• VirtualVirtual CaliforniaCalifornia: Program to simulate interactions between vertical : Program to simulate interactions between vertical strike-slip faults using an elastic layer over a viscoelastic half-space strike-slip faults using an elastic layer over a viscoelastic half-space
• RDAHMMRDAHMM: Time series analysis program based on Hidden Markov : Time series analysis program based on Hidden Markov Modeling. Produces feature vectors and probabilities for Modeling. Produces feature vectors and probabilities for transitioning from one class to another. transitioning from one class to another.
• PARKPARK: Boundary element program to calculate fault slip velocity : Boundary element program to calculate fault slip velocity history based on fault frictional properties; a model for unstable slip history based on fault frictional properties; a model for unstable slip on a single earthquake fault.on a single earthquake fault.