Grid Computing & Tuple Grid Computing & Tuple Space Space Presented by Presented by Nelson Chu Nelson Chu
Jan 10, 2016
Grid Computing & Tuple Space Grid Computing & Tuple Space
Presented byPresented by
Nelson ChuNelson Chu
What is Grid?What is Grid? Computing Grid is a very large distributed Computing Grid is a very large distributed
computing network, Virtual Organizations computing network, Virtual Organizations (VO)(VO)
Grid is an infrastructure to collaborate many Grid is an infrastructure to collaborate many diverse computing resources in a network to diverse computing resources in a network to accomplish a task(s).accomplish a task(s).
What is Grid?What is Grid?
Classification: Classification: Computational GridComputational Grid (eg. WestGrid) (eg. WestGrid)
Desktop GridDesktop Grid (eg. SETI@HOME) (eg. SETI@HOME)
Data GridData Grid (eg. Earth Systems Grid)(eg. Earth Systems Grid)
Global Grid Forum (GGF), Globus, etc. Global Grid Forum (GGF), Globus, etc. Open Grid Services Architecture - OGSAOpen Grid Services Architecture - OGSA Web Service Resource Framework - WSRF Web Service Resource Framework - WSRF
(OGSI)(OGSI)
ESG
Fabric: Physical resources
Resource & Connectivity Protocols: secured access
Collective Services: brokering, monitoring
User Applications/Tools
Grid ArchitectureGrid Architecture
Fabric layer: resource management Fabric layer: resource management e.g. computational resources, storage e.g. computational resources, storage resources, and network resources. resources, and network resources.
Connectivity layer: Communication and Connectivity layer: Communication and authentication authentication
Collective layer: global-naming, brokering, Collective layer: global-naming, brokering, scheduling, workload and workflow scheduling, workload and workflow management, and data replication. management, and data replication.
Application layer: user defined tools and Application layer: user defined tools and applicationapplication
OGSA Architecture
OGSA provides a service oriented architecture to simplify grid services development and to build a
transparent grid structure
Physical resources (for example: storage, network, and computer) and logical resources services
Web services (including WSRF)
OGSA architect services (such as security management, data management, workflow
management, file system management, etc.)
Grid applications
Research Opportunities: GridResearch Opportunities: Grid
SecuritySecurity Multi-institutional users and heterogeneous resources Multi-institutional users and heterogeneous resources Authentication & authorization mechanismsAuthentication & authorization mechanisms Different local name spaces, credentials, and/or Different local name spaces, credentials, and/or
accountsaccounts Trust mechanisms: domain-based, user-based, service-Trust mechanisms: domain-based, user-based, service-
based, network-based, and scheduler-basedbased, network-based, and scheduler-based
Fault ToleranceFault Tolerance Detect and notify a fault occurrenceDetect and notify a fault occurrence Handle any failureHandle any failure
Research Opportunities: GridResearch Opportunities: Grid
SchedulingScheduling Workflows in a service-based environmentWorkflows in a service-based environment Split and distribute jobs to resourcesSplit and distribute jobs to resources Time, cost, performance, completeness, fairness, Time, cost, performance, completeness, fairness,
effectiveness, and security policies. effectiveness, and security policies.
Resource discoveryResource discovery System selectionSystem selection Job submission: resources reservation and Job submission: resources reservation and
supporting operationsupporting operation Monitoring and cleaning upMonitoring and cleaning up
Current Research TrendsCurrent Research Trends
Scheduling, Security, Scheduling, Security, Fault ToleranceFault Tolerance
Replica location services Replica location services
Metadata catalog servicesMetadata catalog services
Reliable file transfer servicesReliable file transfer services
Globus Toolkit 3 / 4: Globus Toolkit 3 / 4: Replica Catalog, GridFTP, Replica Catalog, GridFTP, and (GASS) and (GASS) Global Access to Secondary Storage Global Access to Secondary Storage ..
Globus, IBM, Oracle 10g, Sun, etc.Globus, IBM, Oracle 10g, Sun, etc.
Research Opportunities: DataResearch Opportunities: Data
Data VersioningData Versioning Global name space: data characterization, Global name space: data characterization,
heterogeneous systemsheterogeneous systems Data SecurityData Security Fault tolerance: recoveryFault tolerance: recovery New challenges: inter-domains replication, New challenges: inter-domains replication,
data synchronization, replica registration, data synchronization, replica registration, querying, and meta-catalog. querying, and meta-catalog.
Grid EnvironmentGrid Environment
Resources collaborationResources collaboration
Loosely coupling Loosely coupling Time, space, destinationTime, space, destination
Communication allows significant degree of Communication allows significant degree of interoperability, synchronization, and interoperability, synchronization, and extensibilityextensibility
Simple, flexible, and powerful mechanismSimple, flexible, and powerful mechanism
Grid System Grid System
Open issues: data synchronization, fault Open issues: data synchronization, fault tolerance, persistence, concurrency, tolerance, persistence, concurrency, transparency, etc. transparency, etc.
Tuple Space paradigm Tuple Space paradigm Simple operations provide powerful inter-Simple operations provide powerful inter-
process mechanism for communication and process mechanism for communication and synchronization. synchronization.
Flexible for function expansion: ordered tuple, Flexible for function expansion: ordered tuple, leasing, … leasing, …
Tuple SpaceTuple Space
Developed in a parallel and distributed Developed in a parallel and distributed programming system, Linda, at Yale Uprogramming system, Linda, at Yale U
Basic element ~ TupleBasic element ~ Tuple
Tuple: a vector of typed values / fields / Tuple: a vector of typed values / fields / variablesvariables
Operations: Writing, Reading, Consuming &Operations: Writing, Reading, Consuming &
MatchingMatching
Tuple SpaceTuple Space
TS Vs DBSTS Vs DBS
SpacesTwo
SpacesOne
A
C
Read
Take “B” … … … wait/abort
Write
Processes
Producers
Consumers
Tuple SpaceTuple Space
Globally shared memory space Globally shared memory space (Communication buffer)(Communication buffer)
Tuples are associatively addressed by Tuples are associatively addressed by matchingmatching
Loosely couplingLoosely coupling
Mechanism for inter-process communication Mechanism for inter-process communication and synchronizationand synchronization
Simple, flexible, powerful, …Simple, flexible, powerful, …~> Grid Resource Collaboration~> Grid Resource Collaboration
Tuple Space on the GridTuple Space on the Grid
Service RegistrationService Registration Service DiscoveryService Discovery Service LookupService Lookup Job SchedulerJob Scheduler Data MgmtData Mgmt
Tuple Spaces
(Services / Tasks / Data)
Consumers
Producers / Processes
Tuple Space on the Grid Tuple Space on the Grid with Java Technologywith Java Technology
Grid Enabled
• Resource Mgmt (GRAM)
• Security Feature
• Fault Tolerant
• Data Mgmt
• Look up Service
• Discovery Mgmt
• Leasing
• Plug and Work
• Resources / Services Collaboration
Grid Job SchedulingGrid Job Scheduling
Eg) WestGrid, …Eg) WestGrid, …
Grid IScheduler/
Resources Broker
Resource
Resource
Meta-Scheduler Grid II
Centralized System
• Load balance
• Synchronization
• Fault Tolerance
• Policy
• Meta-Scheduler
Tuple Spaces Based Gird Job SchedulerTuple Spaces Based Gird Job Scheduler
Resource 1 Agent
Resource N Agent
Registrar
Job
Desc.Resources Required
Job
Desc.Resources Required
Result
SecurityCheck()
PolicyCheck()
TakeJob()
WriteResult()
Resources List
Register()
JobSpace
JobSubmit()
Registrar
ResourcesList()
ResourcesListExchange()
JobExchange()
Domain II
Users
Figure 1: Tuple Spaces Based Grid Scheduler
GetResult()
LeaseRenew ()
Fault Tolerance ManagementFault Tolerance Management Leasing Structure for Dynamic ParticipationLeasing Structure for Dynamic Participation On-Demand Service On-Demand Service Automatic Load Balance Automatic Load Balance Global Name SpaceGlobal Name Space Security model ?Security model ?
View 3
View 1
Query… …SearchView()
GetJobTicket()
QueueTicket()
GridFTP() / TransferSchedule()
SecCheck()
PolicyCheck()
TakeJob()
UpdateView()
EventNotify()
UpdateData()
… …
PriorityCheck()
New Data
Derived DataRenewL()
TicketBox
ViewSite
OpenSpace
Figure 2: Data Access Management with Tuple Spaces
Registrar RegistrarResourcesListExchange()
Domain II
Resource Agent
EventNotify()
Update()
Update()
EventNotify()
Register()
JobExchange ()
Replica
Ticket:View(s)/Query(ies)CertificateUser informationJob description… …
Gird Data Gird Data Management System Management System
with Tuple Spaceswith Tuple Spaces
ReferencesReferences
Ian Foster, Carl Kesselman, and Steven Tuecke, “The Anatomy of Ian Foster, Carl Kesselman, and Steven Tuecke, “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, Intl J. the Grid: Enabling Scalable Virtual Organizations”, Intl J. Supercomputer Applications, 2001Supercomputer Applications, 2001
Ian Foster, Carl Kesselman, Jeffrey M. Nick, and Steven Tuecke, Ian Foster, Carl Kesselman, Jeffrey M. Nick, and Steven Tuecke, “The Physiology of the Grid: An Open Grid Services Architecture for “The Physiology of the Grid: An Open Grid Services Architecture for Distributed Systems Integration”, http://www.globus.org, 2002Distributed Systems Integration”, http://www.globus.org, 2002
Gregory T. Byrd, and Praveenkumar Cheruvu, “Tuple Space Gregory T. Byrd, and Praveenkumar Cheruvu, “Tuple Space Computing on the Grid”, North Carolina State UniversityComputing on the Grid”, North Carolina State University
K.A. Hawick, H.A. James, and L.H. Pritchard, “Tuple-Space Based K.A. Hawick, H.A. James, and L.H. Pritchard, “Tuple-Space Based Middleware for Distributed Computing”, Technical Report DHPC-Middleware for Distributed Computing”, Technical Report DHPC-128, University of Wales128, University of Wales
Globus, http://www.globus.orgGlobus, http://www.globus.org
IBM, www.ibm.com/gridIBM, www.ibm.com/grid
DiscussionsDiscussions
Thank you !Thank you !