Computational Grids and Computational Economy: Nimrod/G Approach
David AbramsonDavid AbramsonRajkumar BuyyaRajkumar BuyyaJonathan GiddyJonathan Giddy
Parametric Execution of ApplicationsParametric Execution of ApplicationsCoarse-grained SPMD modelCoarse-grained SPMD model
Execute one application repeatedly for many combinations of Execute one application repeatedly for many combinations of input parametersinput parameters
Legacy applications: add iteration and distribution without Legacy applications: add iteration and distribution without modifying codemodifying code
New applications: remove iteration and distribution from designNew applications: remove iteration and distribution from design
Parametrised modeling experiments:Parametrised modeling experiments:– Require very high levels of performanceRequire very high levels of performance– GenerateGenerate
– Large amounts of work & concurrencyLarge amounts of work & concurrency– Uncoupled computationsUncoupled computations
– Tolerate - moderately high latenciesTolerate - moderately high latencies
Job 1 Job 2 Job 3
Job 4 Job 5 Job 6
Job 7 Job 8 Job 9
Job 10 Job 11 Job 12
Job 13 Job 14 Job 15
Job 16 Job 17 Job 18
Description of Parameters
Working with Small ClustersWorking with Small ClustersWorking with Small ClustersWorking with Small Clusters
Nimrod (1994 - )Nimrod (1994 - )– DSTC funded projectDSTC funded project– Designed for department level clustersDesigned for department level clusters– Proof of conceptProof of concept
Clustor (Activetools) (1997 - )Clustor (Activetools) (1997 - )– Commercial version of NimrodCommercial version of Nimrod– Re-engineeredRe-engineered
FeaturesFeatures– Workstation orientationWorkstation orientation– Access to idle workstationsAccess to idle workstations– Random allocation policyRandom allocation policy– Password securityPassword security
Nimrod (1994 - )Nimrod (1994 - )– DSTC funded projectDSTC funded project– Designed for department level clustersDesigned for department level clusters– Proof of conceptProof of concept
Clustor (Activetools) (1997 - )Clustor (Activetools) (1997 - )– Commercial version of NimrodCommercial version of Nimrod– Re-engineeredRe-engineered
FeaturesFeatures– Workstation orientationWorkstation orientation– Access to idle workstationsAccess to idle workstations– Random allocation policyRandom allocation policy– Password securityPassword security
Clustor limitationsClustor limitations
Manual resource locationManual resource location– static file of machine namesstatic file of machine names
No resource schedulingNo resource scheduling– first come first servedfirst come first served
No cost modelNo cost model– all machines cost alikeall machines cost alike
Single access mechanismSingle access mechanism
Towards Grid Computing….Towards Grid Computing….
Source: www.globus.org & updated
Nimrod/G - Nimrod over Globus/GridNimrod/G - Nimrod over Globus/GridNimrod/G - Nimrod over Globus/GridNimrod/G - Nimrod over Globus/Grid
Wide-Area Network SupportWide-Area Network Support– redesigned architectureredesigned architecture– use of high-performance networksuse of high-performance networks
Scalable SchedulingScalable Scheduling– ““guaranteed” deadlineguaranteed” deadline– use of existing schedulersuse of existing schedulers
Computational EconomyComputational Economy– ““I am willing to pay $$, can you complete the I am willing to pay $$, can you complete the
job by given deadline”job by given deadline”– trading, bidding, resource reservation...trading, bidding, resource reservation...
Wide-Area Network SupportWide-Area Network Support– redesigned architectureredesigned architecture– use of high-performance networksuse of high-performance networks
Scalable SchedulingScalable Scheduling– ““guaranteed” deadlineguaranteed” deadline– use of existing schedulersuse of existing schedulers
Computational EconomyComputational Economy– ““I am willing to pay $$, can you complete the I am willing to pay $$, can you complete the
job by given deadline”job by given deadline”– trading, bidding, resource reservation...trading, bidding, resource reservation...
Layered Architecture (Grid Components)Layered Architecture (Grid Components)
Applications
Core ServicesMetacomputing
Directory Service
GRAMGlobus
Security Interface
Heartbeat Monitor
Nexus
Gloperf
Local Services
LSF
Condor MPI
NQEEasy
TCP
SolarisIrixAIX
UDP
High-level Services and Tools
DUROC globusrunMPI Nimrod/GMPI-IO CC++
GlobusView Testbed Status
GASS
Source: www.globus.org
Nimrod/G ArchitectureNimrod/G Architecture
Grid Middleware Services
Dispatcher
Nimrod/G Client Nimrod/G ClientNimrod/G Client
Grid Directory Services
Schedule Advisor
Resource Discovery
Parametric Engine
GUSTO Test Bed
Persistent Info.
Nimrod/G InteractionsNimrod/G Interactions
MDSserver
Resource location
QueuingSystem
GRAMserver
Resource allocation
(local)
Additional services used implicitly:• GSI (authentication & authorization)• Nexus (communication)
Userprocess
File accessGASSserver
Gatekeeper node
JobWrapper
Computational node
Dispatcher
Root node
Scheduler
Prmtc..Engine
Scheduling AlgorithmScheduling Algorithm
Find a set of machines (MDS search)Find a set of machines (MDS search)
Distribute jobs from root to machinesDistribute jobs from root to machines
Establish job consumption rate for each machineEstablish job consumption rate for each machine
For each machineFor each machine
Can we meet deadline?Can we meet deadline?
If not, then return some jobs to rootIf not, then return some jobs to root
If yes, distribute more jobs to resourceIf yes, distribute more jobs to resource
If cannot meet deadline with current resourceIf cannot meet deadline with current resource
Find additional resourcesFind additional resources
Find a set of machines (MDS search)Find a set of machines (MDS search)
Distribute jobs from root to machinesDistribute jobs from root to machines
Establish job consumption rate for each machineEstablish job consumption rate for each machine
For each machineFor each machine
Can we meet deadline?Can we meet deadline?
If not, then return some jobs to rootIf not, then return some jobs to root
If yes, distribute more jobs to resourceIf yes, distribute more jobs to resource
If cannot meet deadline with current resourceIf cannot meet deadline with current resource
Find additional resourcesFind additional resources
A Nimrod/G ClientA Nimrod/G Client
CostCostDeadlineDeadline
AvailableAvailableMachinesMachines
Sample Applications of NimrodSample Applications of NimrodSample Applications of NimrodSample Applications of Nimrod
Bioinformatics: Protein ModelingBioinformatics: Protein Modeling
Sensitivity experiments on smog formationSensitivity experiments on smog formation
Parametric study of Laser detuningParametric study of Laser detuningCombinatorial Optimization: Simulated Annealing Combinatorial Optimization: Simulated Annealing
Ecological Modeling: Control Strategies for Cattle TickEcological Modeling: Control Strategies for Cattle Tick
Electronic CAD: Field Programmable Gate Arrays Electronic CAD: Field Programmable Gate Arrays
Computer Graphics: Ray TracingComputer Graphics: Ray Tracing
High Energy Physics: Searching for Rare EventsHigh Energy Physics: Searching for Rare Events
Physics: Laser-Atom Collisions Physics: Laser-Atom Collisions
VLSI Design: SPICE SimulationsVLSI Design: SPICE Simulations
Radiation Protection and Nuclear Safety
Electronic CADElectronic CADElectronic CADElectronic CAD
Graph 2 - GUSTO Usage for Ionization Chamber Study
0
10
20
30
40
50
60
70
80
0 2.5 5 7.5 10 12.5 15 17.5 20
Time
Ave
rag
eN
o P
roce
sso
rs
20 Hour deadline15 hour deadline10 hour deadline
Some early results - Some early results -
Graph 5 - GUSTO Usage for 10 Hour Deadline
0
5
10
15
20
25
30
35
0 2.5 5 7.5 10 12.5 15 17.5 20
Time
No
Pro
ce
ss
es 5 CUs
10 CUs
15 CUs
20 CUs
50 CUs
10 Cost Units
50 Cost Units
20 Cost Units
5 Cost Units
15 Cost Units
Graph 4 - GUSTO Usage for 15 Hour Deadline
0
2
4
6
8
10
12
14
16
18
20
0 2.5 5 7.5 10 12.5 15 17.5 20
Time
Ave
rag
e N
o P
roc
es
so
rs
5 CUs
10 CUs
15 CUs
20 CUs
50 CUs
5 Cost Units
50 Cost Units
15 Cost Units
10 Cost Units
Graph 3 - GUSTO Usage for 20 Hour Deadline
0
2
4
6
8
10
12
14
16
18
20
0 2.5 5 7.5 10 12.5 15 17.5 20
Time
Ave
rag
e N
o P
roce
sso
rs
5 CUs
10 CUs
15 CUs
20 CUs
50 CUs
5 Cost Units
10 Cost Units
Related WorksRelated Works
AppLeS (UC. San Diego)AppLeS (UC. San Diego)– application level scheduling & case-by-caseapplication level scheduling & case-by-case
NetSolve (UTK/ORNL)NetSolve (UTK/ORNL)– API for creating farmsAPI for creating farms
DISCWorld (U. Adelaide)DISCWorld (U. Adelaide)– remote information accessremote information access
Millennium (UC. Berkeley)Millennium (UC. Berkeley)– remote execution environment on clusters and supports computational economy remote execution environment on clusters and supports computational economy
ConclusionsConclusions Nimrod/G architecture offers a scalable model for resource management and scheduling on computational gridsNimrod/G architecture offers a scalable model for resource management and scheduling on computational grids
Supports Computational EconomySupports Computational Economy
The current model supporting Parametric Computing can be extended to support parallel jobs or any other computational model.The current model supporting Parametric Computing can be extended to support parallel jobs or any other computational model.
Plan to use the concept of Advance Resource Reservation in order to offer the feature wherein the user can say “I am willing to pay $…, can you complete Plan to use the concept of Advance Resource Reservation in order to offer the feature wherein the user can say “I am willing to pay $…, can you complete my job by this time…”my job by this time…”
Further Information:Further Information: www.csse.monash.edu.au/~davida/nimrod.htmlwww.csse.monash.edu.au/~davida/nimrod.html
