A Case for Economy Grid Architecture for Service Oriented Grid Computing Rajkumar Buyya, David Abramson, Jon Giddy School of Computer Science and Software.

A Case for Economy Grid Architecture for Service

Oriented Grid Computing

Rajkumar Buyya, David Abramson, Jon Giddy

School of Computer Science and Software Engineering,

Monash University, Melbourne, Australia

www.buyya.com/ecogridhttp://www.gridcomputing.com

Overview

A brief introduction to Grid computing Resource Management issues A Glance at Approaches to Grid computing.

Grid Architecture for Computational Economy

Economy Grid = Globus + GRACE Nimrod-G: A Grid Resource Broker Scheduling Experiments Conclusions

Scheduling Economics

Grid

EconomyGrid

2100

2100 2100 2100 2100

2100 2100 2100 2100

Desktop SMPs or SuperComputers

LocalCluster

GlobalCluster/Grid

PERFORMANCE

Inter PlanetCluster/Grid ??

•Individual•Group•Department•Campus•State•National•Globe•Inter Planet•Universe

Administrative Barriers

EnterpriseCluster/Grid

?

Scalable HPC: Breaking Administrative Barriers

Why Grids ? Large Scale Exploration needs them—Killer

Applications. Solving grand challenge applications using

computer modeling, simulation and analysis

Life Sciences

CAD/CAM

Aerospace

Military ApplicationsDigital Biology Military ApplicationsMilitary Applications

Internet & Ecommerce

What is Grid ?

An infrastructure that couples: Computers – PCs, workstations, clusters,

supercomputers, laptops, notebooks, mobile devices, PDA, etc;

Software – e.g., ASPs renting expensive special purpose applications on demand;

Catalogued data and databases – e.g. transparent access to human genome database;

Special devices – e.g., radio telescope – SETI@Home searching for life in galaxy.

People/collaborators. Potentially Offers a simple, consistent,

dependable, and pervasive access across wide-area networks and presents users with an integrated global resource.

Grid Applications-Drivers

Distributed HPC (Supercomputing): Computational science.

High-throughput computing: Large scale simulation/chip design & parameter studies.

Content Sharing Sharing digital contents among peers (e.g., Napster)

Remote software access/renting services: Application service provides (ASPs).

Data-intensive computing: Data mining, particle physics (CERN), Drug Design.

On-demand computing: Medical instrumentation & network-enabled solvers.

Collaborative: Collaborative design, data exploration, education.

Building and Using Grids requires...

Services that make our systems Grid Ready! Security mechanisms that permit resources

to be accessed only by authorized users. (New) programming tools that make our

applications Grid Ready!. Tools that can translate the requirements of

an application into requirements for computers, networks, and storage.

Tools that perform resource discovery, trading, composition, scheduling and distribution of jobs and collects results.

Players in Grid Computing

What users want ?Users in Grid Economy &

Strategy Grid Consumers

Execute jobs for solving varying problem size and complexity

Benefit by selecting and aggregating resources wisely Tradeoff timeframe and cost

Strategy: minimise expenses Grid Providers

Contribute “idle” resource for executing consumer jobs Benefit by maximizing resource utilisation Tradeoff local requirements & market opportunity

Strategy: maximise returns on services

Sources of Complexity in Resource Management for World Wide

Computing Size (large number of nodes, providers, consumers) Heterogeneity of resources (PCs, Workstatations, clusters, and

supercomputers) Heterogeneity of fabric management systems (single system image OS,

queuing systems, etc.) Heterogeneity of fabric management polices Heterogeneity of applications (scientific, engineering, and commerce) Heterogeneity of application requirements (CPU, I/O, memory, and/or

network intensive) Heterogeneity in demand patters Geographic distribution and different time zones Differing goals (producers and consumers have different objectives and

strategies) Unsecure and Unreliable environment

Traditional approaches to resource management are NOT useful for Grid ?

They use centralised policy that need complete state-information and common fabric management policy or decentralised consensus-based policy.

Due to too many heterogenous parameters in the Grid it is impossible to define:

system-wide performance matrix and common fabric management policy that is acceptable to all.

So, we propose the usage of “economics” paradigm for managing resources

proved successful in managing decentralization and heterogeneity that is present in human economies!

We can easy leverage proven Economic principles and techniques Easy to regulate demand and supply User-centric, scalable, adaptable, value-driven costing, etc. Offers incentive (money?) for being part of the grid!

mix-and-match

Object-oriented

Internet-WWW

Problem Solving Approach

Market/Computational Economy

Grid RMS to support

Ack: Globus..

•Authentication (once).

•Specify (code, resources,

etc.).

•Discover resources.

•Negotiate authorization,

acceptable use, Cost, etc.

•Acquire resources.

•Schedule Jobs.

•Initiate computation.

•Steer computation.

•Access remote data-sets.

•Collaborate with results.

•Account for usage.

•Discover resources.

•Negotiate authorisation,

acceptable use, Cost, etc.

•Acquire resources.

•Schedule jobs.

•Initiate computation.

•Steer computation.

Domain 2

Domain 1

Building an Economy Grid “brokerage” system…..

Foundation for the Grid Economy

Economic Models for Resource Trading

Commodity Market Model Posted Prices Models Bargaining Model Tendering (Contract Net) Model Auction Model

English, first-price sealed-bid, second-price sealded-bid (Vickrey), and Dutch.

Proportional Resource Sharing Model Shareholder Model Partnership Model

Grid Node N

Grid Architecture for Computational Economy

Grid User

Application

Grid Resource Broker

Grid Service Providers

Grid Explorer

Schedule Advisor

Trade Manager

Job ControlAgent

Deployment Agent

Trade Server

Resource Allocation

ResourceReservation

R1

Misc. services

Information Server(s)

R2 Rm…

Pricing Algorithms

Accounting

Grid Node1

…

Grid Middleware Services

…

…

HealthMonitor

Grid Market Services

JobExec

Info ?

Secure

Trading

QoS

Storage

Sign-on

Economy Grid = Globus + GRACE

Applications

MDS

GRAMGlobus Security Interface

Heartbeat MonitorNexus

Local Services

LSF

Condor GRD QBank

PBS

TCP

SolarisIrixLinux

UDP

High-level Services and Tools

DUROC globusrunMPI-G Nimrod/GMPI-IO CC++

GlobusView Grid Status

GASS

GRACE-TS

GARA

GridFabric

GridApps.

GridMiddleware

GridTools

GBankGMD

eCash

JVM

DUROC

Core Services

Science

Engineering Commerce Portals ActiveSheet……

GRACE components

A resource broker (e.g., Nimrod/G) Resource trading protocols A mediator for negotiating between users

and grid service providers (Grid Market Directory)

A deal template for specifying resource requirements and services offers

A trade server A pricing policy specification Accounting (e.g., QBank) and payment

management (GBank)

Grid Open Trading Protocols

Get Connected

Call for Bid(DT)

Reply to Bid (DT)

Negotiate Deal(DT)

Confirm Deal(DT, Y/N)

….

Cancel Deal(DT)

Change Deal(DT)

Get Disconnected

Trade Manager Trade Server

Pricing Rules

DT - Deal Template - resource requirements (BM) - resource profile (BS) - price (any one can set) - status - change the above values - negotiation can continue - accept/decline - validity period

API

Open Trading Finite State Machine

DT

OfferTS

DT

DNDA

OfferTM

< TM, Request for Resource >

<< TM, Update >>< TM, Ask Price ><< TS, Update >>

< TM, Final Offer >< TS, Final Offer >

< TM, Accept > < TS, Reject >

<TM, Rej.>

<TS, Bid >

DT - Deal TemplateTM - Trade ManagerTM - Trade ServerDA - Deal AcceptedDN - Deal Not accepted

Pricing, Accounting, Allocations and Job Scheduling Flow @ each site/Grid

Level

QBankQBank

Resource Manager44

IBM-LL/PBS/….

00

55 88

66 77

Compute Resourcesclusters/SGI/SP/...

0. Make Deposits, Transfers, Refunds, Queries/Reports1. Clients negotiates for access cost.2. Negotiation is performed per owner defined policies. 3. If client is happy, TS informs QB about access deal.4. Job is Submitted5. Check with QB for “go ahead”6. Job Starts7. Job Completes8. Inform QB about resource resource utilization.

Trade Server 3311

Pricing PolicyPricing Policy22

DB@Each SiteDB@Each Site

GRID BankGRID Bank(digital transactions)(digital transactions)00

Service Items to be Charged

CPU - User and System time Memory:

maximum resident set size - page size amount of memory used page faults: with/without physical I/O

Storage: size, r/w/block IO operations Network: msgs sent/received Signals received, context switches Software and Libraries accessed Data Sources (e.g. Protein Data Bank)

How to decide Price ? Fixed price model (like today’s Internet) Dynamic/Demand and Supply (like tomorrow’s Internet) Usage Period Loyalty of Customers (like Airlines favoring frequent flyers!) Historical data Advance Agreement (high discount for corporations) Usage Timing (peak, off-peak, lunch time) Calendar based (holiday/vacation period) Bulk Purchase (register 100 .com domains at once!) Voting -- trade unions decide pricing structure Resource capability as benchmarked in the market! Academic R&D/public-good application users can be offered at

cheaper rate compared to commercial use. Customer Type – Quality or price sensitive buyers. Can be Prescribed by Regulating (Govt.) authorities

Payments- Options & Automation

Buy credits in advance / GSPs bill the user later--”pay as you go”

Pay by Electronic Currency via Grid Bank NetCash (anonymity), NetCheque, and Paypal NetCheque: - http://www.isi.edu/gost/info/netcash/

Users register with NC accounting servers, can write electronic cheques and send (e.g email). When deposited, balance is transferred from sender to receiver account.

NetCash - http://www.isi.edu/gost/info/netcheque/ It supports anonymity and it uses the NetCheque system to

clear payments between currency servers. Paypal.com– account+email is linked to credit card.

Enter the recipient’s email address and the amount you wish to request.

The recipient gets an email notification and pays you at www.PayPal.com

A Glance at Nimrod-G Broker

Grid Middleware

Nimrod/G Client Nimrod/G ClientNimrod/G Client

Grid Information Server(s)

Schedule Advisor

Trading Manager

Nimrod/G Engine

GridStore

Grid Explorer

GE GISTM TS

RM & TS

Grid Dispatcher

RM: Local Resource Manager, TS: Trade Server

Globus,Legion, Condor-g,, Ninf,etc.

G

G

CL

Globus enabled node.Legion enabled node.

CL

Condor enabled node.

RM & TSRM & TS

A resource broker for managing and steering task farming (parametric sweep) applications on computational Grids based on deadline and computational economy.

Key Features A single window to manage & control experiment Resource Discovery Trade for Resources Resource Composition & Scheduling Steering & data management

It allows to study the behaviour of some of the output variables against a range of different input scenarios.

Nimrod/G : A Grid Resource Broker

Globus Legion

Fabric

Nimrod Broker

Nimrod ClientsP-Tools (GUI/Scripting)(parameter_modeling)

Legacy Applications

Condor-G GRACE-TS

Farming Engine

Dispatcher (transport and execution management)

Schedule Advisor

Trading Manager

Grid Explorer

Customised Apps(Active Sheet)

Monitoring and Steering Portals

Algorithm1

AlgorithmN

Middleware

. . .

Computers Storage Networks InstrumentsLocal Schedulers

G-Bank. . .

VariablesResource

Programmable Entities Management

Job Task

. . .

ResourceScheduler JobServer

PC/WS/Clusters Radio TelescopeCondor/LL/Mosix/ . . .Database

Meta-Scheduler

Nimrod/G Grid Broker Architecture

A Nimrod/G Client

A Nimrod/G Client

CostCostDeadlineDeadline

Legion hosts

Globus Hosts

Bezek is in both Globus and Legion Domains

Arlington

Alexandria

Richmond

HamptonNorfolk

Virginia BeachChesapeakePortsmouth

Newport News

Roanoke

Ap p om a toxRive r

Ja m esRive r

Shena nd oa hRive r

Ra p p a ha nnoc kRive r

Potom a cRive r

VIRGINIA77

81

64

64

66

85

Nimrod/G Interactions

Grid Infoservers

Resource Discovery

QueuingSystem

Processserver

Resource allocation

(local)

Userprocess

File accessI/Oserver

Gatekeeper node

JobWrapper

Computational node

Dispatcher

Root node

Scheduler

FarmingEngine

Trade Server

Discover Discover ResourcesResources

Distribute JobsDistribute Jobs

Establish Establish RatesRates

Meet requirements ? Remaining Meet requirements ? Remaining Jobs, Deadline, & Budget ?Jobs, Deadline, & Budget ?

Evaluate & Evaluate & RescheduleReschedule

Discover Discover More More

ResourcesResources

Adaptive SchedulingAlgorithms

Execution Time (not beyond deadline)

Execution Cost (not beyond budget)

Time Minimisation Minimise Limited by budgetCost Minimisation Limited by deadline MinimiseNone Minimisation Limited by deadline Limited by budget

Adaptive Scheduling algorithms

Compose & Compose & ScheduleSchedule

Globus+Legion+Condor/G

Australia

Monash Uni.:

Linux cluster

Solaris WS

Nimrod/G

Globus +GRACE_TS

Europe

ZIB/FUB: T3E/Mosix Cardiff: Sun E6500Paderborn: HPCLineLecce: Compaq SCCNR: ClusterCalabria: Cluster CERN: ClusterPozman: SGI/SP2

Globus +GRACE_TS

Asia/Japan

Tokyo I-Tech.:ETL, Tuskuba

Linux cluster

Globus/LegionGRACE_TS

North America

ANL: SGI/Sun/SP2USC-ISI: SGIUVa: Linux Cluster

Internet

Inter-Continental Grid

Experiment-1 Setup

Workload: 165 jobs, each need 5 minute of cpu time

Deadline: 1 hrs. and budget: 800,000 units

Strategy: minimise cost and meet deadline

Execution Cost with cost optimisation AU Peaktime:471205 (G$) AU Offpeak time: 427155 (G$)

Resources Selected & Price/CPU-sec.

Resource Type & Size

Owner and Location

Grid services

Peaktime Cost (G$)

Offpeak cost

Linux cluster (60 nodes)

Monash, Australia

Globus/Condor 20 5

IBM SP2 (80 nodes)

ANL, Chicago, US

Globus/LL 5 10

Sun (8 nodes) ANL, Chicago, US

Globus/Fork 5 10

SGI (96 nodes)

ANL, Chicago, US

Globus/Condor-G

15 15

SGI (10 nodes)

ISI, LA, US Globus/Fork 10 20

Execution @ AU Peak Time

0

2

4

6

8

10

12

Time (minutes)

Jo

bs

Linux clus ter - Monash (20) Sun - ANL (5) SP2 - ANL (5) SGI - ANL (15) SGI - ISI (10)

Execution @ AU Offpeak Time

0

2

4

6

8

10

12

Time (minutes)

Jo

bs

Linux clus ter - Monash (5) Sun - ANL (10) SP2 - ANL (10) SGI - ANL (15) SGI - ISI (20)

AU peak: Resources/Cost in Use

0

50

100

150

200

250

300

350

400

450

500

Tim e (in m in.)

Co

st o

f R

eso

urc

es in

Use

0

5

10

15

20

25

30

35

40

Tim e (in m in.)

Res

ou

rces

(N

o. o

f C

PU

s) in

Use

After the calibration phase, note the difference in pattern of two graphs. This is when scheduler stopped using

expensive resources.

AU offpeak: Resources/Cost in Use

0

50

100

150

200

250

300

350

Time (in min.)

Co

st o

f R

eso

urc

es i

n U

se

0

5

10

15

20

25

30

Time (in min.)

Res

ou

rces

(N

o.

of

CP

Us)

in

Use

DesignDrug@Home: Data Intensive Computing on Grid

A Virtual Laboratory for “Molecular Modelling for Drug Design" on Peer-to-Peer Grid.

It provides tools for examining millions of chemical compounds (molecules) in the Protein Data Bank (PDB) to identify those having potential use in drug design.

In collaboration with: Kim Branson, Structural

Biology, Walter and Eliza Hall Institute (WEHI)

http://www.csse.monash.edu.au/~rajkumar/dd@home/

Active Sheet: Spreadsheet Processing on Grid

NimrodNimrodProxyProxy

Nimrod/GNimrod/G

Related Works (contd)

Mariposa-Distributed Database system (UCB) query with budget, creates sub-query & divides

budget, trades with (remote) servers UCB Millennium clusters

remote execution environment on clusters and supports computational economy rexec for clusters - proportional resource sharing

UNSW Mungi Storage management: allocation of backing store and

garbage collection of unwanted memory segments depending available credit. Amount of credit required to store increases as available storage space becomes minimum.

Related Works

JaWS - Java based Webcomputing system offers market oriented programming and

computing mechanisms on the Web. Xenoservers - Accounted execution of

untrusted code D’Agents - Agents and computational

economy MOSIX - cost based cluster load balancing A number of theoretical works on pricing. FIPA standard Agents Interaction Protocols

(for trading) - we plan to explore this!

“I think there is a world market for about five computers.”Thomas J. Watson Sr., IBM Founder, 1943

Can we Predict its Future ?

Conclusions

The HPC will be dominated by Peer-to-Peer Grid of clusters.

Adaptive, scalable, and easy to use Systems and End-User applications will be prominent.

Access electricity, internet, entertainment (music, movie,…), etc. from the wall socket!

An Economics –based Service Oriented Grid Computing computing needed for eventual success of Grids!

The impact of World-Wide Grid on 21st century economy will be the same as electricity on 20th century economy.

Thank You… Any ??

Thank You… Any ??