WW Grid Economic Models for Management of Resources in Peer-to-Peer (P2P) and Grid Computing R. Buyya, H.Stockinger, J.Giddy, D.Abramson Melbourne, Australia.

WW Grid

Economic Models for Management of Resources in Peer-to-Peer (P2P) and Grid

Computing

R. Buyya, H.Stockinger, J.Giddy, D.Abramson

Melbourne, Australia Switzerland

buyya.com/ecogrid

ITCOM 2001, Denver, Aug 19-24, 2001

2

3

Agenda

A quick glance at today’s Grid computing Resource Management challenges for next

generation Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Economic Models for Resource Management Nimrod/G -- Grid Resource Broker Deadline and Budget Constrained (DBC)

Scheduling Experiments on World Wide Grid testbed

Conclusions

Scheduling Economics

Grid

EconomyGrid

4

2100

2100 2100 2100 2100

2100 2100 2100 2100

Desktop SMPs or SuperComputers

LocalCluster

GlobalCluster/Grid

PERFORMANCE

Inter PlanetaryGrid!

•Individual•Group•Department•Campus•State•National•Globe•Inter Planet•Universe

Administrative Barriers

EnterpriseCluster/Grid

?

Scalable HPC: Breaking Administrative Barriers & new challenges

5

Why Grids? Large Scale Explorations need them—Killer

Applications. Solving grand challenge applications using modeling, simulation and analysis

Life Sciences

CAD/CAM

Aerospace

Military ApplicationsDigital Biology Military ApplicationsMilitary Applications

Internet & Ecommerce

6

7

What is Grid ?

An infrastructure that logically couples distributed resources:

Computers – PCs, workstations, clusters, supercomputers, laptops, notebooks, mobile devices, PDA, etc;

Software – e.g., ASPs renting expensive special purpose applications on demand;

Catalogued data and databases – e.g. transparent access to human genome database;

Special devices – e.g., radio telescope – SETI@Home searching for life in galaxy.

People/collaborators. and presents them as an integrated global resource. It enables the creation of virtual enterprises (VEs)

for resource sharing.

Widearea

data archives

8

P2P/Grid Applications-Drivers

Distributed HPC (Supercomputing): Computational science.

High-Capacity/Throughput Computing: Large scale simulation/chip design & parameter studies.

Content Sharing (free or paid) Sharing digital contents among peers (e.g., Napster)

Remote software access/renting services: Application service provides (ASPs) & Web services.

Data-intensive computing: Virtual Drug Design, Particle Physics, Stock Prediction...

On-demand, realtime computing: Medical instrumentation & Mission Critical.

Collaborative Computing: Collaborative design, Data exploration, education.

Service Oriented Computing (SOC): Computing as Utility: New paradigm and new industries.

9

Building and Using Grids require

Services that make our systems Grid Ready! Security mechanisms that permit resources

to be accessed only by authorized users. (New) programming tools that make our

applications Grid Ready!. Tools that can translate the requirements of

an application/user into the requirements of computers, networks, and storage.

Tools that perform resource discovery, trading, selection/allocation, scheduling and distribution of jobs and collects results.

Globus

?

10

Players in Grid Computing

11

What users want ?Users in Grid Economy &

Strategy Grid Consumers

Execute jobs for solving varying problem size and complexity

Benefit by selecting and aggregating resources wisely Tradeoff timeframe and cost

Strategy: minimise expenses Grid Providers

Contribute (“idle”) resources for executing consumer jobs Benefit by maximizing resource utilisation Tradeoff local requirements and market opportunity

Strategy: maximise return on investment

12

Agenda


generation Grid computing A Glance at Approaches to Grid computing Grid Architecture for Computational Economy Economic Models for Resource Management Nimrod-G -- Grid Resource Broker Deadline and Budget Constrained (DBC)


Conclusions


Grid

EconomyGrid

13

Sources of Complexity in Resource Management for World Wide Grid

Computing Size (large number of nodes, providers, consumers) Heterogeneity of resources (PCs, Workstations, clusters, and

supercomputers, instruments, databases, software) Heterogeneity of fabric management systems (single system image OS,

queuing systems, etc.) Heterogeneity of fabric management polices Heterogeneity of application requirements (CPU, I/O, memory, and/or

network intensive) Heterogeneity in resource demand patterns (peak, off-peak, ...) Applications need different QoS at different times (time critical results). The

utility of experimental results varies from time to time. Geographical distribution of users & located different time zones Differing goals (producers and consumers have different objectives and

strategies) Unsecure and Unreliable environment

14

Traditional approaches to resource management & scheduling are NOT useful

for Grid ? They use centralised policy that need

complete state-information and common fabric management policy or decentralised consensus-based

policy. Due to too many heterogenous parameters in the Grid it is impossible to

define/get: system-wide performance matrix and common fabric management policy that is acceptable to all.

“Economics” paradigm proved to effective institution in managing decentralization and heterogeneity that is present in human economies!

Fall of USSR & Emergence of US as world superpower! (monopoly?) So, we propose/advocate the use of computational economics principles

in management of resources and scheduling computations on world wide Grid.

Think locally and act globally approach to grid computing!

15

Benefits of Computational Economies

It provides a nice paradigm for managing self interested and self-regulating entities (resource owners and consumers)

Helps in regulating supply-and-demand for resources. Services can be priced in such a way that equilibrium is maintained.

User-centric / Utility driven: Value for money! Scalable:

No need of central coordinator (during negotiation) Resources(sellers) and also Users(buyers) can make their own decisions and try to

maximize utility and profit. Adaptable It helps in offering different QoS (quality of services) to different applications

depending the value users place on them. It improves the utilisation of resources It offers incentive for resource owners for being part of the grid! It offers incentive for resource consumers for being good citizens There is large body of proven Economic principles and techniques available, we can

easily leverage it.

16

New challenges of Computational Economy

Resource Owners How do I decide service prices ? (economic models?) How do I specify them ? How do I enforce them ? How do I advertise & attract consumers ? How do I do accounting and handle payments? …..

Resource Consumers How do I decide expenses ? How do I express QoS requirements ? How I trade between timeframe & cost ? ….

Any tools, traders & brokers available to automate the process ?

17

Agenda




Conclusions


Grid

EconomyGrid

18

mix-and-match

Object-oriented

Internet/partial-P2P

Network enabled Solvers

Market/Computational Economy

19

Many Testbeds ? & who pays ?,

who regulates supply and demand ?

GUSTO (decommissioned)

Legion Testbed

NASA IPG

World Wide Grid

WW Grid

20

Agenda




Conclusions


Grid

EconomyGrid

21

Building an Economy Grid(Next Generation Grid

Computing!)

To enable the creation and promotion of:Grid Marketplace (competitive)

ASPService Oriented Computing

. . .And let users focus on their own work (science, engineering, or commerce)!

22

Grid Node N

GRACE: A ReferenceGrid Architecture for Computational Economy

Grid User

Application

Grid Resource Broker

Grid Service Providers

Grid Explorer

Schedule Advisor

Trade Manager

Job ControlAgent

Deployment Agent

Trade Server

Resource Allocation

ResourceReservation

R1

Misc. services

Information Server(s)

R2 Rm…

Pricing Algorithms

Accounting

Grid Node1

…

Grid Middleware Services

…

…

HealthMonitor

Grid Market Services

JobExec

Info ?

Secure

Trading

QoS

Storage

Sign-on

Grid Bank

See PDPTA 2000 paper!

23

GridFabric

GridApps.

GridMiddleware

GridTools

Networked Resources across Organisations

Computers Clusters Data Sources Scientific InstrumentsStorage Systems

Local Resource Managers

Operating Systems Queuing Systems TCP/IP & UDP

…

Libraries & App Kernels …

Distributed Resources Coupling Services

Security Information … QoSProcess

Development Environments and Tools

Languages Libraries Debuggers … Web toolsResource BrokersMonitoring

Applications and Portals

Prob. Solving Env.Scientific …CollaborationEngineering Web enabled Apps

Resource Trading

Grid Components

Market Info

24

Economy Grid = Globus + GRACE

Applications

MDS

GRAMGrid Security Interface

Heartbeat MonitorNexus

Local Services

LSF

Condor GRD QBank

PBS

TCP

SolarisIrixLinux

UDP

High-level Services and Tools

DUROC globusrunMPI-G Nimrod/GCC++

Grid Status

GASS

GRACE-TS

GARA

GridFabric

GridApps.

GridMiddleware

GridTools

GBankGMD

eCash

JVM

DUROC

Core Services

Science

Engineering Commerce Portals ActiveSheet……

See IPDPS HWC 2001 paper!

……

25

Agenda




Conclusions


Grid

EconomyGrid

26

Economic Models

Price-based: Supply,demand,value, wealth of economic system

Commodity Market Model Posted Price Model Bargaining Model Tendering (Contract Net) Model Auction Model

English, first-price sealed-bid, second-price sealed-bid (Vickrey), and Dutch (consumer:low,high,rate; producer:high, low, rate)

Proportional Resource Sharing Model Monopoly (one provider) and Oligopoly (few players)

consumers may not have any influence on prices. Bartering

Shareholder Model Partnership Model

27

Grid Open Trading Protocols

Get Connected

Call for Bid(DT)

Reply to Bid (DT)

Negotiate Deal(DT)

Confirm Deal(DT, Y/N)

….

Cancel Deal(DT)

Change Deal(DT)

Get Disconnected

Trade Manager Trade Server

Pricing Rules

DT - Deal Template - resource requirements (BM) - resource profile (BS) - price (any one can set) - status - change the above values - negotiation can continue - accept/decline - validity period

API

28

Various Criteria for Judging Effectiveness of Economic Models

Social Welfare global good of all

Pareto Efficiency global perspective

Individual Rationality better off by participating in negotiation

Stability mechanisms that cannot be manipulated

Computational Efficiency protocols should not consume too much of time

Distribution and Communication Efficiency communication overhead to capture a desirable global

solution

29

A Commodity Market Model

“Solve this in5hrs for $20”

Grid Market Directory (GMD)

ResourceBroker

Grid Info. Service

GTS

GTS

(Grid Service Provider)

GTS

GTS GTS

“register me as GSP”

“Give me list of GSPs & price?”

“service available?”

(GTS - Grid Trade Server)

(GSP)

“service available?”“service availab

le?”

(RB selects GSPs)

30

How to decide Price ?

Fixed price model (like today’s Internet) Dynamic/Demand and Supply (like tomorrow’s Internet) Usage Period Loyalty of Customers (like Airlines favoring frequent flyers!) Historical data Advance Agreement (high discount for corporations) Usage Timing (peak, off-peak, lunch time) Calendar based (holiday/vacation period) Bulk Purchase (register 100 .com domains at once!) Voting -- trade unions decide pricing structure Resource capability as benchmarked in the market! Academic R&D/public-good application users can be offered at

cheaper rate compared to commercial use. Customer Type – Quality or price sensitive buyers. Can be Prescribed by Regulating (Govt.) authorities

31

Posted Price Model

“Solve this bynext day for $5”


ResourceBroker

“2hrs SP2, $5”

Grid Info. Service

GTS

GTS


GTS

GTS GTS

“T3E, $9/hr, Sunday”

“Free for Genome”“10% discount today”

“Any SP2/T3E? offers”“Free or < $2/hr clusters+matlab”

“5MB free”


(GSP)

32

Bargaining Model



ResourceBroker

Grid Info. Service

GTS

GTS


GTS

GTS GTS

“register me as GSP”

“Give me list of GSPs”

“access price ?, 2, 3 ?”


(GSP)

“access price ?”“access p

rice ?”(RB negotiates for the best price)

33

Tender/Contract-Net Model



ResourceBroker

Grid Info. Service

GTS

GTS


GTS

GTS GTS

“Any Ads for service tenders”

“Post: call for tenders”


(GSP)

(GSPs bid)

“gsp1 bid”

“gsp3 bid?”

“gsp2 bid?”

“g

sp

N b

id?”

Closed Reverse Auction ?Buyers name their price and supplies compete to bid the lowest price.Eg: GotFrom.com

34

Auction Model

Grid Market Auctioneer (GMA)

ResourceBroker

“SP2 time, 9pm-8am”

Grid Info. Service

GTS

GTS


GTS

GTS GTS

“Post: auction T3E service”

“Solve this in20 hrs for $5”


(GSP)

ResourceBroker

….

“$2, gsp

1”

“$4,

gsp

1”

“$2,

gsp

2”


(RBs bid)

35

Auction Types

English Auction - first-price open-cry each bidder free to raise his bid and highest bidder

wins at his bid price. First-price sealed bid

each bidder submits one bid and highest bidder wins at his bid price.

Vickrey Auction- Second-price sealed bid each bidder submits one bid and highest bidder

wins at the price of second highest bid. Dutch (descending) Auction

seller continuously lowers the price until one of the bidders’ bids.

36

Proportional Resource Sharing Model


ResourceBroker

“SP2 time, 9pm-8am”

Grid Info. Service

GTS

GTS


GTS

GTS GTS

“Post: auction T3E service”



(GSP)

ResourceBroker

….

“RB1: $2”


(RBs bid)

“Give me list of GSPs”

“RBn: $4”

Resource Share ?Bid/(sum of all bids).E.g., RB1 share = 1/3 RB n share - 2/3

37

Bartering: Partnership/Shareholder

A group of organizations pool in resources (money) together or govt. funded.

APAC (Australian Partnership for Advance Computing)

VPAC (Victorian PAC) - VIC unis and govt. NPACI in US

Allocation proportional to contribution. When in demand: follow proportional resource

sharing strategy & QoS. A group of individuals pool in resources (idle

cycles) like in Condor pool. Contribute for public good/fame (SETI@Home

or distributed.net)

38

Agenda




Conclusions


Grid

EconomyGrid

39

A resource broker for managing, steering, and executing task farming (parametric sweep/SPMD model) applications on Grid based on deadline and computational economy.

Based on users’ QoS requirements, our Broker dynamically leases services at runtime depending on their quality, cost, and availability.

Key Features A single window to manage & control experiment Persistent and Programmable Task Farming Engine Resource Discovery Resource Trading Scheduling & Predications Generic Dispatcher & Grid Agents Transportation of data & results Steering & data management Accounting

Nimrod/G : A Grid Resource Broker

40

Parametric Processing

Multiple RunsSame ProgramMultiple Data Killer Application for the Grid!

ParametersAge Hair

23 CleanAge Hair

23 Clean23 Beard28 Goatee

Age Hair23 Clean23 Beard

Age Hair23 Clean23 Beard28 Goatee28 Clean

Age Hair23 Clean23 Beard28 Goatee28 Clean19 Moustache

Age Hair23 Clean23 Beard28 Goatee28 Clean19 Moustache10 Clean

Age Hair23 Clean23 Beard28 Goatee28 Clean19 Moustache10 Clean

-4000000 Too much

Courtesy: Anand Natrajan, University of Virginia

Magic Engine

See IPDPS 2000 paper!

41

Parameter Processing on the Grid

Aggregate Job SubmissionAggregate View

0

10

20

30

40

50

60

70

80

90

1st Qtr 2nd Qtr 3rd Qtr 4th Qtr

East

West

North

South

Submit & Play!

42

A Glance at Nimrod-G Broker

Grid Middleware

Nimrod/G Client Nimrod/G ClientNimrod/G Client

Grid Information Server(s)

Schedule Advisor

Trading Manager

Nimrod/G Engine

GridStore

Grid Explorer

GE GISTM TS

RM & TS

Grid Dispatcher

RM: Local Resource Manager, TS: Trade Server

Globus, Legion, Condor, etc.

G

G

CL

Globus enabled node.Legion enabled node.

GL

Condor enabled node.

RM & TSRM & TS

C LSee HPCAsia 2000 paper!

43

A Nimrod/G Monitor

A Nimrod/G Monitor

CostCostDeadlineDeadline

Legion hosts

Globus Hosts

Bezek is in both Globus and Legion Domains

Arlington

Alexandria

Richmond

HamptonNorfolk

Virginia BeachChesapeakePortsmouth

Newport News

Roanoke

Ap p om a toxRive r

Ja m esRive r

Shena nd oa hRive r

Ra p p a ha nnoc kRive r

Potom a cRive r

VIRGINIA77

81

64

64

66

85

44

User Requirements: Deadline/Budget User Requirements: Deadline/Budget

45

Discover Discover ResourcesResources

Distribute JobsDistribute Jobs

Establish Establish RatesRates

Meet requirements ? Remaining Meet requirements ? Remaining Jobs, Deadline, & Budget ?Jobs, Deadline, & Budget ?

Evaluate & Evaluate & RescheduleReschedule

Discover Discover More More

ResourcesResources

Adaptive SchedulingAlgorithms

Execution Time (not beyond deadline)

Execution Cost (not beyond budget)

Time Minimisation Minimise Limited by budgetCost Minimisation Limited by deadline MinimiseNone Minimisation Limited by deadline Limited by budget

Adaptive Scheduling Algorithms

Compose & Compose & ScheduleSchedule

See HPDC AMS 2000 paper!

46

Agenda




Conclusions


Grid

EconomyGrid

47

Experiment Setup

Workload (Hypothetical Application): 165 jobs, each need 5 minute of CPU time

Deadline: 2 hrs. and budget: 396000 units (G$)

Strategy: minimise time / cost Execution Cost with cost optimisation

Optimise Cost: 115200 (G$) (finished in 2hrs.) Optimise Time: 237000 (G$) (finished in 1.25 hr.) In this experiment: Time-optimised scheduling run

costs double that of Cost-optimised. Users are able to trade-off between Time Vs. Cost

depending on QoS requirements.

48

Globus+LegionGRACE_TS

Australia

Monash Uni.:

Linux cluster

Solaris WS

Nimrod/G

Globus +GRACE_TS

Europe

ZIB/FUB: T3E/Mosix Cardiff: Sun E6500Paderborn: HPCLineLecce: Compaq SCCNR: ClusterCalabria: Cluster CERN: ClusterPozman: SGI/SP2

Globus +GRACE_TS

Asia/Japan

Tokyo I-Tech.:ETL, Tuskuba

Linux cluster

Globus/LegionGRACE_TS

North America

ANL: SGI/Sun/SP2USC-ISI: SGIUVa: Linux ClusterUD: Linux clusterUTK: Linux cluster

Internet

World Wide Grid (WWG)

Globus +GRACE_TS South America

Chile: Cluster

WW Grid

WW Grid

49

Resources Selected & Price/CPU-sec.

Cost_Opt.Time_Opt

4

5

1

1

1

153

378Globus, GTS, ForkSGI-ISI, LA, US

42

9

6

7

64

No. of Jobs Executed

7Globus, GTS, ForkSun-ANL, Chicago,US

3Globus, GTS, ForkSolaris/Ultas2

TITech, Tokyo, Japan

4Globus, GTS, ForkLinux-Barbera-CNR, Pisa, Italy

3Globus, GTS, ForkLinux-Prosecco-CNR, Pisa, Italy

2Globus, GTS, Condor

Linux Cluster-Monash, Melbourne, Australia

Cost/CPU sec.in G$

Grid services & Fabric

Resource & Location

Total Experiment Cost (G$) 237000 115200

Time to Complete Exp. (Min.) 70 119

50

DBC Scheduling for Time Optimization

0

2

4

6

8

10

12

Time (in Minute)

No.

of

Tas

ks i

n E

xecu

tion

Condor-Monash Linux-Prosecco-CNR Linux-Barbera-CNR

Solaris /Ultas2-TITech SGI-ISI Sun-ANL

51

DBC Scheduling for Cost Optimization

0

2

4

6

8

10

12

14

Time (in Minute)

No.

of

Tas

ks i

n E

xecu

tion

Condor-Monash Linux-Prosecco-CNR Linux-Barbera-CNR

Solaris /Ultas2-TITech SGI-ISI Sun-ANL

52

Agenda




Conclusions & Pointers


Grid

EconomyGrid

53

Summary and Conclusions P2P and Grid Computing is emerging as a next generation

computing platform for solving large scale problems through sharing of geographically distributed resources.

Resource management is a complex undertaking as systems need to be adaptive, scalable, competitive,…, and driven by QoS.

We proposed a framework based on “computational economies” and discussed several economic models for resource allocation and for regulating supply-and-demand for resources.

Scheduling experiments on World Wide Grid demonstrate our Nimrod-G broker ability to dynamically lease or rent services at runtime based on their quality, cost, and availability depending on consumers QoS requirements.

Economics paradigm for QoS driven resource management is essential to push P2P/Grids into mainstream computing!

54

Thank You… Any ??

Thank You… Any ??

55

Download Software & Information

Nimrod & Parameteric Computing: http://www.csse.monash.edu.au/~davida/nimrod/

Economy Grid & Nimrod/G: http://www.buyya.com/ecogrid/

Virtual Laboratory/Virtual Drug Design: http://www.buyya.com/vlab/

Grid Simulation (GridSim) Toolkit (Java based): http://www.buyya.com/gridsim/

World Wide Grid (WWG) testbed: http://www.buyya.com/ecogrid/wwg/ Looking for new volunteers to grow

Please contact me to barter your & our machines!

Want to build on our work/collaborate: Talk to me now or email: [email protected]

WW Grid Economic Models for Management of Resources in Peer-to-Peer (P2P) and Grid Computing R. Buyya, H.Stockinger, J.Giddy, D.Abramson Melbourne, Australia.

Documents

grid computing slide

generation grid

planetary grid

grid resource broker

expenses grid providers

resource management

ww grid economic models

collaborative computing