Chapter 14 Market-Oriented Resource Management and Scheduling: A Taxonomy and Survey By Saurabh Kumar Garg & Rajkumar Buyya.

Chapter 14Market-Oriented Resource

Management and Scheduling: A Taxonomy and Survey

BySaurabh Kumar Garg &

Rajkumar Buyya

World-Wide Grid

Pushes Grid computing into mainstream computing

Market-Oriented Grid

A view of market-oriented grid pushing grid into mainstream computing

Utility Grids and Preliminaries

GSP1

GSP3GSP4

GSP2

Grid Market Exchange

GSP3

GSPm

GSP5

GSC3

GSC1

GSC2

GSCn

Main Participants of Utility Grid

GSP1

GSP3GSP4

GSP2

Grid Market Exchange

GSP3

GSPm

GSP5

GSC3

GSC1

GSC2

GSCn

• Grid Service Consumers (GSCs)

• Grid Service Providers (GSPs)

• Grid Market Exchange– Grid Market Directories – Trading Mechanism– Accounting

Utility Grid: Infrastructural View

Grid resources:Desktops, servers, clusters, networks, applications, storage,

devices + resource manager + monitor

Security Services:Authentication, Single sign-on, secure communication

Job submission, info services, market model, trading, Accounting, License management , pricing, negotiation

Grid applications:Web Portals, Applications

Grid Fabric

GridApplications

Resource management System

Grid programming environment and tools:Languages, API, libraries, compilers, parallelization tools

Core

Middleware

User LevelMiddleware

Bidding mechanism, market based scheduling and negotiation

Auctioneer and Clearing house

Grid Bank , Grid Market Directory, Information Service,Security, Trading, Pricing, Accounting, Reputation System

Grid Exchange

Utility Grid: Infrastructural View

Grid resources:Desktops, servers, clusters, networks, applications, storage,

devices + resource manager + monitor

Security Services:Authentication, Single sign-on, secure communication

Job submission, info services, market model, trading, Accounting, License management , pricing, negotiation

Grid applications:Web Portals, Applications

Grid Fabric

GridApplications

Resource management System

Grid programming environment and tools:Languages, API, libraries, compilers, parallelization tools

Core

Middleware

User LevelMiddleware

Bidding mechanism, market based scheduling and negotiation

Auctioneer and Clearing house

Grid Bank , Grid Market Directory, Information Service,Security, Trading, Pricing, Accounting, Reputation System

Grid Exchange

• Lowest Layer– Grid Fabric

• Physical Infrastructure

• Core Middleware– Hides underline

hetrogeniety– Job submission– Market-oriented

Component for Provider– Security Services

Utility Grid: Infrastructural View

Grid resources:Desktops, servers, clusters, networks, applications, storage,

devices + resource manager + monitor

Security Services:Authentication, Single sign-on, secure communication

Job submission, info services, market model, trading, Accounting, License management , pricing, negotiation

Grid applications:Web Portals, Applications

Grid Fabric

GridApplications

Resource management System

Grid programming environment and tools:Languages, API, libraries, compilers, parallelization tools

Core

Middleware

User LevelMiddleware

Bidding mechanism, market based scheduling and negotiation

Auctioneer and Clearing house

Grid Bank , Grid Market Directory, Information Service,Security, Trading, Pricing, Accounting, Reputation System

Grid Exchange

• Grid Market Exchange• Auction and Clearing

house Faciltiy• Services to enable

trading between consumers and providers, such as Grid Bank, GMD etc

• Reputation System

Utility Grid: Infrastructural View

Grid resources:Desktops, servers, clusters, networks, applications, storage,

devices + resource manager + monitor

Security Services:Authentication, Single sign-on, secure communication

Job submission, info services, market model, trading, Accounting, License management , pricing, negotiation

Grid applications:Web Portals, Applications

Grid Fabric

GridApplications

Resource management System

Grid programming environment and tools:Languages, API, libraries, compilers, parallelization tools

Core

Middleware

User LevelMiddleware

Bidding mechanism, market based scheduling and negotiation

Auctioneer and Clearing house

Grid Bank , Grid Market Directory, Information Service,Security, Trading, Pricing, Accounting, Reputation System

Grid Exchange

• User Side Infrastructure• Portal to submit Grid

Applications and requirements

• Programming language tools

• Resource Management tools

• Market-oriented scheduling mechanisms to participate in utility grid.

Requirements(Consumer Side)

• User-centric Brokers• Bidding/Valuation Mechanism• Market-oriented Scheduling Mechanisms• Allocation of Multiple Resources• Estimation of Resource Usage

Requirements(Resource Side)

• Resource Management Systems• Pricing/Valuation Mechanism• Admission Control and Negotiation Protocols• Commoditization of the Resources

Requirements(Market Side)

• An Information and Market Directory• Support for Different Market Models• Reputation and Monitoring System• Banking system (Accounting, Billing, Payment

mechanism)• Meta-scheduling/Meta-Brokering• Currency Management• Security and Legal System

TAXONOMY OF MARKET-ORIENTED SCHEDULING

• Market-Based Scheduling Mechanism can be broadly catagorizes into Five Components– Based on the resource allocation decision– Based on the objective of the scheduling– Based on the Market Model used for trading– Based on the Application Model for which mechanism is

developed– Based on the participant for whom mechanism is designed

TAXONOMY OF MARKET-ORIENTED SCHEDULING

GRID RESOURCE MANAGEMENT SYSTEMS

• Can be Catagorized into two– Market Based-Schedulers– System Based-Schedulers

Market-Oriented Schedulers• User Side

– Gridbus Broker(UB)– Nimrod-G

• Provider Side– Tycoon (RMS)– Spawn (RMS)– Bellagio (RMS)– Sharp (RMS)– Mariposa (RMS)– GRIA (RMS)– PeerMart (RMS)

Market-Oriented Schedulers

• Market Exchange System– Shirako (I)– OCEAN (I)– CatNets (I)– SORMA (I)– GridEcon (I)– G-Commerce

System-based Schedulers

• Community Scheduler Framework (CSF)• Computing Centre Software (CCS)• GridWay• Maob (Silver)• Condor/G• Grubber/Di-Grubber• eNanos• APST

Gap Analysis(Scheduling Mechanisms)

• Support for Multiple QoS Parameters• Support for Different Application Type • Support for Market-oriented Meta-

scheduling Mechanisms

Gap Analysis(Market Based Systems)

• User Level Middleware– flexibility for user to trade resources in any market– Automatic Bidding System

• Provider ‘s Resource Management Systems– Current System based scheduler needs to be

extended to allow provider to participate in market exchange

– SLA Monitoring– Support for advanced job models such as parallel

applications and workflow

Gap Analysis(Market Side)

• Market Exchange– Negotiation– Allow trading between multiple users and

providers– Scalable– A reputation system– Support for multiple trading/negotiation policy

Conclusion

• Presents the Requirements of Utility grid from each participant point of view

• All the current state-of-art is catagorized using a Taxonomy.

• Survey of both system and market-oriented scheduler is presented and compared to map the requirements and understand the future directions

• Future directions are presented after this comprehensive analysis of current state-of-art.