Superscheduling and Resource Brokering

Sven Groot (0024821)

Grid Information Service• Not all information available• Grid Information System

– Globus Monitoring and Discovery Service (MDS2)– Grid Monitoring Architecture (GMA)

• Common features– Organise sensors– Static vs. Dynamic data– Extensible– Agreed upon schema

Stages of Grid Scheduling• Phase 1: Resource Discovery

– Authorization filtering– Application Requirement Definition– Minimal requirement filtering

Stages of Grid Scheduling (2)• Phase 2: System Selection

– Dynamic information gathering– System Selection

Stages of Grid Scheduling (3)• Phase 3: Job Execution

– Advance Reservation (optional)– Job Submission– Preparation Tasks– Monitoring Progress– Job Completion– Cleanup Tasks

Application requirements

Application Requirements (2)• General requirements

– Compute-related requirements– Data-related requirements– Network-related requirements

Application Requirements (3)• Challenges

– Application deployment– Metacomputing– Predicting performance

• Theoretical prediction• History based prediction• Testcase-based prediction

– Adaptive brokering

Application Requirements (4)• Related issues

– Application frameworks– Virtual Organizations– Security requirements– Accounting policies– User preferences

Scheduling in GrADS• Scheduling phases

– Launch-time scheduling– Rescheduling– Meta-scheduling

GrADS

GrADS (2)• Focus applications

– ScaLAPACK– Cactus– FASTA– Iterative applications

• Jacobi method• Game of Life• Fish

GrADS: Launch-time scheduling

GrADS: Launch-time scheduling (2)

• Configurable Object Program – Application requirements definition

• AART• ClassAds• Redline

ClassAds sample

GrADS: Launch-time scheduling (3)

• Performance model– General method

• develop an analytic model for well-understood aspects of applicatio or system performance

• test the analytic model against achieved application performance

• develop empirical models for poorly-understood aspects of application or system behavior

– Some application specific methods– Implemented as shared libraries

GrADS: Launch-time scheduling (4)

• Mapper– Maps data and/or tasks to resources– Different mapping methods

• Equal allocation• Time balancing• Data locality

GrADS: Launch-time scheduling (5)

• Search procedure– General steps

• identify a large number of sets of resources that may be good platforms for the application

• use the application-specific mapper and performance model to generate a data map and predicted execution time for those resource sets

• select the resource set that results in the lowest predicted execution time

GrADS: Launch-time scheduling (6)

• Resource-aware search

GrADS: Launch-time scheduling (6)

• Simulated Annealing

GrADS: Rescheduling• Additional complexities

– Lack of built-in mechanisms– Need to distinguish processors that are

running/not running the current process– Overheads can be high

GrADS: Rescheduling (2)

GrADS: Rescheduling (2)• Rescheduling methods

– Application migration– Process swapping

GrADS: Metascheduling

Grid Service Level Agreements

• Contract– Provide some capability– Perform some task

• Types of SLAs– Resource Service Level Agreements– Task Service Level Agreements– Binding Service Level Agreements

Grid SLAs (2)

Grid SLAs (3)• Motivating scenarios

– Community Scheduler Scenario

Grid SLAs (4)• Motivating scenarios (cont’d)

– File transfer scenario

Grid SLAs• Resource virtualization

Multicriteria• Basic definitions

– Pareto Dominance– Pareto Optimality– Pareto-optimal set– Pareto Front

Multicriteria (2)• Motivations

– Various stakeholders and their preferences– Job scheduling– Application-Level scheduling– Hard constraints and soft constraints

Multicriteria (3)• Approach

– Criteria• Related to stakeholders• Related to entire system• Time criteria• Cost criteria• Resource utilization criteria

– Modeling preferences

Multicriteria (4)• Selection method

– Rule-based system requirements• Expression of policies• Execution of different scheduling procedures• Adaptation to the environment• Selection of the best solution

– Multicriteria optimization

Example (cont’d)

Example (cont’d)• Aggregate criteria

– End user satisfaction

– Resource Owner Satisfaction

– VO overall performance

Example (cont’d)