AN INGENIOUS APPROACH FOR IMPROVING TURNAROUND TIME OF GRID JOBS WITH RESOURCE ASSURANCE AND ALLOCATION MECHANISM Shikha Mehrotra Centre for Development.

1

AN INGENIOUS APPROACH FOR IMPROVING TURNAROUND TIME OF GRID JOBS WITHRESOURCE ASSURANCE AND ALLOCATION

MECHANISM

Shikha Mehrotra Centre for Development of Advanced Computing

CDAC, Bangalore, India{[email protected]}

10-12 September 2012 IEEE HPEC'12

IEEE HPEC'12 2

Outline• Indian National grid GARUDA• Need for Reservation in Grid• Approach followed in realizing reservation in Garuda Grid

– Architecture– Features

• Performance analysis– Job flow in Garuda grid– Performance metrics – Turnaround time of grid jobs – Case-study

• Turn-around time without reservation• Turn-around time with reservation• Data analysis• Results

• Conclusion10-12 September 2012

IEEE HPEC'12 3

Grid Computing

• Distributed Computing taken to the next level

• Aggregation of Resources from many participants (geographically distributed in general)– Compute resources– Data resources– Special instruments (Telescopes, microscopes, so on..)

• Unified, Seamless access to these resources– Analogous to the “Power Grid”

10-12 September 2012

IEEE HPEC'12 4

India’s National Grid Computing Initiative: GARUDA


Motivation To Collaborate on Research and Engineering of

Technologies, Architectures, Standards and Applications in Grid Computing

To Contribute to the aggregation of resources in the Grid

Production infrastructure with Gigabit networking backbone (NKN) Large HPC computing resources Massive Storage Tools and Services for Unified Access

Currently Connects more than 60 institutions

Academic & Research labs Spans across 17 cities of India Supports 10 Virtual Organizations

Bioinformatics, Seismic engineering, Climate modeling, Drug discovery ….

IEEE HPEC'12 5

Problem Statement

• As the demand for the resources increases more and more, it becomes really difficult to manage the jobs and allocate resources to them and hence most of the jobs will be in the queued state waiting for the resource to be free.


IEEE HPEC'12 6

Our Approach

• Reduce waiting time• Solution : Advance Reservation of resources

– An advance reservation is a reservation that a user or administrator can request and the scheduler can create.

– It guarantees the availability of resources at specified future time slot.


IEEE HPEC'12 7

Compute Reservation

• An advance reservation is essentially defined by the following:

– Start time which is defined using the standard date-time format

– An end time, which is either defined using the standard date-time format or computed from the start time plus a duration value,

– Number and type of resource to be reserved.


Garuda Reservation Architecture

RESERVATION REPLICA DB

LOCAL RESOURCE MANAGER

RESERVATION MANAGER AND SCHEDULER

GARUDA LRM RESERVATION COMPONENT

GARUDA MIDDLEWARE RESERVATION COMPONENT

GLOBUS MIDDLEWARE

GRIDWAY META-SCHEDULER

GARUDA GRID LEVEL RESERVATION COMPONENT

RESERVATION DB

FAILOVER

API

COMMANDS

APPLICATIONS

Garuda Reservation Features• Advanced and Immediate Reservation of resources across multiple

clusters

– Ensure resource availability

– GSI based reservation: Garuda Reservation

– Grid Reservation Failover mechanism:

– Application Programming Interface

– Intelligent resource allocation based on QoS Parameters

– Virtual Organization support

– Avoiding resource under utilization

– Integration with Gridway Meta-scheduler and Globus Middleware

IEEE HPEC'12 10

Performance Analysis


IEEE HPEC'12 11

Performance Metrics

• Mean waiting time

• Execution time

• Turnaround time


IEEE HPEC'12 12

Turnaround Time

• Turnaround time (total time taken between the submission of a program/process/thread/task (Linux) for execution and the return of the complete output to the customer/user)


Job Submission

Job OutputUser

IEEE HPEC'12 13

Performance Analysis


IEEE HPEC'12 14

Turn-around time without reservation


Job Set Waiting Execution Turnaround

Job Set 1 0:04:00 0:17:16 0:22:02

Job Set 2 0:06:00 0:17:27 0:24:14

Job Set 3 0:44:00 0:18:31 1:02:49

Job Set 4 1:11:00 0:17:27 1:38:42

Job Set 5 1:20:00 0:18:26 1:37:41

Turn-around time without reservation


IEEE HPEC'12 16

Turn-around time with reservation


Job Set Waiting ExecutionTurnaroun

d

Job Set 1 0:00:09 0:08:03 0:08:32

Job Set 2 0:00:09 0:08:05 0:08:35

Job Set 3 0:00:09 0:08:07 0:08:37

Job Set 4 0:00:09 0:08:05 0:08:37

Job Set 5 0:00:08 0:07:15 0:07:45

IEEE HPEC'12 1710-12 September 2012

Turn-around time with reservation

IEEE HPEC'12 18

Comparison of Turnaround times


IEEE HPEC'12 19

• Guarantees the availability of resources

• Eliminates the waiting time

• Reduces Turnaround time considerably

• Well integrates into the Grid Middleware

• Built for the production infrastructure • Analysis has shown results that are really encouraging.


Conclusion

IEEE HPEC'12 20

Thank You


AN INGENIOUS APPROACH FOR IMPROVING TURNAROUND TIME OF GRID JOBS WITH RESOURCE ASSURANCE AND ALLOCATION MECHANISM Shikha Mehrotra Centre for Development.

Documents

reservation slide

ieee hpec12 slide

reservation turn

compute reservation

start time

ieee hpec12 motivation

end time

conclusion slide