FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor.

FACULTY OFENGINEERING &INFORMATION TECHNOLOGIES

A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids

Albert Y. Zomaya | Professor and Director

Centre for Distributed and High Performance Computing School of Information Technologies

The University of Sydney, Sydney, Australia

Javid Taheri | Postdoctoral Research Fellow

2

› Introduction to Grid Computing

› Problem Statement: Data-Aware Job Scheduling

› GA-ParFnt

- Pareto Frontier

- Genetic Algorithm (GA)

› Simulation and Analysis of Results

› Conclusion

3

Grid Computing

4

Problem Statement

› Data Aware Job Scheduling (DAJS)

- (1) the overall execution time of a batch of jobs (NP-Complete)

- (2) transfer time of all datafiles to their dependent jobs (NP-Complete)

Storage

Nodes

Com

puta

tion

Nod

es

Job 1

Job 2

Job 3

Job N

...

File 1

File 2

File 3

...

File M

5

Problem Statement (cont.)

SN

SN

SN

CN

CN

CN

Scheduler

6

Preliminaries

› Pareto Front

› Genetic Algorithm

7

GA for Finding DAJS’ Pareto Front (GA-ParFnt)

8

Simulation

› Test-Grid-4-8

9

Discussion and Analysis

› The shape of Pareto Front

Test-Grid-8-4

10

Discussion and Analysis

› Scheduling Algorithms

11

Conclusion

› GA-ParFnt was effective in finding the Pareto Front of executing jobs vs Transfer time of Datafiles in Grids

› Such Pareto Front could be estimated by exponential funcitons

› Many scheduling algorithms are not optimal, despite their claim.

12

THANK YOUQuestions?