FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor and Director Centre for Distributed and High Performance Computing School of Information Technologies The University of Sydney, Sydney, Australia Javid Taheri | Postdoctoral Research Fellow
12
Embed
FACULTY OF ENGINEERING & INFORMATION TECHNOLOGIES A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids Albert Y. Zomaya | Professor.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
FACULTY OFENGINEERING &INFORMATION TECHNOLOGIES
A Pareto Frontier for Optimizing Data Transfer vs. Job Execution in Grids
Albert Y. Zomaya | Professor and Director
Centre for Distributed and High Performance Computing School of Information Technologies
The University of Sydney, Sydney, Australia
Javid Taheri | Postdoctoral Research Fellow
2
› Introduction to Grid Computing
› Problem Statement: Data-Aware Job Scheduling
› GA-ParFnt
- Pareto Frontier
- Genetic Algorithm (GA)
› Simulation and Analysis of Results
› Conclusion
3
Grid Computing
4
Problem Statement
› Data Aware Job Scheduling (DAJS)
- (1) the overall execution time of a batch of jobs (NP-Complete)
- (2) transfer time of all datafiles to their dependent jobs (NP-Complete)
Storage
Nodes
Com
puta
tion
Nod
es
Job 1
Job 2
Job 3
Job N
...
File 1
File 2
File 3
...
File M
5
Problem Statement (cont.)
SN
SN
SN
CN
CN
CN
Scheduler
6
Preliminaries
› Pareto Front
› Genetic Algorithm
7
GA for Finding DAJS’ Pareto Front (GA-ParFnt)
8
Simulation
› Test-Grid-4-8
9
Discussion and Analysis
› The shape of Pareto Front
Test-Grid-8-4
10
Discussion and Analysis
› Scheduling Algorithms
11
Conclusion
› GA-ParFnt was effective in finding the Pareto Front of executing jobs vs Transfer time of Datafiles in Grids
› Such Pareto Front could be estimated by exponential funcitons
› Many scheduling algorithms are not optimal, despite their claim.