Optimal Scheduling of Optimal Scheduling of File Transfers with File Transfers with Divisible Sizes on Divisible Sizes on Multiple Disjoint Paths Multiple Disjoint Paths Mugurel Ionut Andreica Mugurel Ionut Andreica Polytechnic University of Polytechnic University of Bucharest Bucharest Computer Science Department Computer Science Department
17
Embed
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths Mugurel Ionut Andreica Polytechnic University of Bucharest Computer.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Optimal Scheduling of File Optimal Scheduling of File Transfers with Divisible Sizes on Transfers with Divisible Sizes on
Multiple Disjoint PathsMultiple Disjoint Paths
Mugurel Ionut AndreicaMugurel Ionut AndreicaPolytechnic University of BucharestPolytechnic University of Bucharest
Computer Science DepartmentComputer Science Department
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 22
maximum profit scheduling (files with divisible maximum profit scheduling (files with divisible sizes) – heuristic algorithmsizes) – heuristic algorithm
Online Resource ManagementOnline Resource Management algorithmic framework for the block partitioning algorithmic framework for the block partitioning
method (=data structure)method (=data structure)
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 33
MotivationMotivation
world-wide development and world-wide development and deployment of distributed systems, deployment of distributed systems, services and applicationsservices and applications
competition for bottleneck resources competition for bottleneck resources => poor performance=> poor performance
inefficient usage of available inefficient usage of available resources => poor performanceresources => poor performance
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 44
Maximum Profit SchedulingMaximum Profit Scheduling
n files (divisible sizes)n files (divisible sizes) szszii – size of the i – size of the ithth file file szsz11≤sz≤sz22 ≤... ≤ sz ≤... ≤ sznn szszii=q=qii··szszi-1i-1, q, qii≥1 is an integer≥1 is an integer ppii=profit of the i=profit of the ithth file (transfer request) file (transfer request)
k pathsk paths path j – available in the interval [0,Tpath j – available in the interval [0,Tjj]] unit speedunit speed
schedule the requests non-preemptively + schedule the requests non-preemptively + at most one request per path per time unitat most one request per path per time unit
equivalent to multiple knapsack with equivalent to multiple knapsack with divisible item sizesdivisible item sizes
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 55
Multiple Knapsack with Multiple Knapsack with Divisible Item SizesDivisible Item Sizes
NP-hardNP-hard can be solved in can be solved in O(n·max{TO(n·max{Tjj}}kk)) (extension from (extension from
single knapsack problem)single knapsack problem) heuristicsheuristics
• Greedy1MultipleKnapsack(n,k)Greedy1MultipleKnapsack(n,k) pack the items optimally in the first knapsackpack the items optimally in the first knapsack remove the q packed items and the knapsackremove the q packed items and the knapsack call Greedy1MultipleKnapsack(n-q,k-1) with the call Greedy1MultipleKnapsack(n-q,k-1) with the
remaining items and knapsacksremaining items and knapsacks• Greedy2Greedy2
sort the items according to some criterion (e.g. sort the items according to some criterion (e.g. Profit/size)Profit/size)
insert the items into knapsack using the First Fit insert the items into knapsack using the First Fit heuristicheuristic
new heuristic algorithm – this papernew heuristic algorithm – this paper
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 66
New Heuristic Algorithm (1/2)New Heuristic Algorithm (1/2) (Multiple Knapsack with (Multiple Knapsack with
Divisible Item Sizes)Divisible Item Sizes) split the items into groupssplit the items into groups
all the items in the same group (i) have the same size all the items in the same group (i) have the same size (sg(sgii))
sgsg11>sg>sg22>...>sg>...>sgGG (G=total # of groups) (G=total # of groups) within a group, sort the items in decreasing order within a group, sort the items in decreasing order
of their profitsof their profits prpri,1i,1≥pr≥pri,2i,2≥…≥pr≥…≥pri,nii,ni (n (nii=the # of items in group i)=the # of items in group i)
insert the items into knapsacks using the insert the items into knapsacks using the First FitFirst Fit heuristicheuristic
traverse the items in inreasing order of group number; traverse the items in inreasing order of group number; within a group, in increasing order of item numberwithin a group, in increasing order of item number
choice of knapsack for an item (i,j) – not important => choice of knapsack for an item (i,j) – not important => because of divisible item sizes, we insert the same set because of divisible item sizes, we insert the same set of items of items
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 77
New Heuristic Algorithm (2/2)New Heuristic Algorithm (2/2) (Multiple Knapsack with (Multiple Knapsack with
Divisible Item Sizes)Divisible Item Sizes) repeatedly improve the initial solutionrepeatedly improve the initial solution
replace an item (i,j) in the knapsack with a replace an item (i,j) in the knapsack with a subset Q of items outside the knapsack, s.t. subset Q of items outside the knapsack, s.t. pr(i,j)<profit(Q) && profit(Q)-pr(i,j)=maximumpr(i,j)<profit(Q) && profit(Q)-pr(i,j)=maximum
ignore item (i,j) from now onignore item (i,j) from now on
time complexitytime complexity O(n·S·min{n,S·log(S)})O(n·S·min{n,S·log(S)}) S=maximum size of an itemS=maximum size of an item
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 88
Performance EvaluationPerformance Evaluation(Multiple Knapsack with (Multiple Knapsack with
new heuristicnew heuristic Greedy1MultipleKnapsackGreedy1MultipleKnapsack Greedy2 (with multiple criteria)Greedy2 (with multiple criteria) direct solution (extension from single knapsack)direct solution (extension from single knapsack)
resultsresults most of the cases – solved optimally by the new most of the cases – solved optimally by the new
heuristicheuristic in terms of quality of solution and running time in terms of quality of solution and running time
=> the new heuristic = clear winner=> the new heuristic = clear winner• followed by Greedy1MultipleKnapsackfollowed by Greedy1MultipleKnapsack
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 99
sequence of n identical filessequence of n identical files to be sent consecutively from the same source to be sent consecutively from the same source
to the same destinationto the same destination sending time per file=1 time unitsending time per file=1 time unit
k data transfer providersk data transfer providers provider jprovider j
• cost: Ccost: Cj j per time unitper time unit• lease at most ONE time interval of duration at most lease at most ONE time interval of duration at most
TTmax,imax,i which includes [T which includes [T1,j1,j, T, T2,j2,j]]
default network link: cost Ldefault network link: cost L ii for transferring for transferring the ithe ithth file file
minimize the total costminimize the total cost
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1010
O(nO(n··k) dynamic programmingk) dynamic programming sort the data transfer providers in increasing sort the data transfer providers in increasing
order of Torder of T2,j2,j: : TT2,12,1≤T≤T2,22,2≤…≤T≤…≤T2,n2,n Cmin[i,j]Cmin[i,j]=the minimum total cost for sending =the minimum total cost for sending
the first j files using a subset of the first i the first j files using a subset of the first i providers (in the sorted order)providers (in the sorted order)
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1111
compute an auxiliary array minpcompute an auxiliary array minpi i in O(n) timein O(n) time
for for j in [Tj in [T2,i2,i, T, T1,i1,i+T+Tmax,imax,i]:]: Cmin[i,j]=min{Cmin[i-1,j], Cmin[i,j-1]+LCmin[i,j]=min{Cmin[i-1,j], Cmin[i,j-1]+L jj, , (j-T(j-T1,i1,i)·C)·Cii+minp+minpii[j-T[j-Tmax,imax,i]}]}
Cmin[k,n] – the answerCmin[k,n] – the answer O(nO(n··k) overallk) overall
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1212
scenarioscenario a resource manager receives resource a resource manager receives resource
allocation and reservation requestsallocation and reservation requests request=amount of resource (bandwidth) + request=amount of resource (bandwidth) +
time constraints (fixed duration, earliest start time constraints (fixed duration, earliest start time, latest finish time)time, latest finish time)
modelmodel time – divided into equally-sized time slotstime – divided into equally-sized time slots many requests simultaneously => we need many requests simultaneously => we need
low response times => efficient data structurelow response times => efficient data structure
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1313
array of n cellsarray of n cells each cell – value veach cell – value vii divide the cells: n/k blocks of size kdivide the cells: n/k blocks of size k update and query functions: O(k+n/k)update and query functions: O(k+n/k) k=sqrt(n) => O(k+n/k)=O(sqrt(n))k=sqrt(n) => O(k+n/k)=O(sqrt(n))
point and range queriespoint and range queries vvii=?=? qFunc(vqFunc(vaa, v, va+1a+1, ..., v, ..., vbb)=?)=?
point and range updatespoint and range updates vvii=u=u vvii=uFunc(u, v=uFunc(u, vii), a≤i≤ b), a≤i≤ b
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1414
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1515
solves efficiently many problemssolves efficiently many problems range addition update+range min queryrange addition update+range min query range set update+range maximum sum range set update+range maximum sum
segment querysegment query ... many more... many more
typical resource reservation requeststypical resource reservation requests range addition update (reserve bandwidth for range addition update (reserve bandwidth for
a full time interval)a full time interval) range minimum query (check to see if enough range minimum query (check to see if enough
bandwidth is available in every time slot)bandwidth is available in every time slot)
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1616
ConclusionsConclusions
offline scheduling problemsoffline scheduling problems maximum profit schedulingmaximum profit scheduling
• multiple knapsack with divisible item sizes – multiple knapsack with divisible item sizes – efficient heuristicefficient heuristic
Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint PathsOptimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths 1717