Allocating a Timeshared Resource: An Application to Grid Computing Alan M. Frisch Artificial Intelligence Group Dept. Computer Science University of York In collaboration with: Mark Bartlett (York), Youssef Hamadi (MSR), Ian Miguel(St.Andrews), Armagan Tarim (Cork) Chris Unsworth (Glasgow) With funding from: Microsoft Research
39
Embed
Allocating a Timeshared Resource: An Application to Grid Computing Alan M. Frisch Artificial Intelligence Group Dept. Computer Science University of York.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Allocating a Timeshared Resource:An Application to Grid Computing
Alan M. Frisch Artificial Intelligence Group
Dept. Computer Science
University of York
In collaboration with: Mark Bartlett (York), Youssef Hamadi (MSR), Ian Miguel(St.Andrews), Armagan Tarim (Cork)
Chris Unsworth (Glasgow)
With funding from: Microsoft Research
Introduction
• Effective use of grid computing will require efficient methods to solve a variety of combinatorial problems, such as:• Resource allocation,• Configuration, and• Scheduling.
• Grid resources are not currently reserved in advance, but will in future.
Advanced Reservation
• This talk considers to the problem of allocating a time-shared or space-shared resource to bids for advanced reservations.– Communication bandwidth– Computer memory or disk space– Equivalent rooms in a hotel that handles block booking
• Given bids for quantities of the resource, each offering a price:• Accept a subset of the bids that can be fulfilled and
maximises revenue.
• This talk presents an algorithm for solving large instances of this problem in batch mode.
• Given:– A finite set of times totally ordered by – Along with each time t, capacity(t), a positive integer– A finite set of bids– Along with each bid b
• price(b), demand(b), two positive integers
• duration(b), a non-empty interval of times
• Find: a set accept bids
• Such that: t times demand(b) capacity(t)
Temporal Knapsack Problem (TKP)
b accept|t duration(b)
• Maximising: price(b)b accept
Sample Instance of TKP
Bid3: 5 units, £20
t2 t3 t4 t5t1
Uniform Capacity: 10
Bid2: 6 units, £10Bid1: 6 units, £11
t2 t3 t4 t5t1
Bid2: 6 units, £10Bid1: 6 units, £11
Solution: Objective Fn = £21
TKP Generalises Knapsack• Every instance of Knapsack is an instance of TKP in which |times| = 1.• Hence TKP is NP-hard. (Easy to see that TKP is NP-easy)
Multi-dimensional Knapsack (MDK) generalises TKP
• Multi-dimensional knapsack– each item has n-dimensional vector as its size – knapsack has n-dimensional vector as its capacity– sum of sizes of items in knapsack cannot exceed
capacity of knapsack• Every instance of TKP is an instance of MDK in
which the times are the dimensions and every item has size of form
(0 … 0 demand … demand 0 … 0)
TKP Reduces toInteger Linear Programming
Bid3: 5 units, £20
t2 t3 t4 t5t1
Uniform Capacity: 10
Bid2: 6 units, £10Bid1: 6 units, £11
Maximise: Xb price(b)b bids
Subject to:
b Xb in {0,1}
6X2 6X2+ 5X3
5X3 6X1 + 5X3
6X1
Solving the TKP
• Decomposition algorithm
• Greedy Algorithm (approximation)
• Integer Linear Programming: CPLEX
Decomposition Algorithm
Decomposition Algorithm
• Makes use of three operations:• Reduce the problem into a simpler one• Split into independent sub-problems.• Branch on accept/reject a bid.
• These operations are composed into a branch and bound search for optimal solutions.
Reduce: Forced Reject
• If at any time during the interval required by a bid, available bandwidth is less than that requested, bid rejected.
10 units required.Bid:
AvailableCapacity:
105
t1 t2 t3 t4 t5 t6 t7 t8 t9
Reduce: Remove Times
Bid3: 5 units
t5 t6 t6 t7 t8 t9 t10 t11 t12t1 t2 t3 t4
Capacity: 10
Demand:
Bid2: 6 unitsBid1: 6 units
t5 t8
56
6• Problem reduced to important decision points.
Reduce: Remove Times Forced Accept
Bid3: 5 units
t5 t6 t6 t7 t8 t9 t10 t11 t12t1 t2 t3 t4
Capacity: 10
Demand:
Bid2: 6 unitsBid1: 4 units
t4 t5
56 • Bid1 Accepted.
Split
• Can solve independently, combine solutions.
Bid3: 5 units
t5 t6 t6 t7 t8 t9 t10 t11 t12t1 t2 t3 t4
Bid2: 6 unitsBid1: 6 units
t5t1 t2 t3 t4
Bid2: 6 units
Bid3: 5 units
t6 t6 t7 t8 t9 t10 t11 t12
Bid1: 6 unitsAND
Branch
t5 t8
56
6
t5 t8
66
Capacity 10Demand 11
t5 t8
66
Reject BlueAccept Blue OR
Demand 6Capacity 5
Capacity 10Demand 6
Interaction: Example
• Bid 1: Forced reject.
Bid2: 5 units
t5 t6 t6 t7 t8 t9t1 t2 t3 t4
Bid1: 6 units
Capacity:Demand:
Interaction: Example
• Times removed. So bid 2 accepted.
Bid2: 5 units
t5 t6 t6 t7 t8 t9t1 t2 t3 t4
Capacity:
Demand:
Decomposition Algorithm: Initialisation(P)
• Reduce(P)
• Split P into set of problems S.
• Foreach s S, Solve(s).
Decomposition Algorithm: Solve(P)
• If no bids, return.
• Select a bid b from bids.
• Branch on:1. Reject(b) then Reduce(P).
Split(P) into set of problems S.
Foreach s S, Solve(s).
2. Accept(b) then Reduce(P).
Split(P) into set of problems S.
Foreach s S, Solve(s).
Deterministic Algorithm:AND/OR Search Space
A,B,C,D,E,F
B,E,F C,D,E,F
E,FC,DB,E,F
AND Split
Reject A Accept BAccept A Reject C,D
Split
A Solution is a Tree
A,B,C,D,E,F
B,E,F
E,FC,DB,E,F
AND Split
Reject A Accept BAccept A Reject C,D
Split
Growing the AND/OR Tree
• While tree contains an OR node with bids do– Choose one such OR node, N– Choose a bid in N; call it b– Generate two children of N by branching on b– Reduce each child– Split each child
• How to choose node to expand, N– Or partially-grown solution tree
• How to choose bid to branch on, b
Propagating Bounds
A,B,C,D,E,F
B,E,F C,D,E,F
E,FC,DB,E,F
ANDSplit
Reject AAccept B (11)
Accept A (10) Reject C,D
Split
Using Bounds: Branch and Bound
A,B,C,D,E,F
B,E,F C,D,E,F
E,FC,DB,E,F
ANDSplit
Reject AAccept B (11)
Accept A (10) Reject C,D
Split
Using Bounds: Branch and Bound
A,B,C,D,E,F
B,E,F C,D,E,F
E,FC,DB,E,F
ANDSplit
Reject AAccept B (11)
Accept A (10) Reject C,D
Split
Choosing a Node: Search Strategy
• AO*– Expand tree with highest upper bound– Admissible: first solution found is best– Optimal: expands fewest possible nodes for any given
upper-bound method– Uses a lot of memory
• Depth First– Expand deepest node– Uses little memory, but expands more nodes than AO*
Choosing a Bid to Branch on
• Longest Duration: choose bid with longest duration– Intuition: maximise reduction; get the most information
from one choice
• Force splits: choose adjacent times and branch on all bids that span those two times.– Consequence: we can split problem between the two
times.– Intuition: maximise splitting– Trade-off between splitting into equal subproblems and
minimising number of branches.
• In our experiments forced splits always outperformed longest duration.
Performance
Compare Performance of Three Solvers
• Decomposition algorithm with depth-first search.
• Decomposition algorithm with AO* search.
• CPLEX on ILP formulation
Generating Random Instances
• Given parameters: max-rate, number of bids
• Allocate resources for a period of one month. Granularity of 15 minutes, gives 2,880 times.
• Start time of each bid: uniform random [1, 2880]
• Duration of each bid: uniform random [1, 100]
• Bandwidth of each bid: uniform random [1, 50]
• Rate of each bid: uniform random [1, max-rate]
• Price and end time of each bid derived from above.
Experimental Results
• Number of bids: 400, 450, 500, 550, 600
• MaxRate: 1.0, 1.2, 1.4, 1.6, 1.8, 2.0
• As MaxRate increases, the instances become harder for all solvers.
Experimental Results
• Now consider performance as num of bids varies.
• Use force-splits for decomposition algorithm
• For each number of bids, used a suite of instances with 20 members from each max-rate value.
• Up to 550 orders: all algorithms less than 10 secs.
• Over 550, AO* runs out of memory.
• Over 600, depth-first decomposition too slow.
• Over 650, CPLEX runs out of memory.
Summary and Future Directions
Summary
• Defined TKP and identified it as a model of advanced reservation
• Presented special-purpose algorithm for TKP– Decomposition is distinctive feature
Future Directions: Improve Current Solver
• Employ a search algorithm that takes middle ground between time-hungry, space-efficient DFS
time-efficient, space-hungry AO*– adapt a space-bounded search algorithm to AND/OR
• Better heuristic for choosing where to force splits Careful tradeoff between advantages of – minimising branching needed to force a split– splitting into equal-sized subproblems
• Better data structures– Identify more redundant times
Future Directions: Improve Model of Advanced Reservation
• Allow flexibility in time requested by a bid
• Accommodate multiple resources and combined bids
• Dynamic environment
– Bids arrive as jobs are being executed
– Some jobs can be thrown out or rescheduled, perhaps with a penalty