Page 1
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 1/19
Dynamic request allocation and scheduling forcontext aware applications subject to a percentile
response time SLA in a distributed cloud
Keerthana Boloor∗, Rada Chirkova, Tiia Salo
and Yannis Viniotis∗
∗Department of Electrical and Computer EngineeringDepartment of Computer Science
North Carolina State University
IBM Software GroupResearch Triangle Park
1 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Page 2
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 2/19
Agenda
Agenda
Problem description
Dynamic request allocation and scheduling scheme
Comparison with static allocation and FIFO/WeightedRound Robin scheduling scheme
Conclusion
2 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Page 3
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 3/19
Problem description
Problem description
More web applications are designed to be context aware.
Most context aware applications are built on SOA
principles.
Cloud computing systems - the most preferred platform
for deployment.
Service Level Agreements (SLA) - terms of service andpricing model.
What is this presentation about?
3 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Page 4
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 4/19
Problem description Geographically distributed cloud computing system
Geographically distributed cloud computing system
Clients
Data center hosting
K context-aware
applications
Data center hosting
K context-aware
applications
Data center hosting
K context-aware
applications
Data center hosting
K context-aware
applications
4 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Page 5
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 5/19
Problem description Context aware applications
SOA based context aware application
Contextaware SOA
applications
End servers
Contextdata
stores
Gateway
2. Client request allocated to
and scheduled at end-server
3. Load requiredservice-endpoint
4. Load required
contextdata
DATA CENTER
1. Client request with
context-id
InternetUpdates to contexts at
contextdata stores
5 / 17Cloudcom 2010, Indianapolis, Indiana, USA
P bl d i i M d l f d
Page 6
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 6/19
Problem description Model of an end-server
An end-server serving multiple user classes
Server ‘j’ at data
center ‘i’
Class 1
Class 2
Class K
Each context aware application services multiple classes of users
Each user class is guaranteed different quality of service based on
economic considerations
SLA specifies different service levels and service charges for the
different user classes
6 / 17Cloudcom 2010, Indianapolis, Indiana, USA
P bl d i ti P til S i L l A t
Page 7
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 7/19
Problem description Percentile Service Level Agreements
Percentile Service Level Agreements
P
X 100
Profit
Conformance(%)
0
X % - the fraction of requests of a particular user class which need to have a response
time less than r seconds
$P - The profit charged by the cloud, if the percentile of requests that have response
time less than r seconds is greater than or equal to X %
7 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Problem description Problem statement
Page 8
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 8/19
Problem description Problem statement
Problem statement
Allocate and schedule service requests locally at theend-servers so as to globally:
max
1≤ j ≤K
profit j (1)
where profit j is the profit charged for conformance of the
requests from users of class j .
8 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Problem description Problem statement
Page 9
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 9/19
Problem description Problem statement
Problem statement
Allocate and schedule service requests locally at theend-servers so as to globally:
max
1≤ j ≤K
profit j (1)
where profit j is the profit charged for conformance of the
requests from users of class j .
This problem is NP-hard!!
8 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Solution Management scheme description
Page 10
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 10/19
Solution Management scheme description
Heuristic-based data-oriented request management scheme
Periodic allocation and adaptation at each datacenter.
Al locat ion
phase
Al locat ion
phase
Adaptation
phase
Adaptation
phase
Observation interval (T)
subinterval
Al locat ion
phase
Al locat ion
phase
Al locat ion
phase
Adaptation
phase
Adaptation
phase
Adaptation
phase
9 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Solution Management scheme description
Page 11
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 11/19
Solution Management scheme description
Heuristic-based data-oriented request management scheme
Periodic allocation and adaptation at each datacenter.
Al locat ion
phase
Al locat ion
phase
Adaptation
phase
Adaptation
phase
Observation interval (T)
subinterval
Al locat ion
phase
Al locat ion
phase
Al locat ion
phase
Adaptation
phase
Adaptation
phase
Adaptation
phase
Adaptation phase
Datacenters exchange conformance levels.
Allocation phase
Rank-based request allocation and gi-FIFO scheduling.
Aim at increasing global profit.
9 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Solution Rank-based allocation and gi-FIFO scheduling
Page 12
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 12/19
Solution Rank based allocation and gi FIFO scheduling
Rank-based allocation and gi-FIFO scheduling
Profit-score calculation
Profit: p k
Required global conformance: c k
Current global conformance: cc k
If cc k < c k
Profit-score = p k /(c k −
cc k )
Else
Profit-score = 00 10 20 30 40 50 60 70 80 90 100
0
500
1000
1500
2000
Current conformance of class 1 (%)
P r o f i t − s c o r e
a s s i g n e d
t o
e a c h
a r r i v i n g
r e q u e s t o f c l a s s 1
( $ )
Class 1 SLA − Profit of 2000$ on conformance of 75%
10 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Solution Rank-based allocation and gi-FIFO scheduling
Page 13
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 13/19
g g
Rank-based request allocation
1 Query hash-based lookup table ([context-id,machine-id] or [service-id,machine-id])
2 Rank-based compatibility test
1
The arriving request is assigned a rank based on its profit-score and deadline.
2 Does the arriving request meet its deadline? - Machine compatible!!!
3 Compatible machine not found? - Choose least loaded closest to context DB
11 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Solution Rank-based allocation and gi-FIFO scheduling
Page 14
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 14/19
g g
gi-FIFO scheduling
Choose the request of user class with the highest current profit-score
Choose one with maximum waiting time but which results in a response time less than
or equal to r
If no such request exists, choose the request with higher waiting time resulting in a response time greater than r
gi-FIFO has been proven to be the most suitable for percentile SLAs for a single server
serving multiple classes.
12 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Evaluation
Page 15
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 15/19
Evaluation
Dynamic scheme vs static schemes
5 10 15 20 25 30 35 40 45 500
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
11000
Request rate
P r o f i t i n c u r r e d
( $ )
Dynamic rank based allocation with gi−FIFO scheduling
Static allocation with WRR scheduling
Static allocation with FIFO scheduling
13 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Evaluation
Page 16
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 16/19
Dynamic rank based allocation vs static allocation scheme
0 50 100 1500
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
11000
Request rate
P r o f i t i n c u r r e d
( $ )
Static allocation with gi−FIFO scheduling
Dynamic rank based allocation with gi−FIFO scheduling
14 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Evaluation
Page 17
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 17/19
Variation in subinterval length
0 50 100 150 200 250 300 350 400 450 5000
2000
4000
6000
8000
10000
12000
14000
16000
18000
Subinterval period
P r o f i t o b
t a i n e d ( $ )
Uniform distribution of classes, stringent SLA
Uniform distribution of classes, relaxed SLA
Non−uniform distribution of classes, stringent SLA
Non−uniform distribution of classes, relaxed SLA
Variation in context update interval
0 20 40 60 80 100 120 140 160 180 2000
2000
4000
6000
8000
10000
12000
14000
16000
18000
Contextdata update interval
P r o f i t o b
t a i n e d
( $ )
Low contextdata load times
High contextdata load times
Medium Contextdata load times
15 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Conclusion
Page 18
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 18/19
Conclusion
Identified the need for dynamic request scheduling and allocation for context aware
applications in a distributed cloud.
Proposed a novel rank-based request allocation and gi-FIFO scheduling scheme for
managing percentile SLAs with an aim to maximize profit obtained by the cloud.
16 / 17Cloudcom 2010, Indianapolis, Indiana, USA
Page 19
7/27/2019 Dynamic Request Allocation and Scheduling for Context Aware Applications Subject to a Percentile Response Time SLA in a Distributed Cloud
http://slidepdf.com/reader/full/dynamic-request-allocation-and-scheduling-for-context-aware-applications-subject 19/19
Questions??
17 / 17Cloudcom 2010, Indianapolis, Indiana, USA