Towards Achieving Execution Time Predictability in Web Services Middleware Vidura Gamini Abhaya Supervised by Prof. Zahir Tari and Assoc. Prof. Peter Bertok Distributed Systems and Networking Group School of Computer Science and IT RMIT University Melbourne, Australia 15th June 2012 -: Completion Seminar :-
58
Embed
Towards Achieving Execution Time Predictability in Web Services Middleware
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Towards Achieving Execution Time Predictability
in Web Services Middleware
Vidura Gamini Abhaya
Supervised byProf. Zahir Tari and Assoc. Prof. Peter Bertok
Distributed Systems and Networking GroupSchool of Computer Science and IT
RMIT UniversityMelbourne, Australia
15th June 2012
-: Completion Seminar :-
Problem Overview Solutions Conclusion
Presentation Structure
1 The Problem Area
2 Research Overview
3 Solutions
4 Conclusion
Problem Overview Solutions Conclusion
The Problem
Evolution of the Internet
Transition from:User-Centric → Application Centric → Fully Automated Web
* Facilitated by the use of Web services.
Example:
Problem Overview Solutions Conclusion
The Problem
Web Services Middleware
Container applications services are hosted in. Manages all aspectsof their execution.
Characteristics
Optimised for throughput by design
Requests are accepted unconditionally
Uses thread-pools to execute them in a best-effort manner
Processes multiple requests in-parallel using processor-sharing
* They result in inconsistent execution times
Problem Overview Solutions Conclusion
The Problem
10 20 30 40
050
0010
000
2000
030
000
Average Execution Time by No. of Requests Executing in Parallel
Number of Requests
Ave
rage
Exe
cutio
n T
ime
(ms)
Service has a linear execution time complexity.
Started with 2 requests sharing the processor and increased by five upto 47requests sharing the processor.
Problem Overview Solutions Conclusion
The Problem
Why is predictability of execution important in web services?
To achieve consistent service execution times.
To make service selections based on guaranteed execution times
Prevent clients from being set-up for failure.
To open up new application areas that require predictability in execution, suchas Industrial control systems, Avionics, Robotics and Medical diagnostic systemsto the use of web services as a communications platform.
Problem Overview Solutions Conclusion Aim and Scope Research Questions Related Work
Research Overview
What is predictability of execution?
Execution of a web service completing within a given deadline in aconsistent and repeatable manner.
Aim
Achieve predictability of service execution in stand-alone andcluster-based web services middleware
Scope of Research
Use real-time scheduling principles.
Achieving Predictability is limited to the execution within the web servicesmiddleware.
We assume no delays are experienced on the network.
Problem Overview Solutions Conclusion Aim and Scope Research Questions Related Work
Research Questions
1 How can predictability of execution be achieved in stand-aloneweb services middleware?
2 How can predictability of execution be achieved incluster-based web service deployments?
3 How can web services middleware be engineered to havepredictable execution times?
4 How can performance models for such systems be derived andcompared against other techniques?
Problem Overview Solutions Conclusion Aim and Scope Research Questions Related Work
Related Work
Can be categorised broadly into the following,
Admission control mechanisms used to control execution time in web services[Dyachuk et al., 2007], [Elnikety et al., 2004], [Carlstrom and Rom, 2002], [Erradi and Maheshwari, 2005]
Do not consider any predictability attributes such as a deadline or laxity, in the decisions.
Execution time QoS on stand-alone servers[Sharma et al., 2003], [Ching-Ming Tien, 2005]
Achieves some level of differentiated processing, but do not consider any predictability attributes.
Request dispatching techniques aimed at improving execution time[Pacifici et al. 2005], [Garcıa et al., 2009], [Gmach et al., 2008], [Cao et al., 2010]
Achieves execution times defined in SLAs in a probabilistic manner. Execution times can be inconsistent.
Web services middleware using real-time scheduling[Helander and Sigurdsson, 2005], [Mathes et al., 2009]
Predictability is achieved in closed environments, Task properties are known at design time of the system.
Performance models for systems using deadline based scheduling[Li et al., 2007], [Kargahi and Movaghar, 2006], [Chen and Decreusefond, 1996]
Considers M/M/1 systems which assumes service times to be exponentially distributed. Not a good
representation of web service workloads. Only non-preemptive systems are considered.
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Question 1
How can predictability of execution be achieved in stand-alone webservices middleware?
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Real-time Scheduling
Arrival TimeExecution
Deadline
Start Time End TimeWaiting
Time
Laxity
The ability to delay the execution of a task without compromising its deadline.
Laxity = Deadline − Arrival Time − Exec. Time Requirement
Can also be defined as, Laxity = Exec. Time Requirement(Deadline − Arrival Time)
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Predictability of Execution in Stand-alone Middleware
How can predictability of execution be achieved in cluster-basedweb service deployments?
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Predictability in Cluster-based Middleware
Dispatching Algorithms - Objectives
*Ensures deadline requirement of tasks could be met*Distribute requests evenly among executors based on a condition*Avoids executors going into overload conditions
4 algorithms used
RT-RoundRobin
RT-Sequential
RT-ClassBased
RT-LaxityBased
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
RT-RoundRobin
Details
Requests assignment cyclesthrough all executors in RRfashion
Sched. check is done only onceper request
If sched. check fails on selectedserver → request rejected
Hightlights
POC for how a simple dispatching algorithm could be made real-time ready
Processing overhead is minimum
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
RT-Sequential
Details
Requests are assigned in asequential manner
Req’s sent to one executor tillsched. check fails, then assignedto the second and so on
Sched. check happens againstmultiple executors till a requestcan be assigned
Highlights
If it’s possible to schedule a job within the cluster, it will be guaranteed
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
RT-ClassBased
Details
Requests are divided into classesbased on a priority scheme
Mapping of requests to executorsis pre-determined and definedoffline
Sched. check is only consideredwith the assigned executor
Reference implementation usespriorities based on task sizes
Highlights
Results in the reduction of variance of task sizes at each executor
Pre-defined mapping of request sizes to executors could be done usingpre-profiled execution times or execution time history
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
RT-LaxityBased
Details
Laxity = Ability of delaying arequest whilst still meeting itsdeadline
Higher the laxity the morerequests an executor could service
Sched. check is done only againstthe assigned executor
Highlights
Distribution of requests results in a broad range of laxities at each executor
Keeps track of the last two laxities assigned and prevents them being assignedfor the same executor consecutively
Task Size distribution between 3 Executors (1 - 5000000 (Uniform); 0.1sec - 0.5sec (Uniform) roundrobin 3 exec)
Executor 1Executor 2Executor 3
RT-RR accepts between 20.5% (2 Exec. - fastest) and 99.9% (4 Exec. -slowest) of the requests.
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
RT-ClassBased vs Class Based
0
10
20
30
40
50
60
70
80
90
0 50 100 150 200 250 300 350 400
CP
U U
tiliz
atio
n %
Sample #
CPU utilisation at each Executor - (1 - 5000000 (Uniform); 0.25sec - 1sec (Uniform) classbased 3 exec)
Executor 1Executor 2Executor 3
RT-CB accepts between 28.6% (2 Exec. - fastest) and 100% (4 Exec. - slowest) of the requests.
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
RT-Sequential and RT-Laxity
Highest acceptance rate for any algorithm for 2 executors with highest arrivalrate is 38.5% for RT-LaxityBased
RT-Sequential records the lowest percentage of deadlines met out of allalgorithms. However, given its best acceptance rates out of all, the averagenumber of requests meeting its deadline is second to only RT-LaxityBased.
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Question 3
How can web services middleware be engineered to havepredictable execution times?
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Infrastructure
Dev Platform and OS
Solaris 10 08/05 is used as the Real-time OS
Sun Real-time Java Specification is used as the Dev Platform
An example of how the processing deadline could be conveyed tothe middleware - using SOAP headers
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Priority Model
Three priority levels are introduced (used by the Real-time Scheduler)
High - Cannot be interrupted by GC - Used on a thread to allow execution
Mid - Can be interrupted by the GC - Used on a thread to prevent execution
Low - Can be interrupted by the GC, Highest priority level available on StandardJava - Used for meta data requests i.e. WSDL
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Enhancements made to Axis2
Details
Thread-pools have been replaced with RT-Threadpools
A Real-time scheduler has been introduced
Multiple execution lanes are used
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Enhancements made to Synapse
Details
Enhanced version of Synapse used for dispatching
Thread-pools have been replaced with RT-Thread poolsA Real-time scheduler has been introducedMultiple execution lanes are used
RT-Axis2 instances used as Executors
Request processing happens at Dispatcher, Service Invocation at Executors
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Minimising Priority Inversions
Can be caused by, I/O activities such as reading/writing to files and sockets.
To prevent priority inversions,
Avoid outputs resulting in on-screen messages or log file writes
Use in-memory logging and delayed writes
Use offline debugging techniques/toolse.g. Oracle Thread Scheduling Visualizer (TSV)
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Question 4
How can performance models for such systems be derived andcompared against other techniques?
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Performance model
Why model the system?
To derive different performance attributes not based on deadlines
To better compare the performance of techniques used, to others that do notshare the same performance attributes
To investigate the behaviour of the system analytically, without having to turnto implementations
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
System characteristics
Uses EDF scheduling
Arbitrary number of priority classes
Executions are preemptive-resume (work-conserving)
Waiting time considered to be the primary performance indicator
Requests have arbitrary service times
Requests arrive according to a Poisson process
* The system is modelled as a preemptive-resume M/G/1/./EDF queue.
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
System parameters
N number of priority classes
Each class assigned with a constant deadline offset. i.e a task from stream i onits arrival at the system at time t will have a deadline offset of t + di
Priority of a stream is decided by the associated deadline. i.e. i is consideredhigher priority if i < j ⇒ di ≤ dj
Difference between deadline offsets denoted by Dj,i = dj − di .
Queueing discipline among priority classes is EDF and within the class is FCFS.
Each priority class has a different arrival rate denoted by λi
The resultant load for each priority class is denoted by ρi
* The work of Chen K. and Decreusefond L., that approximates waiting time for a
non-preemptive EDF system. Our model, extend their work to a preemptive-resume
system.
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Performance Model
Wi = W0i+
∑ik=1 ρkWk +
∑Nk=i+1 ρk max(0,Wk −Dk,i ) +
∑i−1k=1 ρk min(Wi ,Di,k)
Parameters
In the point of view of a newly arriving request (tagged request)
- Mean waiting time experienced by stream i requests
- Mean residual service time as experienced by stream i requests
- Requests from higher priority classes found in the system by a newly arrived task and are served prior tothe newly arrived.
- Requests from lower priority classes found in the system by a newly arrived task and are served prior to thenewly arrived.
- Requests from higher priority classes arriving at the system after a given task and being serviced prior to it.
Newly Arrived
Request
( d - d )2 1
t2 t + d2 2J
t1 t1+ d1I
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Performance Model
Wi = W0i+
∑ik=1 ρkWk +
∑Nk=i+1 ρk max(0,Wk −Dk,i ) +
∑i−1k=1 ρk min(Wi ,Di,k)
Parameters
In the point of view of a newly arriving request (tagged request)
- Mean waiting time experienced by stream i requests
- Mean residual service time as experienced by stream i requests
- Requests from higher priority classes found in the system by a newly arrived task and are served prior tothe newly arrived.
- Requests from lower priority classes found in the system by a newly arrived task and are served prior to thenewly arrived.
- Requests from higher priority classes arriving at the system after a given task and being serviced prior to it.
t1 t1+ d1
t2 t + d2 2
( d - d )2 1
J
I
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Mean residual service time
For a preemptive M/G/1 queue (not using EDF),
Ci = Xi +∑i−1
j=1(λjCiXj)
Parameters
- Ci - Mean time required to complete service for a stream i request, includingthe time it is preempted
- Xi - Mean of the service time distribution for stream i requests
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Mean residual service time
Ci
s
Ci
si
Ci = Xi +∑i−1
j=1 (ρj min(Di,j ,Ci ))
Let j∗ = subscript j = 1, 2, ..i − 1 such that Di,j ≤ Ci
Let j′
= subscript j = 1, 2, ..i − 1 such that Di,j > Ci
Ci = Xi +∑
j∗(ρj∗Di,j∗) +∑
j′ (ρ
j′Ci )
Ci =Xi+
∑j∗ (ρj∗Di,j∗ )
(1−∑
j′ ρ
j′ )
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Mean residual service time
Let Pi be the probably of a request from stream i being in service at an arrival. Pi
could be defined as:
Pi = λiCi
Mean residual service time of a system can be defined as the the sum of allprobabilities of a job of a given class is in service, times the mean residual service time
for the given class [Kleinrock, 1976]. Therefore, we could define W0i,
W0i=
∑ik=1
(
PkRk
)
Replacing Pk we get,
W0i=
∑ik=1 Rk
(
ρi+∑
j∗ (ρj′Di,j∗ )λi
(1−∑
j′ ρ
j′ )
)
where Ri =Xi
2
2Xifor an M/G/1 system. (Xi
2is the second moment of the service time distribution for
class i requests.
Problem Overview Solutions Conclusion Q1 Q2 Q3 Q4
Evaluation - Analytical results with uniformly distributed service times
0
2000
4000
6000
8000
10000
12000
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Wai
ting
Tim
es (
ms)
Overall Load
Comparison of Analytical Waiting Times - Uniform, 2 Priorities
Preemptive P1Preemptive P2
0
2000
4000
6000
8000
10000
12000
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Wai
ting
Tim
es (
ms)
Overall Load
Comparison of Analytical Waiting Times - Uniform, 3 Priorities
Preemptive P1Preemptive P2Preemptive P3
0
2000
4000
6000
8000
10000
12000
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Wai
ting
Tim
es (
ms)
Overall Load
Comparison of Analytical Waiting Times - Uniform, 4 Priorities
Problem Overview Solutions Conclusion Contributions Outcomes Future Work Acknowledgements
Summary of Contributions
1 How can predictability of execution be achieved in stand-alone web servicesmiddleware?Mathematical model and algorithm for run-time laxity based admission control.Introduction of deadline based scheduling.
2 How can predictability of execution be achieved in cluster-based web servicedeployments?Four request dispatching algorithms based on the laxity property.
3 How can web services middleware be engineered to have predictable executiontimes?Software engineering techniques, algorithms, designs and tools for incorporatingpredictability into web services middleware.
4 How can performance models for such systems be derived and compared againstother techniques?Queueing theory based performance model for a preemptive work conservingM/G/1/./EDF queue.
Problem Overview Solutions Conclusion Contributions Outcomes Future Work Acknowledgements
Thesis Outcomes
V. Gamini Abhaya, Z. Tari, and P. Bertok. Achieving Predictability and Service Differentiation in WebServices. In ICSOC-ServiceWave 09: Proceedings of the 7th International Conference on Service-OrientedComputing, Stockholm, Sweden, November 24-27, 2009, pages 364-372. Springer, 2009.
V. Gamini Abhaya, Z. Tari, and P. Bertok. Using Real-Time Scheduling Principles in Web Service Clustersto Achieve Predictability of Service Execution. In Service Oriented Computing: 8th InternationalConference, ICSOC 2010, San Francisco, CA, USA, December 7-10, 2010. Proceedings, pages 197-212.Springer, 2010.
V. Gamini Abhaya, Z. Tari, and P. Bertok. Building web services middleware with predictable serviceexecution. In Web Information Systems Engineering - WISE 2010: 11th International Conference, HongKong, China, December 12-14, 2010, Proceedings, pages 23-37. Springer-Verlag New York Inc.(* Won best student paper award)
V. Gamini Abhaya, Z. Tari, and P. Bertok. Building web services middleware with predictable executiontimes. World Wide Web Journal, pages 1-60, 28 Mar 2012.
V. Gamini Abhaya, Z. Tari, P. Bertok. P. Zeephongsekul Waiting time analysis for multi-classpreemptive-resume M/G/1/./EDF queues. Journal of Parallel and Distributed Computing. (In Work)
Problem Overview Solutions Conclusion Contributions Outcomes Future Work Acknowledgements
Future Work
Predictability in the network layer
Extending predictability across application boundaries
Reducing request rejections through selective re-transmissiontechniques
Improvements to the preemptive M/G/1/./EDF model
Performance models for preemptive G/G/1/./EDF queue
Problem Overview Solutions Conclusion Contributions Outcomes Future Work Acknowledgements
Acknowledgements
Prof. Zahir Tari and Assoc. Prof. Peter Bertok
Prof. Panlop Zeephongsekul
Miss. Dora Drakopoulos and Mrs. Beti Stojkovski
School of Computer Science and IT and its admin staff
Mr. Don Gingrich
DSN staff and students
My Family
Problem Overview Solutions Conclusion Contributions Outcomes Future Work Acknowledgements