Building Web Services Middleware with Predictable Service Execution

Building Web Services Middleware with

Predictable Service Execution

Vidura Gamini AbhayaProf. Zahir Tari and Assoc. Prof. Peter Bertok

Distributed Systems and Networking GroupSchool of Computer Science and IT

RMIT UniversityMelbourne, Australia

December 13, 2010

[WISE 2010 - pages 23-37 in proceedings]

Intro. Guidelines Case Study Results Conclusion

Presentation Structure

1 Introduction

2 Guidelines for Predictability

3 Case Study

4 Empirical Results

5 Conclusion

V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings

Intro. Guidelines Case Study Results Conclusion Motive. QoS in the Cloud ThreadPools WS Stack

1 Introduction


3 Case Study

4 Empirical Results

5 Conclusion



Motivation

Web Services enable communication between DistributedHeterogeneous Systems.



Motivation

Web Services enable communication between DistributedHeterogeneous Systems. It is the foundation for...



Motivation




Motivation




Motivation - QoS in the Cloud



Motivation - QoS in the Cloud

Increased the importance of QoS

Lack of attempts in achieving Execution-time QoS

On achieving consistent and predictable Execution times



Web Services Middleware

WS-Middleware are optimised for throughput by design

Requests are accepted unconditionally

Uses thread-pools to execute them in a best-effort manner

Processes multiple requests in-parallel in a processor sharingmanner



ThreadPool Pattern

Increases throughput

At the increase of the mean execution time of a web service



ThreadPool Pattern

Increases throughput

At the increase of the mean execution time of a web service

No. of requests served in-parallel ∝ Mean time to service a request



Web Services Software Stack

The execution of a web service request goes through multiplelayers of software



Web Services Software Stack

Predictability at the upper layers can only be achieved if it issupported at the lower levels

Thread priority level mismatches between dev platform and OS

Service invocations can be interrupted by a process/thread with higher priority

Delays in resource reservations at OS level


Intro. Guidelines Case Study Results Conclusion Guidelines

1 Introduction


3 Case Study

4 Empirical Results

5 Conclusion



Guidelines for Predictability in Service Execution

- Enable Web Service Middleware to achieve Predictability in Service Execution.- Can be followed in building new middleware or to enhance existing.- Applicable for both SOAP and REST based services



Guidelines for predictability in service execution

G1. Dev platform and OS used must guarantee predictability of execution





G2. Schedule requests based on their deadlines






G3. Select requests based on their laxity







G4. Introduce fine-grain differentiation in request processing







G4. Introduce fine-grain differentiation in request processing

G5. Minimise priority inversions


Intro. Guidelines Case Study Results Conclusion Overview Infra. Commons Standalone Cluster PI

1 Introduction


3 Case Study

4 Empirical Results

5 Conclusion



Case Study - Overview

Infrastructure

Common enhancements made

Schedulability CheckPriority ModelReal-time Scheduler

Stand-alone web service engine

Apache Axis2

Web service cluster

Dispatcher - Apache SynapseExecutors - Apache Axis2



Infrastructure

Dev Platform and OS

Following G1,

Solaris 10 08/05 is used as the Real-time OS

Sun Real-time Java Specification is used as the Dev Platform



Schedulability Check

Following G3, every incoming request is subjectto a Schedulability Check (Ref: Algorithm 1 onpaper and mathematical model in [5]).



Schedulability Check

Following G3, every incoming request is subjectto a Schedulability Check (Ref: Algorithm 1 onpaper and mathematical model in [5]).



Priority Model

Following G4, three priority levels are introduced (used by Real-time schedulerfollowing Algorithm 3 on paper)

High - Cannot be interrupted by GC

Mid - Can be interrupted by the GC

Low - Can be interrupted by the GC, Highest priority level available on StandardJava



Real-time Scheduler

Schedules requests based on their deadlines

Follows Earliest Deadline First (EDF) technique

Uses the priority model on threads to decide which request isexecuted at what time

..and follows G2 in doing so.



Enhancements made to Axis2

Details

Thread-pools have been replaced with RT-Threadpools

A Real-time scheduler has been introduced

Multiple execution lanes are used



Enhancements made to Synapse

Details

RT-Axis2 instances used as Executors

Enhanced version of Synapse used for dispatching

Thread-pools have been replaced with RT-Thread poolsA Real-time scheduler has been introducedMultiple execution lanes are used

Request processing happens at Dispatcher, Service Invocation at Executors



Minimising Priority Inversions

Can be caused by, I/O activities such as reading/writing to files and sockets.

To prevent priority inversions (following G5),

Avoid outputs resulting in on-screen messages or log file writes

Use in-memory logging and delayed writes

Use offline debugging techniques/toolse.g. Oracle Thread Scheduling Visualizer (TSV) [16]



Minimising Priority Inversions - TSV


Intro. Guidelines Case Study Results Conclusion Stand Alone Cluster

1 Introduction


3 Case Study

4 Empirical Results

5 Conclusion



Stand-alone Engine performance

10

20

30

40

50

60

70

80

90

100

0.1s - 0.25s 0.1s - 0.5s 0.25s - 1s 0.25s - 2s

Per

cent

age

of d

eadl

ines

met

off

perc

enta

ge a

ccep

ted

Inter-arrival times

Comparison of Unmod. Axis2 vs RT-Axis2 - Deadline Achievement Rates

Unmod. Axis2RT-Axis2

RT-Axis2 accepts between 18.1% (fastest) and 96.7% (slowest) of the requests.



Stand-alone Engine performance

0

20000

40000

60000

80000

100000

120000

140000

160000

180000

0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06

Exe

cutio

n T

ime

(ms)

Task Size

1 - 5000000 (Uniform); 0.25sec - 1sec (Uniform)

Real-tme Axis2Unmodified Axis2

0

50000

100000

150000

200000

250000

0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06E

xecu

tion

Tim

e (m

s)

Task Size

1 - 5000000 (Uniform); 0.1sec - 0.5sec (Uniform)

Real-tme Axis2Unmodified Axis2



RT-RoundRobin vs Round-Robin

10

20

30

40

50

60

70

80

90

100

0.05s â�� 0.1s 0.1s â�� 0.25s 0.1s - 0.5s 0.25s - 1s

Per

cent

age

of d

eadl

ines

met

Inter-arrival times

Comparison of Round-robin vs. RT-RoundRobin - Deadline Achievement Rates

RR - 2 Execs.RR - 3 Execs.RR - 4 Execs.

RT-RR - 2 Execs.RT-RR - 3 Execs.RT-RR - 4 Execs.

RT-RR accepts between 20.5% (2 Exec. - fastest) and 99.9% (4 Exec. -slowest) of the requests.



RT-RoundRobin vs Round-Robin

0

20000

40000

60000

80000

100000

120000

0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06

Exe

cutio

n T

ime

(ms)

Task Size

Comparison of execution times (1 - 5000000 (Uniform); 0.1sec - 0.25sec (Uniform) roundrobin 3 exec)

RT-RoundRobinSimple Round-Robin

0

2000

4000

6000

8000

10000

12000

0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06E

xecu

tion

Tim

e (m

s)Task Size

Task Size distribution between 3 Executors (1 - 5000000 (Uniform); 0.1sec - 0.5sec (Uniform) roundrobin 3 exec)

Executor 1Executor 2Executor 3


Intro. Guidelines Case Study Results Conclusion Future Work Recap

1 Introduction


3 Case Study

4 Empirical Results

5 Conclusion



Future Work

Further analysis of cluster behaviour with mix of tasks

Advanced dispatching and re-dispatching models

Predictability in the networking layer



1 Introduction


3 Case Study

4 Empirical Results

5 Conclusion



Conclusion

With the growing popularity of Cloud Computing and it’sservice offerings, Execution level QoS has become animportant aspect

Existing WS-Middleware are optimized for throughput therebylacks predictability of execution

Presented Guidelines for WS-Middleware to achievepredictability of execution

Discussed 2 case studies as examples

Presented empirical data, on evaluating the twoimplementations



Thank You !

&

Questions or Comments ?


Building Web Services Middleware with Predictable Service Execution

Technology

gamini abhaya et

mean execution time

executiontime qoson

parallel mean time

predictable execution

web servicev

web serviceno

itrmit universitymelbourne