Evaluating the Correctness and Effectiveness of a Middleware QoS Configuration Process in DRE Systems Institute for Software Integrated Systems Dept of.

Evaluating the Correctness and Effectiveness of a Middleware QoS

Configuration Process in DRE Systems

Institute for Software Integrated Systems

Dept of EECS, Vanderbilt UniversityNashville, TN, USA

Amogh Kavimandan, Anantha Narayanan, Aniruddha Gokhale, Gabor

[email protected]

www.dre.vanderbilt.edu/~gokhalePresented at ISORC 2008, Orlando, FL

May 5-7, 2008

Distributed Real-time and Embedded (DRE) Systems• DRE systems traits:

• Composed from diverse, complex sub-systems

• Stringent requirements on resources

• Multiple, simultaneous QoS requirements at local- (sub-system) and global- (application) level

• Heterogeneous platforms

• Increasing scale

2

• Trend towards component-based application development

• Functionality realized via composition, deployment & configuration of application components on execution platforms

Specification

Composition

Deployment Planning

Configuration

ComponentImpl Impl Impl

Properties Properties



compose

DRE Systems Software Development Processes• DRE systems illustrate the

following (non-exhaustive) development stages:

• Specification – functional description, interface definition, implementation

3

Specification

Composition

Deployment Planning

Configuration





Application Packages

Assembly

ComponentComponent

ComponentComponent

compose

create




• Composition – functional integration, hierarchical organization & packaging

4

Specification

Composition

Deployment Planning

Configuration






Assembly

ComponentComponent

ComponentComponent

compose

Deployment Plan

create

System Resource Profile

Bridge

Firewall

Computing Resources

Network Topology

Infrastructure





• Deployment – computing resource allocation, node placement

5

Specification

Composition

Deployment Planning

Configuration






QoS Design

Service Levels

Preserve priority

Event Filtering

Prioritize invocations

Assembly

ComponentComponent

ComponentComponent

compose

Deployment Plan

create

System Resource Profile

Bridge

Firewall

Computing Resources

Network Topology

Infrastructure





• Deployment – computing resource allocation, node placement

• Configuration – choosing right set of parameters for hosting infrastructure (e.g., middleware) for actuating application QoS requirements

6Focus on DRE systems middleware QoS configuration

CCM Platform ThreadPool

Allow_request_buffering;max_buffered_requests;stacksize;max_buffer_size;allow_borrowing;

PriorityModelPolicypriority_model;priority_value;

BandedConnectionlow;high;

Laneslong static_threads; long lane_priority;long dynamic_threads;

RTECFactoryec_dispatching;ec_dispatch_thrds;ec_filterting;ec_scheduling;

PublisherBehaviorpublisher_control;pub_cntrl_period;publisher_lock;

FilterGroup

SubscriberBehaviorsubscriber_control;sub_cntrl_period;subscriber_lock;

Observer

System QoS RequirementsMultiple service levels

Preserve invocation priority

On-demand event delivery

Publisher/subscriber/topic event filtering

Prioritize service invocations

Reactive event control

Group events

Middleware QoS Configuration Process

ProxyCollection

7

• What vs how - Middleware platforms provide what is required to achieve system QoS not always how it can be achieved• No centralized orchestrator to realize QoS

from options providing individual QoS control

• Non-trivial to manually perform QoS configuration activity• Choose appropriate configuration

mechanisms in an application-specific manner, particularly for large applications

• Middleware does not prevent developers from choosing semantically invalid configurations to achieve QoS

Middleware QoS Configuration: Hard Challenges

Lack of effective QoS configuration tools result in QoS policy mis-configurations that are hard to analyze & debug

Automated Middleware QoS Configuration: Hard Challenges

8

• Use DRE system QoS requirements to automated middleware QoS configurations

• Different DRE systems (e.g., shipboard computing environment, emergency response services) exhibit variability in domain-specific QoS requirements

• How do we deal with this variability?

• Must deal with plethora of configuration mechanisms of hosting middleware platforms for muliple m/w platforms

• How do we bridge the gap between domain-specific requirements and configuration mechanisms?

Control Center

SystemResourceManager

Tool must address the variability and bridge this gap

– Shields developers from configuration semantics

• Automated translation using model transformation to generate system

QoS configurations– Reusable, one-step translation– Encodes best practices in QoS mapping

QoS Policy Modeling

request/response

publish/subscribe

brokered notification

reliable messaging

QoS Configurations

Metamodel

Target Model

instanceof

Graph Rewriting Rules

G G’ G G’

G G’ G G’ G G’

Requirements Metamodel

Source Model

instanceof

Transformation Engine

Model Transformation

Non-functional Analysis

DRE Developer

System QoS Configuration

Evolution

System-level QoS

requirements


QoS Configurations

Metamodel

QoS Mapping

• QUality of service pICKER (QUICKER):

• Domain-independent QoS modeling languages – express system QoS in terms of requirements semantics

– Easier to model, evolve, lesser modeling effort QoS Policy Modeling

request/response

publish/subscribe


reliable messaging

9

Solution Approach: QUICKER

ISORC 2007: General idea of QUICKER

RTAS 2008: Model transformation algorithms for CCM/RT

QoS Policy Modeling

request/response

publish/subscribe


reliable messaging

Validation & Verification

ComponentS2

S3S4

S1

ComponentS2

S3S4

S1

ComponentS2

S3S4

S1

ComponentS2

S3S4

S1

10

ISORC 2008: Evaluating QUICKER

1. Is the transformation process correct?• We use structural

correspondence to prove the transformations correct

2. Are the generated artifacts correct?• We use model

checking

3. Do the generated configurations deliver the desired QoS?• We use empirical

validation

QoS Configurations

Metamodel

Target Model

instanceof


G G’ G G’



Source Model

instanceof




DRE Developer


Evolution

System-level QoS

requirements


QoS Configurations

Metamodel

QoS Mapping

QoS Policy Modeling

request/response

publish/subscribe


reliable messaging

Focus on the correctness & effectiveness of our QoS configuration process

• Verification framework• Specify correctness properties at meta-level• Add annotations for each instance (correspondence rules)• Use annotations to automatically verify whether the instances

satisfy the correctness properties

• We do not attempt to prove the general correctness of the transformation itself

Source Meta

Target Meta

Source Model

Target Model

CorrectnessSpecification


CorrectnessChecker

AnnotationsCertificate

(1) Verifying the Correctness of QoS Mapping Algorithms

11

RealTimeConfiguration

-priority_model : Policy-default_priority : long

PriorityModelPolicy

10..1

StructuralCorrespondence-bursty_client_requests : bool-fixed_priority_service_execution : bool

RTRequirements

11 1 1

• Add cross links to identify corresponding elements• Rules specify correspondence conditions for selected types• At the end of the transformation, the instance models are checked

if they satisfy all the correspondence conditions

Input Model Output Model

CorrespondenceRules

crosslink

(1) Verifying the Correctness of QoS Mapping Algorithms

12

13

Container

COMPONENTEXECUTORS

ComponentHome

POA

CallbackInterfaces

I nt e

r na

lIn

terf

ac e

s

Eve

ntS

inks

Face

t s

Rec

ep

t acl

esE

ven

tS

ou

r ce

s

ComponentReference

Co

mp

on

en

t C

on

t ex

t

COMPONENT SERVER 1

Container

COMPONENTEXECUTORS

ComponentHome

POA

CallbackInterfaces

I nte

r na

lI n

t erf

ace

s

Ev

ent

Sin

ksF

ace

t s

Re

ce

pt a

cle

sE

v en

tS

ou

rce

s

ComponentReference

Co

mp o

nen

t C

on

t ex

t

COMPONENT SERVER 2

ORB

End-to-End PriorityPropagation

ThreadPools

Portable PrioritiesProtocol Properties

Priority Band

• Dependencies may span beyond “immediate neighbors”, e.g.,

– application execution path

– components belonging to separate assemblies

• Empirically validating configuration changes slows down development & QA process considerably

• Several iterations before desired QoS is achieved (if at all)

Container

COMPONENTEXECUTORS

ComponentHome

POA

CallbackInterfaces

I nt e

r na

lIn

terf

ac e

s

Eve

ntS

inks

Face

t s

Rec

ep

t acl

esE

ven

tS

ou

r ce

s

ComponentReference

Co

mp

on

en

t C

on

t ex

t

COMPONENT SERVER 1

Assembly 1 Assembly n

PriorityModel

Priority Band

(2) Verifying the Generated QoS Configurations

Comm<Lanes, PModel>

Ana1<Bands>

Ana2<Bands>

Ana3<Bands>

Flt1<Bands>

Flt2<Bands>

Flt3<Bands>

Gzm1<Bands>

Gzm2<Bands>

Gzm3<Bands>

Depends on

options of dependent component(s) triggers detection of potential mismatches• e.g., dependency between Gizmo invocation priority & Comm lane priority

14

• Leveraging Bogor model checking framework• Dependency structure

maintained in Bogor used to track dependencies between QoS options of components, e.g.:

– Analysis & Comm are connected

– Gizmo & Comm are dependent

• Change(s) in QoS

Detect mismatch if either values change


15

• Representation of middleware QoS options in Bogor model-checker

• BIR extensions allow representing domain-level concepts in a system model

• QUICKER defines new BIR extensions for QoS options

– Allows representing QoS options & domain entities directly in a Bogor input model

– e.g., CCM components, Real-time CORBA lanes/bands are first-class Bogor data types

• Reduces size of system model by avoiding multiple low-level variables to represent domain concepts & QoS options


16

• Representation of properties (that a system should satisfy) in Bogor

• BIR primitives define language constructs to access & manipulate domain-level data types, e.g.:

– Used to define rules that validate QoS options & check if property is satisfied

• Automatic generation of BIR of DRE system from QUICKER-generated output models

Model interpreters auto-generate Bogor Input Representation of a system from its model


of correctness of QUICKER’s QoS configuration

process• Verified the generated QoS configurations through model-checking

• QUICKER toolchain provides

• QoS requirements modeling languages

• QoS mapping algorithms for mapping requirements to middleware QoS options

• We discussed verification



DRE Developer


Evolution

System-level QoS

requirements


QoS Configurations

Metamodel

Validation & Verification

ComponentS2

S3S4

S1

ComponentS2

S3S4

S1

ComponentS2

S3S4

S1

ComponentS2

S3S4

S1

QoS Configurations

Metamodel

Target Model

instanceof


G G’ G G’



Source Model

instanceofQoS Policy Modeling

request/response

publish/subscribe


reliable messaging

QoS Mapping


20

Concluding Remarks

QUICKER can be downloaded from www.dre.vanderbilt.edu/CoSMIC/

• Verified the correctness of QoS Mapping Algorithms through structural correspondence• Empirically validated the configurations by applying the QUICKER process to

representative DRE system case study• Future work based on capturing variability in requirements & middleware

Questions?

21

1-5 10-20 21-100

BACKUP

deve

lops

QoS Design

Multi-service Levels

Preserve invocn. priority

Event delivery on-demand

Event Filtering


Component

Impl Impl Impl


deploys

Application Specification

Platform Specification

Overview of QUICKER: Specifying QoS Requirements• Challenge 1. QoS

requirements specification

• DRE developers are domain experts who understand domain-level issues, system QoS specification must be expressible at the same level of abstraction

23

24

• Challenge 1. QoS requirements specification

• DRE developers are domain experts who understand domain-level issues, system QoS specification must be expressible at the same level of abstraction

• Large gap between what is required (by the application) and how it can be achieved (by the middleware platform)

• Configurations can not be reused; difficult to scale to large-scale systems

deve

lops

QoS Design

Multi-service Levels

Preserve invocn. priority

Event delivery on-demand

Event Filtering


Component

Impl Impl Impl


deploys

Application Specification

Platform Specification

Overview of QUICKER: Specifying QoS Requirements

25

• Application requirements are expressed as QUICKER QoS policy models

• QUICKER captures policies in a platform-independent manner

– Specifying QoS is tantamount to answering questions about application; rather than using low-level mechanisms (such as type of publisher proxy collection, event dispatching mechanism etc.) to achieve QoS

Require support for request buffering.

Gizmo 1Gizmo 2Gizmo 3

Filter 1Filter 2Filter 3

Analysis 1Analysis 2Analysis 3

Comm Ground

Science Agent

Requests executed at the invocation priority.

Support varying service levels for clients.

Requires concurrency support in order to handle all Gizmo requests.


• Representation at multiple levels of granularity– e.g., component- or assembly-level

26

• Application requirements are expressed as QUICKER QoS policy models

• QUICKER captures policies in a platform-independent manner

– Specifying QoS is tantamount to answering questions about application; rather than using low-level mechanisms (such as type of publisher proxy collection, event dispatching mechanism etc.) to achieve QoS

Requirement

OutEventPort

InEventPort

RequiredRequestPort

ProvidedRequestPort

RequestConnectorEventConnector

*

-src0..*

-dst0..*

*

*-src0..1

-dst0..*

*

Component

ComponentAssembly

1

0..*

1

0..*

1

0..*

1

0..*

1

0..*

-src*

-dst1

-src*

-dst1

-src

*

-dst

1

-src

*

-dst

1


27

• Benefits of QUICKER QoS policy modeling

• QoS policy specifications can be containment inherited, reused

• QoS policy inherited by all contained objects

• More than one connections can share QoS policies

• Scalable, flexible QoS policy models


28

Pub-side filtering Sub-side filteringEvent-type

filtering

Dispatching Scheduling

Event Grouping Pub-proxy collection

Sub-proxy collection

Event TimeoutObserverPub-disconnect

control

Sub-disconnect control

// Get the correct bands from the <server_declared_obj>. policies[0] = server_declared_obj->_get_policy (RTCORBA::PRIORITY_BANDED_CONNECTION_POLICY_TYPE); RTCORBA::PriorityBandedConnectionPolicy_var

bands_policy = RTCORBA::PriorityBandedConnectionPolicy ::_narrow (policies[0]); RTCORBA::PriorityBands_var bands = bands_policy->priority_bands ();

// Set the proper bands at the object level. Note that a new // object is returned. object = client_propagated_obj->_set_policy_overrides(policies,

CORBA::SET_OVERRIDE);

iterator:

COPY_ON_READ

COPY_ON_WRITE

DELAYED

IMMEDIATE

dispatching:

REACTIVE

PRIORITY

MT

scheduling:

NULL

PRIORITY

bands:

low_prio

high_prio

fltrgrp:

DISJUNCTION

CONJUNCTION

LOGICAL_AND

TPool:

stacksize

lane_borrowing

request_bufferring

lanes:

static_thrds

dyna_thrds

• Challenge 2. QoS realization

• Very large configuration space providing high degree of flexibility and configurability

• Semantic compatibility of QoS configurations enforced via low-level mechanisms – tedious, error-prone

• Prune middleware configuration space, instantiate configuration set selected, validate their values

Overview of QUICKER: Realizing System QoS

29

• Mapping application QoS policies onto configuration options using model transformation developed in GReAT

• Semantic translation algorithms specified in terms of input & output languages

– e.g., rules that translate multiple application service requests & service level policies to corresponding QoS options

• Transformation output as system model – allow for further analysis & translation

• Simplifies application development & enhances traceability

PriorityModel

Lane Lane Lane

Pro

vid

er

Ser

vice

Req

ues

t

Provider Service Levels

Level 1

Level 2

Level 3

Multiple Service Requests Service Levels

Priority Model PolicyThread Pool Lanes


30

Challenge 2 Resolved: Realizing System QoS• Algorithm shown is RT-CCM QoS mapping

that uses• Application structural properties to

automatically deduce configurations– Lines 7-16 show thread resource

allocation scheme– Line 27 shows client-side QoS

configurations• QoS policies specified used for the

remaining configurations– Service invocation profile used to

assign thread resources– Line 28 resolves priority dependency

of connected components

# of client components and interface operations used to calculate # of threads required

Comm Ground

Gizmo 1 Filter 1 Analysis 1

Gizmo 2 Filter 2 Analysis 2Science Agent

Gizmo 3 Filter 3 Analysis 3

31

• Challenge 3. QoS options dependency resolution of application sub-systems

• Configurations of connected components may exhibit dependency relationship

– e.g., server-side priority model and client-side priority bands must match

• Manually tracking dependencies between components is hard


Evaluating the Correctness and Effectiveness of a Middleware QoS Configuration Process in DRE Systems Institute for Software Integrated Systems Dept of.

Documents

interface definition

simultaneous qos requirements

local subsystem

dre systemsinstitute

node placementconfiguration

usaamogh kavimandan

vanderbilt universitynashville

right set of parameters