QoS-enabled Middleware for Managing and Controlling High …schmidt/PDF/ORB-control4.pdf · Embedded Middleware Douglas C. Schmidt ACE and TAO Statistics Over 30 person-years of effort

QoS-enabled Middleware for Managingand Controlling High-Speed Networks

Douglas C. SchmidtAssociate Professor & Computer Science Dept.Director of the Center for Washington University, St. LouisDistributed Object computing www.cs.wustl.edu/�schmidt/

SponsorsNSF, DARPA, Bellcore/Telcordia, BBN, Boeing, CDI/GDIS, Hughes,

Kodak, Lockheed, Lucent, Microsoft, Motorola, Nokia, Nortel, OCI, OTI,SAIC, Siemens SCR, Siemens MED, Siemens ZT, Sprint, USENIX

April 13th, 1999

Embedded Middleware Douglas C. Schmidt

Motivation: the High-speed Network Software Crisis

www.arl.wustl.edu/arl/� Symptoms

– Network element hardware getssmaller, faster, cheaper

– Control/management software getslarger, slower, more expensive

� Culprits

– Inherent and accidental complexity

� Solution Approach

– Manage & control network elementsvia QoS-enabled embeddedmiddleware

Washington University, St. Louis 1


General Switch Management Protocol (GSMP)

SIGNALING

CLIENT

SIGNALING PROCESSOR

GSMPSERVER

Middleware

IIOP

ADD_BRANCH

SIGNALING

CLIENT

GSMPSERVER

GSMP Messages

NETWORK ELEMENT

GSMP LIB

Conventional Implementation CORBA Based Implementation

ADD_BRANCH

NETWORK ELEMENT

SIGNALING PROCESSOR

TAOTAO

TAO

Features

� Setup & release connections

� Add & deletepoint-to-multipoint leaves

� Manage ATM switch ports

� Request configurationinformation & statistics

Low-level C APIs send RFC2297 GSMP ATM messages



GSMP Proxy Server Configuration

ATMSwitch

CONTROL PROCESSOR

Network Switch

GSMPSERVER

PROXY

GSMP SERVER

PROXY CONTROL PROCESSOR

SIGNALING

SERVER

SIGNALING PROCCESSOR

TAO - Middlewareconnection request

Inter-ORB Protocol

Establish Connectmethod

ATMSwitch

Network Switch

ATMSwitch

Network Switch

Control cellsGSMP

PROCESSOR

GSMP messageadd_branch

config port

SIGNALING

SERVER

SIGNALING PROCCESSOR

CONTROL PROCESSOR

Features

� Supports standard CORBAprogramming API

� Can use standard ORB

� Transparent to existingGSMP servers

� Scales to distributedconfiguration

– i.e., one CP can controlmultiple switches



GSMP Embedded ORB Configuration

M A N A G E M E N T

AGENT

NETWORK OPERATIONS CENTER

SIGNALING AGENT

SIGNALING PROCESSOR

ATM

Switch

Control Processor

Network Switch

GSMP

server

TAO - Middleware

Host A Host B

Connect Request

add_branchConnect Request

INTER-ORB PROTOCOL

get stat

Features

� Leverages middlewareflexibility andstandardization

� Multiple protocols can besupported

– GSMP in-line bridging,IIOP, etc.

Washington University, St. Louis 4 Em

bedd

edM

iddl

ewar

eD

o

Goa

l:A

pply

Em

bedd

edM

iddl

ewar

eto

Net

wor

kE

lem

entM

anag

men

t&C

ontro

lM

AN

AG

EM

EN

T

CLI

EN

T

RIO

TA

ON

ET

WO

RK

OP

ER

AT

ION

S C

EN

TE

R

SIG

NA

LIN

G

SE

RV

ER

RIO

TA

OS

IGN

AL

ING

PR

OC

ES

SO

R

SIG

NA

LIN

G

SE

RV

ER

RIO

TA

OS

IGN

AL

ING

PR

OC

ES

SO

R

GS

MP

SE

RV

ER

SW

ITC

H C

ON

TR

OL

LE

R

TA

OR

IO

WU

GS

WU

GS

WU

GS

AD

D_B

RA

NC

H

AD

D_B

RA

NC

HG

ET

ST

AT

S

EV

EN

T

PO

RT

DO

WN

CO

NN

EC

T_R

EQ

UE

ST

AD

D_B

RA

NC

H GE

T S

TA

TS

EN

DS

YS

TE

M A

UN

I

CL

IEN

T

TA

OIn

ter-

OR

B P

roto

col

CO

NN

EC

T_R

EQ

UE

ST

EN

DS

YS

TE

M A

UN

I

CL

IEN

TTA

O

AT

MS

WIT

CH

AT

MS

WIT

CH

AT

MS

WIT

CH

GS

MP

SE

RV

ER

SW

ITC

H C

ON

TR

OL

LE

R

TA

OR

IO

GS

MP

SE

RV

ER

SW

ITC

H C

ON

TR

OL

LE

R

TA

OR

IO

Dom

ain

Cha

lleng

es

�

Hig

h-sp

eed

(20

Gbp

s)AT

Msw

itche

sw

/em

bedd

edco

ntro

llers

�

Low

-late

ncy

and

stat

istic

alre

al-t

ime

dead

lines

�

CO

TS

infr

astr

uctu

re,o

pen

syst

ems,

and

smal

lfo

otpr

int

Was

hing

ton

Uni

vers

ity,S

t.Lo

uis


Problem: Lack of QoS-enabled Middleware

LOWLOW--LEVELLEVELMIDDLEWAREMIDDLEWARE

OPERATINGOPERATING

SYSTEMS SYSTEMS &&PROTOCOLSPROTOCOLS

HIGHERHIGHER--LEVELLEVELMIDDLEWAREMIDDLEWARE

COMMONCOMMONSERVICESSERVICES

APPLICATIONSAPPLICATIONS

ConsConsEVENTEVENT

CHANNELCHANNELConsCons

ConsCons

� Many telecom applications requireQoS guarantees

– e.g., call-processing,network/switch management,wireless

� Building these applicationsmanually is hard

� Existing middleware doesn’tsupport QoS effectively

– e.g., CORBA, DCOM, DCE, Java

� Solutions must be integratedhorizontally & vertically



Candidate Solution: CORBA

INTERFACE

REPOSITORY

IMPLEMENTATION

REPOSITORY

IDLCOMPILER

DII ORBINTERFACE

ORBORB CORECORE GIOPGIOP//IIOPIIOP//ESIOPSESIOPS

IDLIDLSTUBSSTUBS

operation()operation()in args

out args + return value

CLIENTOBJECT(SERVANT)

OBJ

REF

STANDARD INTERFACE STANDARD LANGUAGE MAPPING

ORB-SPECIFIC INTERFACE STANDARD PROTOCOL

INTERFACE

REPOSITORY

IMPLEMENTATION

REPOSITORY

IDLCOMPILER

IDLSKELETON

DSI

OBJECT

ADAPTER

www.cs.wustl.edu/�schmidt/corba.html

Goals of CORBA

� Simplify distributionby automating

– Object location &activation

– Parametermarshaling

– Demultiplexing– Error handling

� Provide foundationfor higher-levelservices



Caveat: Limitations of CORBA for QoS

INTERFACE

REPOSITORY

IMPLEMENTATION

REPOSITORY

IDLCOMPILER

DII ORBINTERFACE


IDLIDLSTUBSSTUBS




OBJ

REF



INTERFACE

REPOSITORY

IMPLEMENTATION

REPOSITORY

IDLCOMPILER

IDLSKELETON

DSI

OBJECT

ADAPTER

www.cs.wustl.edu/�schmidt/RT-ORB.ps.gz

Limitations

� Lack of QoSspecifications

� Lack of QoSenforcement

� Lack ofreal-timeprogrammingfeatures

� Lack ofperformanceoptimizations



Overview of the Real-time CORBA Specification

OS KERNEL

OS IOS I//O SUBSYSTEMO SUBSYSTEM

NETWORK ADAPTERSNETWORK ADAPTERS

MUTEXMUTEX

IDLIDL

ENDEND--TOTO--ENDEND

PRIORITYPRIORITY

PROPAGATIONPROPAGATION

ORBORB CORECORE

OBJECTOBJECT

ADAPTERADAPTER

CLIENTCLIENT

GIOPGIOP

OBJECTOBJECT((SERVANTSERVANT))

PROTOCOLPROTOCOL

PROPERTIESPROPERTIES

THREADTHREAD POOLSPOOLS

EXPLICITEXPLICIT

BINDINGBINDING

NETWORKNETWORK

OS KERNELOS KERNEL


NETWORK ADAPTERSNETWORK ADAPTERS

Features

1. End-to-end prioritypropagation

2. Protocol properties

3. Thread pools

4. Explicit binding

5. Mutex IDL



Our Approach: The ACE ORB (TAO)

NETWORKNETWORK

ORBORB RUN RUN--TIMETIME

SCHEDULERSCHEDULER

operation()operation()

IDLIDLSTUBSSTUBS

IDLIDLSKELETONSKELETON

in argsin args

out args + return valueout args + return value

CLIENTCLIENT

OS KERNELOS KERNEL

HIGHHIGH--SPEEDSPEED

NETWORK INTERFACENETWORK INTERFACE

REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM


OS KERNELOS KERNEL

HIGHHIGH--SPEEDSPEED


REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM

ACEACE COMPONENTSCOMPONENTS

OBJOBJ

REFREF

REALREAL--TIMETIME ORBORB CORECOREIOPIOP

PLUGGABLEPLUGGABLE

ORBORB & & XPORTXPORT

PROTOCOLSPROTOCOLS

IOPIOPPLUGGABLEPLUGGABLE

ORBORB & & XPORTXPORT

PROTOCOLSPROTOCOLS

REALREAL--TIMETIME

OBJECTOBJECT

ADAPTERADAPTER

www.cs.wustl.edu/�schmidt/TAO.html

TAO Overview !

� An open-source,standards-based,real-time,high-performanceCORBA ORB

� Runs on POSIX,Win32, &embedded RTplatforms– e.g., VxWorks,

Chorus, LynxOS� Leverages ACE



The ADAPTIVE Communication Environment (ACE)

PROCESSES//THREADSTHREADS

DYNAMICDYNAMIC

LINKINGLINKING

MEMORYMEMORY

MAPPINGMAPPING

SELECTSELECT//IO COMPIO COMP

SYSTEMSYSTEM

VV IPCIPCSTREAMSTREAM

PIPESPIPES

NAMEDNAMED

PIPESPIPES

CAPISS

SOCKETSSOCKETS//TLITLI

COMMUNICATIONCOMMUNICATION

SUBSYSTEMSUBSYSTEM

VIRTUAL MEMORYVIRTUAL MEMORY

SUBSYSTEMSUBSYSTEM

GENERAL POSIX AND WIN32 SERVICES

PROCESSPROCESS//THREADTHREAD

SUBSYSTEMSUBSYSTEM

FRAMEWORKS ACCEPTORACCEPTOR CONNECTORCONNECTOR

SELF-CONTAINED

DISTRIBUTED

SERVICE

COMPONENTS

NAMENAME

SERVERSERVER

TOKENTOKEN

SERVERSERVER

LOGGINGLOGGING

SERVERSERVER

GATEWAYGATEWAY

SERVERSERVER

SOCKSOCK__SAPSAP//TLITLI__SAPSAP

FIFOFIFO

SAPSAP

LOGLOG

MSGMSG

SERVICESERVICE

HANDLERHANDLER

TIMETIME

SERVERSERVER

C++WRAPPER

FACADES

SPIPESPIPE

SAPSAP

CORBACORBA

HANDLERHANDLER

SYSVSYSVWRAPPERSWRAPPERS

SHAREDSHARED

MALLOCMALLOC

THE ACE ORBTHE ACE ORB

((TAOTAO))

JAWS ADAPTIVEJAWS ADAPTIVE

WEB SERVERWEB SERVER

MIDDLEWARE

APPLICATIONS

REACTORREACTOR//PROACTORPROACTOR

PROCESSPROCESS//THREADTHREAD

MANAGERSMANAGERS

STREAMSSTREAMS

SERVICESERVICE

CONFIGCONFIG--URATORURATOR

SYNCHSYNCH

WRAPPERSWRAPPERS

MEMMEM

MAPMAP

OS ADAPTATION LAYER

www.cs.wustl.edu/�schmidt/ACE.html

ACE Overview !

� A concurrentOO networkingframework

� Available inC++ and Java

� Ported toPOSIX, Win32,and RTOSs

Related work !

� x-Kernel

� SysVSTREAMS



ACE and TAO Statistics

� Over 30 person-years of effort

– ACE > 185,000 LOC– TAO > 100,000 LOC– TAO IDL compiler > 100,000 LOC– TAO CORBA Object Services >

150,000 LOC

� Ported to UNIX, Win32, MVS, andRTOS platforms

� Large user community

– www.cs.wustl.edu/�schmidt/ACE-users.html

� Currently used by dozensof companies

– Bellcore, Boeing,Ericsson, Kodak,Lockheed, Lucent,Motorola, Nokia, Nortel,Raytheon, SAIC,Siemens, etc.

� Supported commercially

– ACE !

www.riverace.com– TAO !

www.ociweb.com



Optimization Challenges for QoS-enabled ORBs

ORBORBCORECORE

PLUGGABLEPLUGGABLE

PROTOCOLSPROTOCOLS

PLUGGABLEPLUGGABLE

PROTOCOLSPROTOCOLS

OS KERNELOS KERNEL


NETWORK INTERFACESNETWORK INTERFACES

OS KERNELOS KERNEL



NETWORKNETWORK

ORBORBINTERFACEINTERFACE


IDLIDLSTUBSSTUBS

OBJECTOBJECT

ADAPTERADAPTER


in argsin args


11

22

33

44

55

66

77

1)1) CLIENT MARSHALING CLIENT MARSHALING

2)2) CLIENT PROTOCOL CLIENT PROTOCOL

3)3) NETWORK LATENCY NETWORK LATENCY

4)4) SERVER PROTOCOL SERVER PROTOCOL

5) THREAD DISPATCHING

6) REQUEST DEMUXING

7) OPERATION DEMUXING

8) SERVANT DEMARSHALING


88

CLIENTCLIENT

IOPIOP IOPIOP

Key Challenges

� Alleviate priority inversionand non-determinism

� Reduce demultiplexinglatency/jitter

� Ensure protocol flexibility� Specify QoS requirements

� Schedule operations

� Eliminate (de)marshalingoverhead

� Minimize footprint



Problem: Optimizing Complex Software

DIAGNOSTIC STATIONSDIAGNOSTIC STATIONS

ATMATMMANMAN

ATM

LAN

ATM

LAN

MODALITIES

(CT, MR, CR) CENTRAL

BLOB STORE

CLUSTER

BLOB

STORE

DX

BLOB

STORE

www.cs.wustl.edu/�schmidt/JSAC-99.ps.gz

Common Problems !

� Optimizing complex softwareis hard

� Small “mistakes” can be costly

Solution Approach (Iterative) !

� Pinpoint overhead viawhite-box metrics– e.g., Quantify and

VMEtro

� Apply patterns and frameworkcomponents

� Revalidate via white-box andblack-box metrics



Solution: Patterns and Framework Components

ACCEPTORCONNECTOR

ABSTRACT

FACTORY

SERVANTCLIENT

OS KERNELOS KERNEL

ACTIVE

OBJECT

THREAD-SPECIFIC

STORAGE

SERVICE

CONFIGURATOR

STRATEGY

REACTORREACTOR

WRAPPER FACADESWRAPPER FACADES

www.cs.wustl.edu/�schmidt/ORB-patterns.ps.gz

Definitions

� Pattern

– A solution to a problem ina context

� Framework

– A “semi-complete”application built withcomponents

� Components

– Self-contained, “pluggable”ADTs


Em

bedd

edM

iddl

ewar

eD

o

OR

BO

ptim

izat

ion

Prin

cipl

eP

atte

rns

Defi

nitio

n

�

Opt

imiz

atio

npr

inci

ple

patte

rns

docu

men

tru

les

for

avoi

ding

com

mon

desi

gnan

dim

plem

enta

tion

prob

lem

sth

atca

nde

grad

eth

epe

rfor

man

ce,s

cala

bilit

y,an

dpr

edic

tabi

lity

ofco

mpl

exsy

stem

s

Key

Prin

cipl

eP

atte

rns

Use

din

TAO

#P

rinci

ple

Pat

tern

1O

ptim

ize

for

the

com

mon

case

2R

emov

egr

atui

tous

was

te3

Rep

lace

inef

ficie

ntge

nera

l-pur

pose

func

tions

with

effic

ient

spec

ial-p

urpo

seon

es4

Shi

ftco

mpu

tatio

nin

time,

e.g.

,pre

com

pute

5S

tore

redu

ndan

tsta

teto

spee

d-up

expe

nsiv

eop

erat

ions

6P

ass

hint

sbe

twee

nla

yers

and

com

pone

nts

7D

on’t

betie

dto

refe

renc

eim

plem

enta

tions

/mod

els

8U

seef

ficie

nt/p

redi

ctab

leda

tast

ruct

ures

Was

hing

ton

Uni

vers

ity,S

t.Lo

uis


ORB Latency and Priority Inversion Experiments

��

C 1C 0

Requests

C n

��

Client

...

...

2ATM Switch

Ultra 2 Ultra 2

I/O SUBSYSTEM

Server

Object Adapter

ORB Core

Servants RU

NT

IME

SCH

ED

UL

ER

www.cs.wustl.edu/�schmidt/RT-perf.ps.gz

Method

� Vary ORBs, hold OSconstant

� Solaris real-time threads

� High priority client C0

connects to servant S0

with matching priorities� Clients C1 : : : Cn have

same lower priority

� Clients C1 : : : Cn

connect to servant S1

� Clients invoke twowayCORBA calls that cube anumber on the servantand returns result



ORB Latency and Priority Inversion Results

0

4

8

12

16

20

24

28

32

36

40

44

48

52

1 5 10 15 20 25 30 35 40 45 50

Number of Low Priority Clients

Late

ncy p

er

Tw

o-w

ay R

eq

uest

in M

illiseco

nd

s

CORBAplus High Priority CORBAplus Low Priority

MT-ORBIX High Priority MT-ORBIX Low Priority

miniCOOL High Priority miniCOOL Low Priority

TAO High Priority TAO Low Priority

Synopsis of Results

� TAO’s latency is lowest forlarge # of clients

� TAO avoids priority inversion

– i.e., high priority clientalways has lowest latency

� Primary overhead stemsfrom concurrency andconnection architecture

– e.g., synchronization andcontext switching



ORB Jitter Results

1 5 10 15 20 25 30 35 40 45 50

TAO High Priority

TAO Low Priority

miniCOOL High Priority

miniCOOL Low Priority

MT-ORBIX High Priority

MT-ORBIX Low Priority

CORBAplus High Priority

CORBAplus Low Priority

0

50

100

150

200

250

300

Jitt

er in

mill

isec

on

ds

Number of Low Priority Clients

Definition

� Jitter ! standarddeviation fromaverage latency

Synopsis of Results

� TAO’s jitter islowest and mostconsistent

� CORBAplus’ jitteris highest andmost variable



Problem: Improper ORB Concurrency Models

OBJECTOBJECTADAPTERADAPTER

SERVANTSERVANT DEMUXERDEMUXER

SERVANTSERVANTSKELETONSSKELETONS

U

ORBORB CORECORE

FILTERFILTER

FILTERFILTER

FILTERFILTER

SERVANTSERVANTSKELETONSSKELETONS

U

2: enqueue(data)2: enqueue(data)

3: dequeue,3: dequeue,filterfilter

request,request,&enqueue&enqueue

4: dispatch4: dispatchupcall()upcall()

II//OO SUBSYSTEMSUBSYSTEM

THREADTHREAD

FILTERFILTER

1: recv()1: recv()

CONNECTION THREADSCONNECTION THREADS

Common Problems

� High context switching andsynchronization overhead

� Thread-level andpacket-level priorityinversions

� Lack of application controlover concurrency model

www.cs.wustl.edu/�schmidt/CACM-arch.ps.gz



Problem: ORB Shared Connection Models

SERVERSERVERORBORB CORECORE



COMMUNICATIONCOMMUNICATION LINKLINK


SERVANTSSERVANTS

ONEONE TCPTCP

CONNECTIONCONNECTION

ORBORB CORECORE

SEMAPHORESSEMAPHORES

2: select()2: select()4: release()

3: read()

BO

RR

OW

ED

TH

RE

AD

BO

RR

OW

ED

TH

RE

AD

S

APPLICATION

5: dispatch_return()

1: invoke_twoway()

www.cs.wustl.edu/�schmidt/RTAS-98.ps.gz

Common Problems

� Request-levelpriority inversions

– Sharing multiplepriorities on asingle connection

� Complex connectionmultiplexing

� Synchronizationoverhead



Problem: High Locking Overhead

0

94

199

175

0

231216

599

0

100

200

300

400

500

600

700

TAO miniCOOL CORBAplus MT ORBIX

ORBs Tested

Use

r L

evel

Lo

ck O

per

atio

ns

per

Req

ues

t

client server

Common Problems

� Locking overhead affectslatency and jitter significantly

� Memory managementcommonly involves locking

www.cs.wustl.edu/�schmidt/RTAS-98.ps.gz



Solution: TAO’s ORB Endsystem Architecture

RR

UU

NN

TT

II

MM

EE

S S

CC

HH

EE

DD

UU

LL

EE

RR

HIGH-SPEED NETWORKINTERFACES(e.g., APIC, VME)

ZZ

EE

RR

OO

CC

OO

PP

YY

BB

UU

FF

FF

EE

RR

SSRTRT I/O I/OSUBSYSTEMSUBSYSTEM

RTRT OBJECTOBJECTADAPTERADAPTER

OO(1) (1) REQUESTREQUEST DEMUXERDEMUXER

CLIENTSCLIENTS

STUBSSTUBS

SERVANTSSERVANTS

SKELETONSSKELETONS

U

RTRT ORBORB CORECORE

REACTORREACTOR

((PP11))

REACTORREACTOR

((PP22))

REACTORREACTOR

((PP33))

REACTORREACTOR

((PP44))

SOCKETSOCKET QUEUEQUEUE DEMUXERDEMUXER

PLUGGABLEPLUGGABLE PROTOCOLSPROTOCOLS

Solution Approach !

� Integrate scheduler into ORBendsystem

� Support multiple schedulingstrategies

� Co-schedule threads

Principle Patterns !

� Pass hints, precompute,optimize common case,remove gratuitous waste,store state, don’t be tied toreference implementations &models



Problem: Reducing Demultiplexing Latency

OP

ER

AT

ION1

OP

ER

AT

ION2

OP

ER

AT

IONK

2: DEMUX TO

I/O HANDLE

...

...

...POA1

OS I/O SUBSYSTEM

NETWORK ADAPTERS

SERVANT 1

5: DEMUX TO

SKELETON

6: DISPATCH

OPERATION

1: DEMUX THRU

PROTOCOL STACK

4: DEMUX TO

SERVANT

ORB COREORB CORE

ROOT POA3: DEMUX TO

OBJECT

ADAPTER

POA2 POAN

SERVANT N

...SKEL 1 SKEL 2 SKEL N

SERVANT 2

OS

KERNEL

LAYER

ORB

LAYER

SERVANT

LAYER

Design Challenges

� Minimize demuxing layers

� Provide O(1) operationdemuxing through all layers

� Avoid priority inversions

� Remain CORBA-compliant

www.cs.wustl.edu/�schmidt/POA.ps.gz



Solution: TAO’s Request Demultiplexing Optimizations

.........

.........IDLIDL

SKEL SKEL 22

OBJECT ADAPTER

SERVANT SERVANT 500500

(A)(A) LAYERED DEMUXING LAYERED DEMUXING,,LINEAR SEARCHLINEAR SEARCH

SERVANT SERVANT 11 SERVANT SERVANT 22

IDLIDL

SKEL SKEL 11

OP

ER

AT

ION

OP

ER

AT

ION

100

100

OP

ER

AT

ION

OP

ER

AT

ION22

.........

OP

ER

AT

ION

OP

ER

AT

ION

11

IDLIDL

SKEL NSKEL N

... ..................

SE

RV

AN

TS

ER

VA

NT500::

500::

OP

ER

AT

ION

OP

ER

AT

ION

100

100

SE

RV

AN

TS

ER

VA

NT500::

500::

OP

ER

AT

ION

OP

ER

AT

ION

11

SE

RV

AN

TS

ER

VA

NT1::

1::

OP

ER

AT

ION

OP

ER

AT

ION

100

100

SE

RV

AN

TS

ER

VA

NT1::

1::

OP

ER

AT

ION

OP

ER

AT

ION

22

SE

RV

AN

TS

ER

VA

NT1::

1::

OP

ER

AT

ION

OP

ER

AT

ION

11

OBJECT ADAPTER

.........

.........IDLIDL

SKEL SKEL 22

OBJECT ADAPTER

(C) LAYERED DEMUXING,PERFECT HASHING

SERVANT SERVANT 11 SERVANT SERVANT 22

OP

ER

AT

ION

OP

ER

AT

ION

100

100

OP

ER

AT

ION

OP

ER

AT

ION22

.........

OP

ER

AT

ION

OP

ER

AT

ION

11

IDLIDL

SKEL NSKEL N

search(object key)

hash(operation)

IDLIDL

SKEL SKEL 11

(D) DE-LAYERED ACTIVE DEMUXING

hash(object key)

search(operation)

index(object key/operation)

(B) LAYERED DEMUXING,DYNAMIC HASHING

SERVANT SERVANT 500500

Demuxing

� www.cs.wustl.edu/�schmidt/

fieee tc-97,COOTS-99g.ps.gz

Perfect hashing

� www.cs.wustl.edu/�schmidt/gperf.ps.gz


Em

bedd

edM

iddl

ewar

eD

o

Ser

vant

Dem

ultip

lexi

ngR

esul

ts

110

020

030

040

050

0

0

50

100

150

200

Latency (us)

No

. of

Ob

ject

s

Act

ive

Dem

ux

Dyn

amic

Dem

ux

Line

ar D

emux

Syn

opsi

sof

Res

ults

�

Line

arde

mux

isco

stly

�

Act

ive

dem

uxis

mos

teffi

cien

t&pr

edic

tabl

e

Prin

cipl

es

�

Pre

com

pute

,pas

shi

nts,

use

spec

ial-p

urpo

se&

pred

icta

ble

data

stru

ctur

es

Was

hing

ton

Uni

vers

ity,S

t.Lo

uis


Operation Demultiplexing Results

110

2030

4050

0

5

10

15

20

25

Lat

ency

(u

s)

No. of Methods

Perfect Hashing

Binary Search

Dynamic Hashing

Linear Search

Synopsis of Results !

� Perfect Hashing– Highly predictable– Low-latency

� Others strategiesslower

Principle Patterns !

� Precompute, usepredictable datastructures, removegratuitous waste


Em

bedd

edM

iddl

ewar

eD

o

TAO

Req

uest

Dem

ultip

lexi

ngS

umm

ary

...

...

...

...

...

...

PO

AP

OA

11

SE

RV

AN

T

SE

RV

AN

T 11 O

RB

C

OR

EO

RB

C

OR

E

RO

OT

P

OA

AC

TIV

E

DE

MU

XIN

G

PO

A2

PO

AN

SE

RV

AN

T N

...

SK

EL

1S

KE

L 2

SK

EL

N

SE

RV

AN

T 2

AC

TIV

E

DE

MU

XIN

G

PE

RF

EC

TH

AS

HIN

G

Dem

ultip

lexi

ngS

tage

Abs

olut

eT

ime

(

�

s)1.

Req

uest

pars

ing

22.

PO

Ade

mux

23.

Ser

vant

dem

ux3

4.O

pera

tion

dem

ux2

5.P

aram

eter

dem

arsh

alin

gop

erat

ion

depe

nden

t6.

Use

rup

call

serv

antd

epen

dent

7.R

esul

tsm

arsh

alin

gop

erat

ion

depe

nden

t

Was

hing

ton

Uni

vers

ity,S

t.Lo

uis


Real-time ORB/OS Performance Experiments

[P ]0

[P ]1

SCH

ED

UL

ER

0

RU

NT

IME

[P ]

I/O SUBSYSTEM

Server

0 nS1

nC

��Pentium II

S S0C 1C

...

...

Object AdapterServants

ORB Core

Client

...

[P]

Requests

Priority

...

[P ][P ]1

[P ]

n

n

www.cs.wustl.edu/�schmidt/RT-OS.ps.gz

Method

� Vary OS, hold ORBsconstant

� Single-processor IntelPentium II 450 Mhz, 256Mbytes of RAM

� Client and servant run onthe same machine

� Client Ci connects toservant Si with priority Pi

– i ranges from 1 : : : 50

� Clients invoke twowayCORBA calls that cube anumber on the servant andreturns result



Real-time ORB/OS Performance Results'&

$%

0

500

1000

1500

2000

2500

3000

3500

0 1 2 5 10 15 20 25 30 35 40 45 50

Low Priority Clients

Tw

o-w

ay R

equ

est

Lat

ency

, use

c

Linux

LynxOS

NT

Solaris

VxWorks

'&

$%

0

2000

4000

6000

8000

10000

12000

0 1 2 5 10 15 20 25 30 35 40 45 50


Tw

o-w

ay R

equ

est

Lat

ency

, use

c

Linux

LynxOS

NT

Solaris

VxWorks

High-priority Client Latency Low-priority Clients Latency



Real-time ORB/OS Jitter Results'&

$%

0

2

10

20

30

40

50

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

Tw

o-w

ay J

itte

r, u

sec


LynxOSLinuxSolarisNTVxWorks

'&

$%

0 2

10

20

30

40

50

0

2000

4000

6000

8000

10000

12000

Tw

o-w

ay J

itte

r, u

sec


LynxOSLinuxVxWorksSolarisNT

High-priority Client Jitter Low-priority Clients Jitter


Em

beddedM

iddleware

Douglas

C.S

chmidt

Problem

:H

ard-codedO

RB

Messaging

andTransportP

rotocolsB

OA

RD

B

OA

RD

11

VM

EV

ME

15

53

15

53

BO

AR

D

BO

AR

D 22

�

GIO

P/IIO

Pare

notsufficient,e.g.:

–G

IOP

message

footprintmay

betoo

large–

TC

Placks

necessaryQ

oS–

Legacycom

mitm

entsto

existingprotocols

�

Existing

OR

Bs

don’tsupport“pluggable

protocols”

–T

hism

akesO

RB

sinflexible

andinefficient

Washington

University,S

t.Louis

32

Embedded Middleware Do

One Solution: Hacking GIOP

� GIOP requests include fields that aren’t neededin homogeneous embedded applications

– e.g., GIOP magic #, GIOP version, byte order,request principal, etc.

� TAO’s gioplite option save 15 bytesper-request, yielding these calls-per-second:

Marshaling-enabled Marshaling-disabledmin max avg min max avg

GIOP 2,878 2,937 2,906 2,912 2,976 2,949GIOPlite 2,883 2,978 2,943 2,911 3,003 2,967

� The result is a measureable, but small (2%),improvement in throughput/latency

� Our pluggable protocols framework will all muchgreater decreases in IOP request sizes, as wellas more flexible support for multiple transportprotocols

� However, there will be no changes required tothe standard CORBA programming model

Washington University, St. Louis

Embedded Middleware Do

Better Solution: TAO’sPluggable Protocols Framework

CLIENT

STUBS SKELETONS

TCP

MULTICAST

IOP

VMEUDP

ORB MESSAGING COMPONENT

ORB TRANSPORT ADAPTER COMPONENT

ESIOP

REAL -TIME

IOPEMBEDDED

IOP

RELIABLE,BYTE-STREAM

ATMUDP

OTHER

ORBCORE

SERVICES

COMMUNICATION INFRASTRUCTUREHIGH SPEED NETWORK INTERFACE

REAL -TIME I /O SUBSYSTEM

ORB MESSAGE

FACTORY

ORB TRANSPORT

ADAPTER FACTORY

OBJECT ADAPTER

GIOP GIOPLITE

ADAPTIVE Communication Environment (ACE)

OBJECT (SERVANT)operation (args)IN ARGS

OUT ARGS & RETURN VALUE

RELIABLE,BYTE-STREAM

POLICY

CONTROL

MEMORY

MANAGEMENT

CONCURRENCY

MODEL

Features

� Pluggable ORBmessaging andtransport protocols

� Highly efficient andpredictable behavior

Principle Patterns

� Replacegeneral-purposefunctions(protocols) withspecial-purposeones

Washington University, St. LouisE

mbedded

Middlew

areD

ouglasC

.Schm

idt

CO

RB

AP

rotocolInteroperabilityA

rchitecture

OR

B M

ES

SA

GIN

G

CO

MP

ON

EN

T

OR

B T

RA

NS

PO

RT

AD

AP

TE

R C

OM

PO

NE

NT

TR

AN

SP

OR

T L

AY

ER

NE

TW

OR

K L

AY

ER

GIO

P

IIOP

TC

P

IP

VM

E

DR

IVE

R

AT

M

(AA

L5)A

TM

GIO

PL

ITE

VM

E-IO

P

ES

IOP

AT

M-IO

PR

ELIA

BLE

SE

QU

EN

CE

D

PR

OT

OC

OL C

ON

FIG

UR

AT

ION

S

ST

AN

DA

RD

CO

RB

A P

RO

GR

AM

MIN

G A

PI

Features

!

�

Presentation

layer–

e.g.,CD

R

�

Message

formats

–e.g.,G

IOP

�

Transportassum

ptions–

e.g.,TC

P

�

Objectaddressing

–e.g.,IIO

PIO

R

ww

w.cs.w

ustl.edu/ �

schmidt/pluggable

protocols.ps.gz

Washington

University,S

t.Louis

35


Embedded System Benchmark Configuration

MARSHALMARSHAL

PARAMSPARAMS

OBJECT OBJECT ((SERVANTSERVANT))

OBJECT ADAPTEROBJECT ADAPTER

ACTIVEACTIVE

OBJECTOBJECT

MAPMAP


VME BUSVME BUS

OS KERNELOS KERNEL

VMEVME

DRIVERDRIVER

OS KERNELOS KERNEL

VMEVME

DRIVERDRIVER

IDLIDLSTUBSSTUBS

ORB MESSAGINGORB MESSAGING

ORB TRANSPORTORB TRANSPORT

CLIENTCLIENT

INC

OM

ING

INC

OM

ING

CDRCDR

DATA COPYDATA COPY

DATA COPYDATA COPY

DMADMA COPY COPY



CDRCDR

DATA COPYDATA COPY

DEMARSHALDEMARSHAL

PARAMSPARAMS

DATA COPYDATA COPY

OU

TG

OIN

GO

UT

GO

ING

ORBORBCORECORE

obj->op (params)

VxWorks running on PowerPC over 320 Mbps VME & Ethernet


bedd

edM

iddl

ewar

eD

o

Eth

erne

t&V

ME

Two-

way

Late

ncy

Res

ults

010

0020

0030

0040

0050

0060

0070

0080

0090

00

void

shor

t

octe

t

long

stru

ct

shor

t seq

<oc

tet>

long

seq

<oc

tet>

shor

t seq

<lo

ng>

long

seq

<lo

ng>

Data Type

Lat

ency

(u

sec)

VM

E/G

IOP

lite

avg.

VM

E/G

IOP

avg

.

Eth

erne

t/GIO

Plit

e av

g.

Syn

opsi

sof

Res

ults

�

VM

Epr

otoc

olis

muc

hfa

ster

than

Eth

erne

t

�

No

appl

icat

ion

chan

ges

are

requ

ired

tosu

ppor

tV

ME

Was

hing

ton

Uni

vers

ity,S

t.Lo

uis


Pinpointing ORB Overhead with VMEtro Timeprobes


RECVRECV

OBJECT (SERVANT)

ORBORBCORECORE

OBJECT ADAPTEROBJECT ADAPTER


VME BUSVME BUS

OS KERNELOS KERNEL

VME DRIVERVME DRIVER

OS KERNELOS KERNEL

VME DRIVERVME DRIVER



ACTIVEACTIVE

OBJECTOBJECT

MAPMAP

GET OBJECTGET OBJECT

REFREF

IDLIDLSTUBSSTUBS

MARSHALMARSHAL

PARMATERSPARMATERS


SENDSEND




SENDSEND

II//O SENDO SEND

II//O RECVO RECV77

88


RECVRECV99

CLIENTCLIENT

DEMARSHALDEMARSHAL

PARAMETERSPARAMETERS

7799

II//O RECVO RECV


RECVRECV


RECVRECV

PARSE OBJECTPARSE OBJECT

KEY KEY

OBJECTOBJECT

DEMUXDEMUX

DEMARSHALDEMARSHAL


USER UPCALLUSER UPCALL


SENDSEND


SENDSEND

II//O SENDO SEND

MARSHALMARSHAL


OPERATIONOPERATION

DEMUXDEMUX

131311

22

44

661111

1010

1212

33

88INITIALIZATIONINITIALIZATION

33

55

44

11

22

66

1010

OUTGOINGOUTGOING INCOMINGINCOMING

INC

OM

ING

INC

OM

ING

OU

TG

OIN

GO

UT

GO

ING

POAPOA

DEMUXDEMUX55

� Timeprobes use VMEtro monitor,which measures end-to-end time

� Timeprobe overhead isminimal, i.e., 1 �sec


bedd

edM

iddl

ewar

eD

o

OR

B&

VM

EO

ne-w

ayO

verh

ead

Res

ults

200

200

198

200

203

205

212

226

257

309

51

49

50

50

49

51

50

51

52

56

24

23

23

22

23

23

22

23

23

23

68

66

66

66

66

66

67

66

69

72

0%10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

48

1632

6412

825

651

210

2420

48S

eque

nce

leng

th (

byte

s)

usecs

VM

E +

OS

ove

rhea

dO

RB

clie

nt p

re-w

rite

OR

B c

lient

pos

t-w

rite

OR

B s

erve

r ov

erhe

ad

Syn

opsi

sof

Res

ults

�

OR

Bov

erhe

adis

rela

tivel

yco

nsta

ntan

dlo

w

–e.

g.,�

110

�

secs

per

end-

to-e

ndop

erat

ion

�

Bot

tlene

ckis

VM

Edr

iver

and

OS

,not

OR

B

Was

hing

ton

Uni

vers

ity,S

t.Lo

uis


Client Whitebox Latency ResultsDirection Client Activities Absolute Time ( �s)Outgoing 1. Initialization 36

2. Get object reference 123. Parameter marshal operation dependent4. ORB messaging send 45. ORB transport send 26. I/O send operation dependent

Incoming 7. I/O receive operation dependent8. ORB transport recv 29. ORB messaging recv 1210. Parameter demarshal operation dependent



Server Whitebox Latency ResultsDirection Server Activities Absolute Time ( �s)Incoming 1. I/O receive operation dependent

2. ORB transport recv 103. ORB messaging recv 334. Parsing object key 125. POA demux 36. Servant demux 67. Operation demux 48. Parameter demarshal operation dependent9. User upcall servant dependent

Outgoing 10. Return value marshal operation dependent11. ORB messaging send 3412. ORB transport send 313. I/O send operation dependent



Problem: Overly Large Memory Footprint

INTERFACE

REPOSITORY

IMPLEMENTATION

REPOSITORY

IDLCOMPILER

DII ORBINTERFACE


IDLIDLSTUBSSTUBS




OBJ

REF



INTERFACE

REPOSITORY

IMPLEMENTATION

REPOSITORY

IDLCOMPILER

IDLSKELETON

DSI

OBJECT

ADAPTER

www.cs.wustl.edu/�schmidt/COOTS-99.ps.gz

� Problem

– ORB footprint is toobig for many telecomapps

� UnnecessaryFeatures

– DSI & DII– Dynamic Any– Interface Repository– Advanced POA

features– CORBA/COM

interworking



Solution: Minimum CORBA

Component CORBA Minimum PercentageCORBA Reduction

POA 281,896 207216 26.5ORB Core 347,080 330,304 4.8Dynamic Any 131,305 0 100CDR Interpreter 68,687 68,775 0IDL Compiler 10,488 10,512 0Pluggable Protocols 14,610 14,674 0Default Resources 7,919 7,975 0

Total 861,985 639,456 25.8

Applying Minimum CORBA subsetting to TAO reduces memoryfootprint by �25% and increases ORB determinism



Lessons Learned Developing QoS-enabled ORBs

� Avoid dynamic connection management

� Minimize dynamic memory managementand data copying

� Avoid multiplexing connections fordifferent priority threads

� Avoid complex concurrency models

� Integrate ORB with OS and I/Osubsystem and avoid reimplementingOS mechanisms

� Guide ORB design by empiricalbenchmarks and patterns

'&

$%'

&

$%



Concluding Remarks

� Researchers and developers of distributed, real-time, embeddedtelecom applications confront common challenges

– e.g., service initialization and distribution, error handling, flow control,scheduling, event demultiplexing, concurrency control, persistence, faulttolerance

� Successful researchers and developers apply patterns,frameworks, and components to resolve these challenges

� Careful application of patterns can yield efficient, predictable,scalable, and flexible middleware

– i.e., middleware performance is largely an “implementation detail”

� Next-generation ORBs for telecom will be highly QoS-enabled,though many research challenges remain


Em

bedd

edM

iddl

ewar

eD

o

Cur

rent

Sta

tus

ofTA

O

SC

HE

DU

LIN

GS

CH

ED

UL

ING

PR

OP

ER

TY

PR

OP

ER

TY

EV

EN

TS

EV

EN

TS

TR

AD

ING

TR

AD

ING

NA

MIN

GN

AM

ING

CO

NC

UR

RE

NC

YC

ON

CU

RR

EN

CY

TIM

ET

IME

AA//V

S

TR

EA

MIN

GV

S

TR

EA

MIN

G

NE

TW

OR

KN

ET

WO

RK

OR

BO

RB

Q

OS

INT

ER

FA

CE

RID

LS

TU

BS

RE

AL-T

IME

OB

JE

CT

OB

JE

CT

AD

AP

TE

RA

DA

PT

ER

RID

LR

IDL

SK

EL

ET

ON

SK

EL

ET

ON

CL

IEN

TC

LIE

NT

OS

K

ER

NE

LO

S K

ER

NE

L

OS

I

OS

I//

O

SU

BS

YS

TE

MO

S

UB

SY

ST

EM

NE

TW

OR

K A

DA

PT

ER

SN

ET

WO

RK

A

DA

PT

ER

S

OS

K

ER

NE

LO

S K

ER

NE

L

OS

I

OS

I//

O

SU

BS

YS

TE

MO

S

UB

SY

ST

EM

NE

TW

OR

K A

DA

PT

ER

SN

ET

WO

RK

A

DA

PT

ER

S

GIO

PG

IOP//R

IOP

RIO

P

OB

JE

CT

OB

JE

CT

((SE

RV

AN

TS

ER

VA

NT))

LO

GG

ING

CORBA SERVICES

IDL

CO

MP

ILE

R

RE

AL-T

IME

OR

B C

OR

E

AC

EC

OM

PO

NE

NT

S

LIF

EC

YC

LE

Was

hing

ton

Uni

vers

ity,S

t.Lo

uis


Summary of TAO Research ProjectCompleted work

� First POA and first deployedreal-time CORBA schedulingservice

� Pluggable protocols framework

� Minimized ORB Core priorityinversion and non-determinism

� Reduced latency via demuxingoptimizations

� Co-submitters on OMG’sreal-time CORBA spec

Ongoing work

� Dynamic/hybrid schedulingof CORBA operations

� Distributed QoS, ATM I/OSubsystem, & opensignaling

� Implement Real-timeCORBA spec

� Tech. transfer via DARPAQuorum program andwww.ociweb.com



Summary: Real-time Optimizations in TAO

NETWORKNETWORK

OS KERNELOS KERNEL



ORBORBINTERFACEINTERFACE

ORBORBCORECORE


IDLIDLSTUBSSTUBS

OBJECTOBJECT

ADAPTERADAPTER


in argsin args


CLIENT

GIOP

OBJECT(SERVANT)

CONCURRENCY

MODELS

TRANSPORT

PROTOCOLS

I/OSUBSYSTEM

NETWORK

ADAPTER

PRESENTATION

LAYER

SCHEDULING,DEMUXING, &DISPATCHING

DATA COPYING

& MEMORY

ALLOCATION

CONNECTION

MANAGEMENT

OS KERNEL

OS I/O SUBSYSTEM

NETWORK INTERFACES

OBJ

REF



Next Steps: New TAO Features and Optimizations

MUTEXMUTEX

IDLIDL

ENDEND--TOTO--ENDEND

PRIORITYPRIORITY

PROPAGATIONPROPAGATION

ORBORB CORECORE

OBJECTOBJECT

ADAPTERADAPTER

CLIENTCLIENTOBJECTOBJECT((SERVANTSERVANT))

THREADTHREAD

POOLSPOOLS

EXPLICITEXPLICIT

BINDINGBINDING

NETWORKNETWORK

OS KERNELOS KERNEL



OS KERNELOS KERNEL



PLUGGABLEPLUGGABLEPROTOCOLSPROTOCOLS & &

PROPERTIESPROPERTIES

GIOPGIOP GIOPGIOP

� New Features

– Real-time CORBA– Minimum CORBA– CORBA Messaging– Fault tolerance

� New Optimizations

– Startup time– Event Channel– POA hierarchies

www.cs.wustl.edu/�schmidt/TAO-status.html



Next Steps: Integrating TAO with ATM I/O Subsystem

REACTOR

(20 HZ)

REACTOR

(10 HZ)

REACTOR

(5 HZ)

REACTOR

(1 HZ)

ORB CORE

APIC ATM DRIVER

I/OSUBSYSTEM

PER-VC

SOCKET

QUEUES

APIC PACING

QUEUES

Z

E

R

O

C

O

P

Y

B

U

F

F

E

R

S

�schmidt/RIO.ps.gz

Key Features

� Vertical integration of QoSthrough ORB, OS, and ATMnetwork

� Real-time I/O enhancementsto Solaris kernel

� Provides rate-based QoSend-to-end

� Leverages APIC features forcell pacing and zero-copybuffering



Next Steps: Strategized Scheduling Framework ssttrruucctt RRTT__IInnffoo {{ wwcc__eexxeecc__ttiimmee__;; ppeerr iioodd__;; ccrr iittiiccaalliittyy__;; iimmppoorr ttaannccee__;; ddeeppeennddeenncciieess__;; }};;

SSCCHHEEDDUULLIINNGG

SSTTRRAATTEEGGYY

RRTT__IINNFFOO

RREEPPOOSSII TTOORRYY

OOFFFF--LL II NNEE

SSTTRRAATTEEGGIIZZEEDD

SSCCHHEEDDUULLEERR

OOFFFF--LLIINNEE

OONN--LLIINNEE

RRTT__IINNFFOO

RREEPPOOSSII TTOORRYY

RRUUNN--TTIIMMEE

SSCCHHEEDDUULLEERR

11.. SSPPEECCIIFFYY RRTT__OOPPEERRAATTIIOONN

CCHHAARRAACCTTEERRIISSTTIICCSS AANNDD

DDEEPPEENNDDEENNCCIIEESS

55.. AASSSSEESSSS SSCCHHEEDDUULLAABBIILL IITTYY

33.. AASSSSII GGNN SSTTAATTIICC PPRRIIOORRIITTYY AANNDD SSTTAATTIICC SSUUBBPPRRIIOORRIITTYY

44.. MMAAPP SSTTAATTIICC PPRRIIOORRIITTYY,, DDYYNNAAMMII CC SSUUBBPPRRII OORRII TTYY,, AANNDD

SSTTAATTIICC SSUUBBPPRRIIOORRIITTYY II NNTTOO DDIISSPPAATTCCHHIINNGG PPRRIIOORRIITTYY

AANNDD DDIISSPPAATTCCHHIINNGG SSUUBBPPRRIIOORRIITTYY

66.. AASSSSIIGGNN DDIISSPPAATTCCHHIINNGG QQUUEEUUEE CCOONNFFIIGGUURRAATTIIOONN

22.. PPOOPPUULLAATTEE

RRTT__IINNFFOO

RREEPPOOSSIITTOORRYY

77.. SSUUPPPPLLYY DDIISSPPAATTCCHHIINNGG QQUUEEUUEE

CCOONNFFIIGGUURRAATTIIOONN TTOO TTHHEE OORRBB

RRTTOOppeerraattiioonn



II //OO SSUUBBSSYYSSTTEEMM

OORRBB CCOORREE

OOBBJJEECCTT AADDAAPPTTEERR

99.. SSUUPPPPLLYY SSTTAATTIICC PPOORRTTIIOONNSS OOFF

DDIISSPPAATTCCHHIINNGG PPRRIIOORRIITTYY AANNDD

DDIISSPPAATTCCHHIINNGG SSUUBBPPRRIIOORRIITTYY

TTOO TTHHEE OORRBB

((SSCCHHEEDDUULLEERR''SS OOUUTTPPUUTT

IINNTTEERRFFAACCEE))

((SSCCHHEEDDUULLEERR''SS IINNPPUUTT

IINNTTEERRFFAACCEE))

88.. CCOONNFFIIGGUURREE QQUUEEUUEESS BBAASSEEDD

OONN DDIISSPPAATTCCHHIINNGG QQUUEEUUEE

CCOONNFFIIGGUURRAATTIIOONN

1100.. DDYYNNAAMMIICC QQUUEEUUEESS AASSSSIIGGNN

DDYYNNAAMMIICC PPOORRTTIIOONNSS OOFF

DDIISSPPAATTCCHHIINNGG SSUUBBPPRRIIOORRIITTYY

((AANNDD PPOOSSSSIIBBLLYY

DDIISSPPAATTCCHHIINNGG PPRRIIOORRIITTYY)) OORRBB EENNDDSSYYSSTTEEMM

www.cs.wustl.edu/�schmidt/dynamic.ps.gz



Next Steps: Open ATM Signaling & Control

MANAGE

MIBTAO

RIO

APIC

NETWORK OPERATIONS

CENTER - ENDSYSTEM

GET STATSRECONFIG

SWITCH STATUS

CHANGE

RIO

APIC

TAO objB

objA

LOW LATENCY

CONTROLHIGH - BANDWIDTH

VIDEO/AUDIO DATA

ENDSYSTEM A

ENDSYSTEM B

HIGH - BANDWIDTH

BULK DATA

ENDSYSTEM C

HIGH -BANDWIDTH

VIDEO/AUDIO

MULTICAST

NETWORK SERVICE PROCESSOR

SPC

WUGS

SPC

SPC

SPC

SPC

SPC

SC

CLASSIFIER /ROUTER

ACTIVE NETWORK

RESOURCE CONTROLLER

NATIVE ATM BYPASSES THROUGH APIC

ATM PORT INTERCONNECT CONTROLLER (APIC)

USER

KERNEL

NSP

HIGH -PERFORMANCE

NETWORK ELEMENT

WEIGHTED

FAIR

QUEUING

IP FLOW

CLASSIFICATION

AND ROUTING

TAO

WUGS

OBJECT

SIGNALING

SERVER

TAO

RESOURE

MANAGER

SWITCH

MANAGER

SWITCH CONTROLLER

HIGH -PERFORMANCE NETWORK ELEMENT

QOSMANAGER

TAO

PORT

CONTROLLER

SMART PORT CARD


bedd

edM

iddl

ewar

eD

o

Nex

tSte

ps:

Aud

io/V

ideo

Str

eam

ing

End

-poi

ntFl

ow d

ata

Ada

ptor

Stre

am

(Sou

rce)

Flow

dat

a

Ada

ptor

Obj

ects

And

Man

agem

ent

Obj

ect

Con

trol

Inte

rfac

e

OR

B C

OR

E

POA

End

-poi

nt

Stre

am

Con

trol

Inte

rfac

e

Obj

ect

Stre

am

Stre

am POA

Con

trol

(Sin

k)

MU

LT

IME

DIA

STR

EA

M

ww

w.c

s.w

ustl.

edu/

�

schm

idt/a

v.ps

.gz

Effi

cien

cy

�

Soc

kets

for

data

tran

sfer

toge

thig

hpe

rfor

man

ce

Fle

xibi

lity

�

Use

sC

OR

BA

for

cont

rol

mes

sage

san

dpr

oper

ties

Was

hing

ton

Uni

vers

ity,S

t.Lo

uis


Web URLs for Additional Information

� These slides: �schmidt/TAO4.ps.gz

� More information on CORBA: �schmidt/corba.html

� More info on ACE: �schmidt/ACE.html

� More info on TAO: �schmidt/TAO.html

� TAO Event Channel: �schmidt/JSAC-98.ps.gz

� TAO static scheduling: �schmidt/TAO.ps.gz

� TAO dynamic scheduling: �schmidt/dynamic.ps.gz

� ORB Endsystem Architecture: �schmidt/RIO.ps.gz

� Pluggable protocols: �schmidt/pluggable protocols.ps.gz



Web URLs for Additional Information (cont’d)� Performance Measurements:

– Demuxing latency: �schmidt/COOTS-99.ps.gz

– SII throughput: �schmidt/SIGCOMM-96.ps.gz

– DII throughput: �schmidt/GLOBECOM-96.ps.gz

– ORB latency & scalability: �schmidt/ieee tc-97.ps.gz

– IIOP optimizations: �schmidt/JSAC-99.ps.gz

– Concurrency and connection models: �schmidt/RT-perf.ps.gz

– RTOS/ORB benchmarks:

�schmidt/RT-OS.ps.gz

�schmidt/words-99.ps.gz


QoS-enabled Middleware for Managing and Controlling High …schmidt/PDF/ORB-control4.pdf · Embedded Middleware Douglas C. Schmidt ACE and TAO Statistics Over 30 person-years of effort

Documents