Applying a Real-time CORBA ORB for Avionics …schmidt/PDF/TAO-usecases.pdfApplying a Real-time CORBA ORB for Avionics Mission Computing ... The ACE ORB (TAO) NETWORK ORB QOS ... SOCK_SAP

Applying a Real-time CORBA ORB forAvionics Mission Computing

Douglas C. [email protected]

Washington University, St. Louiswww.cs.wustl.edu/�schmidt/TAO4.ps.gz

SponsorsBoeing and CDI/GDIS

Douglas C. Schmidt High-performance, Real-time ORBs

Mission Computing Design Requirements and Forces

� Integrate real-time scheduling/dispatching in ORB and I/O subsystemfor Boeing military aircraft product families

– i.e., Harrier (AV/8b), F-15, and F/A-18

� Provide all applications with real-time capabilities

– Both method-oriented and event-oriented applications

� Meet deterministic and statistical QoS requirements

– i.e., minimize latency, context switching, priority inversion, andnon-determinism

Washington University, St. Louis 1


Motivation for CORBA for Mission Computing

DIIDII ORBORBINTERFACEINTERFACE

ORBORBCORECORE

operation()operation()

OBJECTOBJECT

ADAPTERADAPTER

IDLIDLSKELETONSKELETON

DSIDSI

in argsin args

out args + return valueout args + return value

CLIENTCLIENT

GIOPGIOP//IIOPIIOP

SERVANTSERVANT

STANDARD INTERFACESTANDARD INTERFACE STANDARD LANGUAGESTANDARD LANGUAGE

MAPPINGMAPPING

ORB-ORB-SPECIFIC INTERFACESPECIFIC INTERFACE STANDARD PROTOCOLSTANDARD PROTOCOL

IDLIDLSTUBSSTUBS

www.cs.wustl.edu/�schmidt/corba.html

� Benefits

– Simplify distributionby automating

� Object locationand activation

� Parametermarshaling

� Demultiplexing

� Error handling– Provide foundation

for higher-levelservices



The ACE ORB (TAO)

NETWORKNETWORK

ORBORB QQOOSSINTERFACEINTERFACE

REALREAL--TIMETIMEORBORB CORECORE

operation()operation()

RIDLRIDLSTUBSSTUBS

REALREAL--TIMETIME

OBJECTOBJECT

ADAPTERADAPTER

RIDLRIDLSKELETONSKELETON

in argsin args

out args + return valueout args + return value

CLIENTCLIENT

OS KERNELOS KERNEL

HIGHHIGH--SPEEDSPEED

NETWORK ADAPTERSNETWORK ADAPTERS

REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM

RIOPRIOP

SERVANTSERVANT

OS KERNELOS KERNEL

HIGHHIGH--SPEEDSPEED

NETWORK ADAPTERSNETWORK ADAPTERS

REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM

www.cs.wustl.edu/�schmidt/TAO.html

� TAO Overview

– A real-time,high-performanceORB

– Leverages ACE

� Runs on POSIX,Win32, RTOSs

� Related work

– U. RI, Mitre– QuO at BBN– ARMADA at U.

Mich.



The ADAPTIVE Communication Environment (ACE)

PROCESSESPROCESSES//THREADSTHREADS

DYNAMICDYNAMIC

LINKINGLINKING

MEMORYMEMORY

MAPPINGMAPPING

SELECTSELECT//IO COMPIO COMP

SYSTEMSYSTEM

VV IPCIPCSTREAMSTREAM

PIPESPIPES

NAMEDNAMED

PIPESPIPES

CAPIS

SOCKETSSOCKETS//TLITLI

COMMUNICATIONCOMMUNICATION

SUBSYSTEMSUBSYSTEM

VIRTUAL MEMORYVIRTUAL MEMORY

SUBSYSTEMSUBSYSTEM

GENERAL POSIX AND WIN32 SERVICES

PROCESSPROCESS//THREADTHREAD

SUBSYSTEMSUBSYSTEM

FRAMEWORKS ACCEPTORACCEPTOR CONNECTORCONNECTOR

SELF-CONTAINED

DISTRIBUTED

SERVICE

COMPONENTS

NAMENAME

SERVERSERVER

TOKENTOKEN

SERVERSERVER

LOGGINGLOGGING

SERVERSERVER

GATEWAYGATEWAY

SERVERSERVER

SOCKSOCK__SAPSAP//TLITLI__SAPSAP

FIFOFIFO

SAPSAP

LOGLOG

MSGMSG

SERVICESERVICE

HANDLERHANDLER

TIMETIME

SERVERSERVER

C++WRAPPERS

SPIPESPIPE

SAPSAP

CORBACORBA

HANDLERHANDLER

SYSVSYSVWRAPPERSWRAPPERS

SHAREDSHARED

MALLOCMALLOC

THE ACE ORBTHE ACE ORB

((TAOTAO))

JAWS ADAPTIVEJAWS ADAPTIVE

WEB SERVERWEB SERVER

MIDDLEWARE

APPLICATIONS

REACTORREACTOR//PROACTORPROACTOR

PROCESSPROCESS//THREADTHREAD

MANAGERSMANAGERS

ADAPTIVE SERVICE EXECUTIVE ADAPTIVE SERVICE EXECUTIVE (ASX)(ASX)

SERVICESERVICE

CONFIGCONFIG--URATORURATOR

SYNCHSYNCH

WRAPPERSWRAPPERS

MEMMEM

MAPMAP

OS ADAPTATION LAYER

www.cs.wustl.edu/�schmidt/ACE.html

� ACE Overview– Concurrent OO

networkingframework

– Ported to C++and Java

– Runs on RTOSs,POSIX, andWin32

� Related work– x-Kernel– SysV STREAMS



ACE Statistics

� ACE contain > 135,000 lines of C++

– Over 15 person-years of effort

� Ported to UNIX, Win32, MVS, andembedded platforms

– e.g., VxWorks, LynxOS, pSoS

� Large user community

– www.cs.wustl.edu/�schmidt/ACE-users.html

� Currently used bydozens of companies

– Bellcore, Boeing,Ericsson, Kodak,Lockheed, Lucent,Motorola, SAIC,Siemens, StorTek,etc.

� Supported commercially

– www.riverace.com



Applying TAO to Avionics Mission Computing

OBJECTOBJECT REQUESTREQUEST BROKERBROKER

2:2:PUSH PUSH ((EVENTSEVENTS))

AirAirFrameFrame

SensorSensorproxyproxy

HUDHUD

NavNav



3:3:PUSH PUSH ((EVENTSEVENTS))

1:1: SENSORS SENSORS

GENERATEGENERATE

DATADATA

EVENTEVENT

CHANNELCHANNEL

www.cs.wustl.edu/�schmidt/oopsla.ps.gz

� Domain Challenges

– Periodic deterministic(and some statistical)real-time deadlines

– COTS infrastructure– Open systems

� Related work

– Deng, Liu, and J. Sun ’96– Gopalakrishnan and

Parulkar ’96– Wolfe et al. ’96



TAO’s Real-time ORB Endsystem Architecture

RR

UU

NN

TT

II

MM

EE

S S

CC

HH

EE

DD

UU

LL

EE

RR

SOCKETSOCKET QUEUEQUEUE DEMUXERDEMUXER

ATM PORT INTERFACECONTROLLER (APIC)

ZZ

EE

RR

OO

CC

OO

PP

YY

BB

UU

FF

FF

EE

RR

SS

II//OOSUBSYSTEMSUBSYSTEM

OBJECTOBJECTADAPTERADAPTER

SERVANTSERVANT DEMUXERDEMUXER

CLIENTSCLIENTSSTUBSSTUBS

SERVANTSSERVANTSSKELETONSSKELETONS

U

ORBORB CORECORE

REACTORREACTOR

(20 (20 HZHZ))

REACTORREACTOR

(10 (10 HZHZ))

REACTORREACTOR

(5 (5 HZHZ))

REACTORREACTOR

(1 (1 HZHZ))

QQooSSAPIAPI

� Solution Approach

– Integrate RT dispatcher intoORB endsystem

– Support multiple requestscheduling strategies

� e.g., RMS, EDF, and MUF– Requests ordered across

thread priorities by OSdispatcher

– Requests ordered withinpriorities based on datadependencies andimportance



Server Request Reception Use-case

RR

UU

NN

TT

II

MM

EE

S S

CC

HH

EE

DD

UU

LL

EE

RR II//OO SUBSYSTEMSUBSYSTEM

ORBORB CORECORE


OBJECTOBJECT ADAPTERADAPTER

REACTORREACTOR

(20 (20 HZHZ))

REACTORREACTOR

(10 (10 HZHZ))

REACTORREACTOR

(5 (5 HZHZ))

REACTORREACTOR

(1 (1 HZHZ))

SERVANTSSERVANTS

SKELETONSKELETON

SERVANTSSERVANTS

SKELETONSKELETONSERVANTSSERVANTS

SKELETONSKELETON

SOCKETSOCKET QUEUEQUEUE DEMUXERDEMUXER 1:1: I I//O SUBSYSTEMO SUBSYSTEM

RECEIVES INCOMING RECEIVES INCOMING

CLIENT REQUEST CLIENT REQUEST

2:2: RUN RUN--TIME SCHEDULERTIME SCHEDULER

DETERMINES PRIORITY DETERMINES PRIORITY

OF REQUEST OF REQUEST

3:3: REQUEST QUEUED REQUEST QUEUED

AND DEQUEUED AND DEQUEUED

ACCORDING TO ACCORDING TO

PRIORITY PRIORITY//RATERATE

4:4: REQUEST DEQUEUED REQUEST DEQUEUED

BY THREAD WITH BY THREAD WITH

SUITABLE SUITABLE OSOS PRIORITY PRIORITY

5:5: REQUEST DISPATCHED REQUEST DISPATCHED

TO SERVANT TO SERVANT

� Synopsis

– I/O subsystemuses portnumbers todemuxrequests toqueues andRT threadsper rate group

– A Reactordemuxes/dispatchesrequests foreach rategroup



Event Channel Reception Use-case

PROXYPROXY

SERVANTSERVANTPROXYPROXY

SERVANTSERVANTPROXYPROXY

SERVANTSERVANT

EVENTEVENT CHANNELCHANNEL

SERVANTSSERVANTSSERVANTSSERVANTSSERVANTSSERVANTS

RR

UU

NN

TT

II

MM

EE

S S

CC

HH

EE

DD

UU

LL

EE

RR

II//OO SUBSYSTEMSUBSYSTEM

ORBORB CORECORE


OBJECTOBJECT ADAPTERADAPTER

REACTORREACTOR

(20 (20 HZHZ))

REACTORREACTOR

(10 (10 HZHZ))

REACTORREACTOR

(5 (5 HZHZ))

REACTORREACTOR

(1 (1 HZHZ))

QUEUEQUEUE DEMUXERDEMUXER

1:1: I I//O SUBSYSTEMO SUBSYSTEM

RECEIVES INCOMING RECEIVES INCOMING

CLIENT REQUEST CLIENT REQUEST

2:2: RUN RUN--TIME SCHEDULERTIME SCHEDULER

DETERMINES PRIORITY DETERMINES PRIORITY

OF REQUEST OF REQUEST

3:3: REQUEST QUEUED REQUEST QUEUED

AND DEQUEUED AND DEQUEUED

ACCORDING TO ACCORDING TO

PRIORITY PRIORITY//RATERATE

4:4: REQUEST DEQUEUED REQUEST DEQUEUED

BY THREAD WITH BY THREAD WITH

SUITABLE SUITABLE OSOS PRIORITY PRIORITY

5:5: REQUEST DISPATCHED REQUEST DISPATCHED

TO PROXY SERVANT TO PROXY SERVANT

6:6: EVENT CHANNEL EVENT CHANNEL

COMPUTES EVENT COMPUTES EVENT

IMPORTANCE AND IMPORTANCE AND

DEPENDENCIES AND DEPENDENCIES AND

DISPATCHES TO DISPATCHES TO

CONSUMER SERVANTS CONSUMER SERVANTS

� Synopsis

– Event Channelthreads handleevent importanceand dependencies

– I/O subsystemand ORB Corehandle priorities



ORB Latency and Priority Inversion Experiments

��

C 1C 0

Requests

C n

��

Client

...

...

2ATM Switch

Ultra 2 Ultra 2

I/O SUBSYSTEM

Server

Object Adapter

ORB Core

Servants RU

NT

IME

SCH

ED

UL

ER

www.cs.wustl.edu/�schmidt/RT-perf.ps.gz

� Vary ORBs, hold OSconstant

� Methodology

– 1 high-priority client– 1..n low-priority clients– Server uses

thread-per-priority

� Highest real-timepriority forhigh-priority client

� Lowest real-timepriority forlow-priority clients



ORB Latency and Priority Inversion Results

0

4

8

12

16

20

24

28

32

36

40

44

48

52

1 5 10 15 20 25 30 35 40 45 50

Number of Low Priority Clients

Late

ncy p

er

Tw

o-w

ay R

eq

uest

in M

illiseco

nd

s

CORBAplus High Priority CORBAplus Low Priority

MT-ORBIX High Priority MT-ORBIX Low Priority

miniCOOL High Priority miniCOOL Low Priority

TAO High Priority TAO Low Priority

� Synopsis of results

– TAO’s latency is lowest– TAO avoids priority

inversion

� i.e., high-priority clientalways has lowestlatency

– Overhead stems fromconcurrency andconnection architecture

� e.g., synchronizationand context switching



ORB Jitter Results

1 5 10 15 20 25 30 35 40 45 50

TAO High Priority

TAO Low Priority

miniCOOL High Priority

miniCOOL Low Priority

MT-ORBIX High Priority

MT-ORBIX Low Priority

CORBAplus High Priority

CORBAplus Low Priority

0

50

100

150

200

250

300

Jitt

er in

mill

isec

on

ds


� Definition

– Variance fromaverage latency

� Synopsis ofresults

– TAO’s jitter islowest andmost consistent

– CORBAplus’jitter is highestand mostvariable



User-level and Kernel-level Locking Overhead

41

94

199

175

100

231216

599

0

100

200

300

400

500

600

700

TAO miniCOOL CORBAplus MT ORBIX

ORBs Tested

Use

r L

evel

Lo

ck O

per

atio

ns

per

Req

ues

t

client server

0 0

4 4

0

13

8

14

0

2

4

6

8

10

12

14

16

TAO miniCOOL CORBAplus MT ORBIX

ORBs Tested

Ker

nel

Lev

el L

ock

Op

erat

ion

s p

er R

equ

est

client server

TAO is carefully designed to minimize memory allocation and locking



Real-time OS/ORB Performance Experiments

[P ]0

[P ]1

SCH

ED

UL

ER

0

RU

NT

IME

[P ]

I/O SUBSYSTEM

Server

0 nS1

nC

��Pentium Pro

S S0C 1C

...

...

Object AdapterServants

ORB Core

Client

...

[P]

Requests

Priority

...

[P ]n

[P ]1

[P ]n

www.cs.wustl.edu/�schmidt/RT-OS.ps.gz

� Vary OS, hold ORBsconstant

� Methodology

– 1 high-priority client– 1..n low-priority clients– Server uses

thread-per-priority

� Highest real-timepriority forhigh-priority client

� Lowest real-timepriority forlow-priority clients



Real-time OS/ORB Performance Results

0

1

2

3

4

5

6

7

8

9

10

1 5 10 15 20 25 30 35 40 45 50


Late

ncy p

er

Tw

o-w

ay R

eq

uest

in M

illi

seco

nd

s

Solaris x86 High Priority Solaris x86 Low PriorityLynxOS High Priority LynxOS Low PriorityVxWorks High Priority VxWorks Low PriorityWindows NT High Priority Windows NT Low Priority

� Synopsis of results

– RTOS’s providelowest latency

– RTOS’s minimizepriority inversion

– ORB (TAO) provideslow latency and avoidspriority inversion

� i.e., high priorityclient always haslowest latency



Real-time OS/ORB Jitter Results

15

1015

2025

3035

4045

50

LynxOS High Priority

LynxOS Low Priority

VxWorks High Priority

VxWorks Low Priority

Windows NT High Priority

Windows NT Low PrioritySolaris x86 High Priority

Solaris x86 Low Priority

0

0.2

0.4

0.6

0.8

1

1.2

1.4

Jitt

er in

mill

isec

on

ds


� Definition

– Standarddeviation fromaverage latency

� Synopsis ofresults

– Some RTOS’sprovide lowjitter

– ORB (TAO)doesn’tintroduce jitter



Real-time OS/ORB CPU Utilization Experiments

I/O SUBSYSTEM

��

C

Requests

Client

Utilization

��

Intel x86��

Object AdapterServants R

UN

TIM

E

SCH

ED

UL

ERORB Core

Server

SPARC

Ethernet Switch

www.cs.wustl.edu/�schmidt/RT-OS.ps.gz

� Vary ORBs, hold OSconstant

� Methodology

– 1 client thread– 2 server threads

� 1 thread servicesclient

� 1 thread factorsprime numbers



Real-time OS/ORB CPU Utilization Results

46

20 20

5254

80 80

48

0

10

20

30

40

50

60

70

80

90

Solaris x86 LynxOS VxWorks Windows NT

RTOSs Tested

Per

cen

tag

e o

f U

tiliz

atio

n o

f C

PU

CPU UtilizationIDLE Time � Synopsis of results

– RTOS’s providehighest effectiveutilization

– ORB (TAO)processing uses

�20% of the CPU



Concluding Remarks

� TAO is currently used at Boeing for avionics mission computing

– Initial flight dates are mid-summer 1998

� Extensive benchmarks demonsrate it is possible to meet stringentperformance goals with real-time CORBA

– e.g., for Boeing, target latency for CORBA oneway operations is150 �secs for 100 Mhz PowerPC running over MVME 177 boards

� Technology transfer to commercial vendors via OMG RT SIG andDARPA Quorom program


Applying a Real-time CORBA ORB for Avionics …schmidt/PDF/TAO-usecases.pdfApplying a Real-time CORBA ORB for Avionics Mission Computing ... The ACE ORB (TAO) NETWORK ORB QOS ... SOCK_SAP

Documents