An Overview of the OMG Data Distribution Service (DDS) Dr. Douglas C. Schmidt [email protected] schmidt Vanderbilt University.

An Overview of the OMG Data Distribution Service (DDS)

Dr. Douglas C. Schmidt [email protected]

www.dre.vanderbilt.edu/~schmidt

Vanderbilt University Nashville, Tennessee

Institute for Software Integrated

Systems

Overview of OMG Data Distribution Service (DDS)

High Performance real-time data-centric publish/subscribe middleware

‣ The right data, at the right place, at the right time—all the time

‣ Fully distributed, multicast-enabled, high performance, highly scalable, & high availability, hot-swap/hot-hot architecture

Blends data-centric & real-time publish/subscribe technologies

‣ Content-based subscriptions, (continuous) queries, & filters

‣ Fine-grained, policy-based tuning of resource usage, data delivery, availability QoS

‣ Optimal networking & computing resources usage

Global Data Space

Loosely coupled

‣ Plug & play architecture with dynamic discovery

‣ Time & space decoupling

Open standard

‣ OMG DDS v1.2 latest version www.omg.org/dds

DDS Architectural Elements

Data-Centric Publish-Subscribe (DCPS)– The lower layer APIs apps can use

to exchange topic data with other DDS-enabled apps according to designated QoS policies

Data Local Reconstruction Layer (DLRL)– The upper layer APIs that define

how to build a local object cache so apps can access topic data as if it were local

•DDS spec only defines policies & interfaces between application & service

•Doesn’t address protocols & techniques for different actors implementing the service

•Doesn’t address management of internal DDS resources

DDS Application Architecture

DCPS

Application

DLRL

Application

DLRL

Application

DLRL

Application

DLRL

Communication

The Application

DDS Domains & Domain Participants

• The domain is the basic construct used to bind individual applications together for communication• Like a VPN

Key DCPS Entities

DCPS Entities include

‣Topics

‣Typed data

‣Publishers

‣Contain DataWriters

‣Subscribers

‣Contain DataReaders

‣DomainParticipants

‣Entry points

• Data can be accessed in two ways

– Wait-based (synchronous calls)

– Listener-based (asynchronous callbacks)

• Sophisticated support for filtering

– e.g., Topic, Content-FilteredTopic, or MultiTopic

• Configurable via (many) QoS policies

DDS: Foundational Abstractions

‣ Information Model. Defines the structure, relations, & QoS, of the information exchanged by the applications, & supports flat, relational, & object-oriented modeling

‣ Typed Global Data Space. A logical data space in which applications read & write data anonymously & asynchronously, decoupled in space & time

‣ Publisher/Subscriber. Produce/Consume information into/from the Global Data Space

‣ QoS. Regulates the non-functional properties of information in the Global Data Space, e.g., reliability, availability, & timeliness, etc.

Global Data Space

Flat Modeling

‣ Modeling. Items distributed within the system are modeled as Topics

‣ Topics. Defined as associations between a data type & a set of QoS & identified by a unique name

‣ Data Types. The data type associated to a Topic is specified by means of IDL

‣ Subscriptions. Topics, Content-Filtered Topics, or Multi-Topics (join between topics) can be used to specify subscriptions. SQL expression are used to specify filters & joins

struct StockQuote { string symbol; string name; sting exchange; float quointe; string xml_extra;};

How do I define the data & subscribe to it?

Relational Modeling

‣ Modeling. As in a Relational database, a DDS information model can be represented by means of Entity Relationship (ER) diagrams

‣ Topics. The entities, represented by means of Topics, are in turns an association between a data type & a set of QoS & identified by a unique name (like tables in an RDBMS)

‣ Data Types. The data type associated to a Topic must be a structured type expressed in IDL

‣ Instances. Key values in a datatype uniquely identify an instance (like rows in table)

‣ Correlation. SQL Expressions can be used to correlate information by means of key values

struct StockQuote { string symbol; string name; sting exchange; float quote; string xml_extra;};#pragma keylist StockQuote symbol

How do I define the data & subscribe to it?

Object-Oriented Modeling

DDS supports object-oriented distributed information modeling providing:

‣ Reduced Complexity & Improved Productivity‣ Focus on the architecture & business logic, while hiding away the details

involved with the diffusion of shared objects state

‣ Encapsulation‣ Attributes are only accessible through dedicated getter/setter operations,

i.e., don’t need to the messaging middleware or the application to have privileged access to business objects representation

‣ Local Operations‣ Besides getters/setters, all other kind of manipulations can be done using

custom operations

‣ Inheritance‣ Single inheritance supported for Data Local Reconstruction Layer (DLRL)

Objects

‣ Navigable Relationships‣ Single Relationships

‣ Multi Relationships (Set, Map, List)

Unleashing the power of Objects...

RelationaRelationall

Object OrientedObject Oriented

Relational ↔ OO Mapping

OO → Relational Mapping

‣ Middleware can automatically manage the generation & association between the DDS object-oriented model & the relational model

Relational → OO Mapping

‣ The Relational Model can be mapped to an object-oriented model

‣ The mapping is under control of the architect

Unleashing the power of Objects...

DDS: Foundational Abstractions

‣ Information Model. Defines the structure, relations, & QoS, of the information exchanged by the applications, & supports flat, relational, & object-oriented modeling

‣ Typed Global Data Space. A logical data space in which applications read & write data anonymously & asynchronously, decoupled in space & time

‣ Publisher/Subscriber. Produce/Consume information into/from the Global Data Space

‣ QoS. Regulates the non-functional properties of information in the Global Data Space, e.g., reliability, availability, & timeliness, etc.

Global Data Space

Global Data Space

‣ Global Data Space can organized into domains, which in turn can have partitions

‣ Each partition is mapped to an IP Multicast Address

‣ QoS policies control the availability & consistency of data

Partition

IP Multicast per Partition Hardware Filtering Traffic Confinement

Information Segregation Traffic Confinement

How can I organize my data?

DDS: Foundational Abstractions

‣ Information Model. Defines the structure, relations, & QoS, of the information exchanged by the applications, & supports flat, relational, & object-oriented modeling

‣ Typed Global Data Space. A logical data space in which applications read & write data anonymously & asynchronously, decoupled in space & time

‣ Publisher/Subscriber. Produce/Consume information into/from the Global Data Space

‣ QoS. Regulates the non-functional properties of information in the Global Data Space, e.g., reliability, availability, & timeliness, etc.

Global Data Space

Publisher/DataWriter & Subscriber/DataReader

Publisher/DataWriter

‣ Publishers manage dissemination of data samples

‣ Dissemination is driven by QoS policies associated with DataWriter, Publisher, & Topic

‣ A DataWriter is associated with one Publisher & one Topic; it embeds knowledge of dealing with Topic’s Data Type`

Subscriber/DataReader

‣ Subscribers manage reception of data samples

‣ Presentation of data is driven by QoS policies associated with DataReader, Subscriber, & Topic

‣ A DataReader is associated with one Subscriber & one Topic; it embeds knowledge of dealing with Topic’s Data Type

NetworkNetwork

DDS: Foundational Abstractions

‣ Information Model. Defines the structure, relations, & QoS, of the information exchanged by the applications, & supports flat, relational, & object oriented modeling

‣ Typed Global Data Space. A logical data space in which applications read & write data anonymously & asynchronously, decoupled in space & time

‣ Publisher/Subscriber. Produce/Consume information into/from the Global Data Space

‣ QoS. Regulates the non-functional properties of information in the Global Data Space, e.g., reliability, availability, & timeliness, etc.

Global Data Space

DDS QoS Model

‣ QoS policies can be associated with all relevant DDS entities

‣ Some QoS policies are matched based on a Request vs. Offered Model

‣ Publications & Subscriptions match only if the declared & requested QoS are compatible

‣ e.g., it is not possible to match a publisher that delivers data unreliably with a subscriber that requires reliability

How is QoS matched in the System?

QoS Policies

‣ Rich set of QoS policies allows applications to configure several different aspects of data availability, delivery, & timeliness

‣ Applications can use QoS policies to control/optimize network & computing resources

QoS Policy Applicability RxO Modifiable

DURABILITY T, DR, DW Y N Data Availability

DURABILITY SERVICE

T, DW N N

LIFESPAN T, DW - Y

HISTORY T, DR, DW N N

PRESENTATION P, S Y N Data Delivery

RELIABILITY T, DR, DW Y N

PARTITION P, S N Y

DESTINATION ORDER

T, DR, DW Y N

OWNERSHIP T, DR, DW Y N

OWNERSHIP STRENGTH

DW - Y

DEADLINE T, DR, DW Y Y Data Timeliness

LATENCY BUDGET

T, DR, DW Y Y

TRANSPORT PRIORITY

T, DW - Y

TIME BASED FILTER

DR - Y Resources

RESOURCE LIMITS

T, DR, DW N N

USER_DATA DP, DR, DW N Y Configuration

TOPIC_DATA T N Y

GROUP_DATA P, S N Y

What are the key QoS policies supported?

Data Timeliness

Deadline QoS Policy

DEADLINE QoS policy defines the maximum inter-arrival time between data samples

You can’t be later than...

‣ DataWriter indicates that the application commits to write a new value at least once every deadline period

‣ DataReaders are notified by the DDS when the DEADLINE QoS contract is violated

‣ DataWriter & DataReader policies must “match”

Latency Budget QoS Policy

LATENCY_BUDGET QoS policy specifies maximum acceptable delay from time data is written until data is inserted in receiver's application-cache

‣ The default value of the duration is zero indicating that the delay should be minimized

‣ This policy is a hint to DDS, not something that must be monitored or enforced

‣ This QoS policy can be used to implement “batching”

I need to get there in at most...

Transport Priority QoS Policy

TRANSPORT_PRIORITY QoS policy is a hint to DDS infrastructure how to set priority of underlying transport used to send the data

VIP Data, stay clear!

‣ TRANSPORT_PRIORITY can be used to implement priority bands that ensure high priority data preempts lower priority data

‣ With priority bands, priority inversion can be controlled by developers who design/deploys the system

Data Delivery

Reliability QoS Policy

RELIABILITY QoS policy indicates the level of assurance offered by the DDS in delivering data to subscribers via the following values:

‣ Reliable. In steady-state the middleware ensures that all samples in DataWriter history will eventually be delivered to all DataReaders

‣ Best Effort. Indicates that it is acceptable to not retry propagation of any samples

How much effort should be taken to deliver data?

Ownership QoS Policy

OWNERSHIP QoS policy specifies whether multiple DataWriters can write same instance of data & if so how these modifications should be arbitrated, via following values

‣ Shared. Multiple writers are allowed to update the same instance & all the updates are made available to the reader

‣ Exclusive. Indicates that each instance can only be owned by one DataWriter, but owner of an instance can change dynamically due to liveliness changes

‣ Owner selection for exclusive controlled by OWNERSHIP_ STRENGTH QoS policy

Who owns the data?

Ownership Strength QoS Policy

OWNERSHIP_STRENGTH QoS policy specifies the value of “strength” used to arbitrate among DataWriters that attempt to modify same data instance

‣ Data instance are identified by tuple <Topic, Key>

‣ QoS policy applies only if OWNERSHIP is EXCLUSIVE

How strong are you?

Data Availability

Durability QoS Policy

‣ Volatile. No need to keep data instances for late joining data readers

‣ Transient Local. Data instance availability for late joining data reader is tied to the data writer availability

‣ Transient. Data instance availability outlives the data writer

‣ Persistent. Data instance availability outlives system restarts

DURABILITY QoS policy controls data availability wrt late joiners via the following variants

DURABILITY_SERVICE QoS policy provide control over configuration of service that implements transient & persistent durability features

For whom will this data be available?

Lifespan QoS PolicyFor how long will this data be available?

LIFESPAN QoS policy allows applications to control what happens to stale data

‣ It specifies the validity (time) interval for data written by the DataWriter

‣ The default validity interval is infinite

‣ Different values regulate how long samples reside in caches with a DDS implementation

History QoS Policy

HISTORY QoS policy controls whether DDS delivers only most recent value, attempt to deliver all intermediate values, or do something in between, via the following semantics:

‣ Keep Last. The DDS will only attempt to keep the most recent “depth” samples of each instance of data identified by its key

‣ Keep All. The DDS will attempt to keep all the samples of each instance of data identified by its key.

‣ Samples on DataWriter side are kept until delivered to all known subscribers

‣ Samples on DataReader side samples are kept until the application “takes” them

How many data samples should I keep?

Concluding Remarks

‣ Next-generation QoS-enabled information management for tactical applications requires innovations & advances in tools & platforms

‣ Conventional SOA middleware technologies are poorly suited for these types of systems due to insufficient QoS support

Benefits of DDS

Standard-based pub/sub

Very low latency & predictable data dissemination

Support for >>100K messages/sec

Stability under overload condition

Scalability

Fairness

Control over latency/throughput tradeoffs

Traffic Engineering Hardware Filtering

Traffic Shaping

Priority driven delivery

High Performance Persistence

Event Processing