Connext dds 5 1 scale intelligent systems to the internet of things

Your systems. Working as one.

Connext DDS 5.1: Scale Intelligent Systems to the Internet of Things

February 27, 2014

© 2014 REAL-TIME INNOVATIONS, INC. 2

Topics

• Intelligent Systems• Role of Connext DDS• Driver: Internet of Things• New in Connext DDS 5.1• Next Steps

February 27, 2014


Intelligent Systems

February 27, 2014


Connext DDS: Communications Platform forIntelligent Systems

Real-Time Analytics &

ControlOperator HMI

Sensors Actuators

IT / Cloud / SoS Integration

Network / Bus / IPC

February 27, 2014


Connext DDS Professional

Connext DDS Product Family

February 27, 2014

DDS-RTPS Wire Interoperability Protocol

DDS & JMS Libraries

Routing Service

Database Integration

Connext DDS Micro

Connext DDS Cert

Administration

Monitoring

Microsoft Excel

Recording

Replay

Wireshark

Persistence

Logging

Prototyper

General PurposeReal-Time Apps

Remote Apps

Disparate Apps

Adapter

Small Footprint Apps

High Assurance Apps (DO-178C)


Broad Interoperability for Heterogeneous Systems

• Programming languages and environments– C, C++, C#/.NET, Java, Ada– LabVIEW, MATLAB, Simulink, UML

• Operating systems– Windows, Linux, Unix, Mac OS– Embedded, real time, partitioned– Mobile

• Processor families– x86, ARM, PowerPC…– 32- and 64-bit

• Transport types– Shared memory– LAN (incl. multicast)– WAN– Wireless– Low bandwidth

Completely application transparentFebruary 27, 2014


Trend: Internet of Things (IoT)

• Connecting more devices,applications and systems

• To each other,data center,cloud

February 27, 2014


IoT Benefits

Sharing more data across devices and applications for better:• Situational awareness, decision making• Automation, responsiveness, efficiency• Asset utilization

• Patient safety, clinician efficiency• Energy efficiency, reliability

February 27, 2014


IoT in Practice

ActuatorsSensors

Analytics, control HMI

ActuatorsSensors


ActuatorsSensors



Centralized analytics

Decentralized devices, analytics, control

February 27, 2014


IoT Challenges

• Scalability – aggregate data volumes

• Extensibility – incremental evolution

• Adaptability – can’t characterize performance at dev time

• Debugability

ActuatorsSensors


ActuatorsSensors


ActuatorsSensors



February 27, 2014


Introducing Connext DDS Professional 5.1

February 27, 2014


Routing Service Content Filter Propagation

• Reduce bandwidth and CPU utilization in a hierarchical system where DataReaders subscribe to a subset of data

• DataWriter and Routing Service only publish data that is subscribedFebruary 27, 2014

Routing Service

Apps Devices

Routing Service

Apps Devices

Routing Service

Apps Devices

Routing Service


Routing Service – Content Filter PropagationCo

nnex

t DD

S 5.

1Pr

ior t

o 5.

1

February 27, 2014

DataReader

DataWriter

RS1

X = 1

DataReader

X > 5

DataReader

X = 7

DataWriter

Communication direction

1 2

3 4

11 2 3 4

RSNRS2

DataReader

DataWriter

RS1

X = 1

DataReader

X > 5

DataReader

X = 7

DataWriter

Communication direction

1 2

3 4

1

RSN

1

X =1 or X > 5 or X = 7

Filter propagation directionSamplefiltering

Samplefiltering

RS1


Enhanced TCP Transport Scalability usingI/O Completion Ports (IOCP)

Connext DDS 5.0 Connext DDS 5.1

Asynchronous I/O operationusing IOCP API

Windows server

Windows server

clientsclients

clients

clients

clientsclients

clients

clientsclientsclients

clients

clients

clientsclients

clients

clientsclients

clients

clients

clients

clientsclients

clients

clients

clientsclients

clients

clients

clientsclients

clients

clients

Synchronous I/O operationusing Select () API

>1500 clients~ 600 clients

• To enable IOCP simply set the property socket_monitoring_kind in TCP plugin to WINDOWS_IOCP

Supports far more TCP connections in Windows-based deployments

February 27, 2014


Extensible Types (DDS-XTypes)

• Enables dynamic upgrades• Types can evolve while maintaining backward

interoperability• Essential for large and mission-critical systems

FINAL • Can’t be modified or derived• All primitive fields are final

EXTENSIBLE <default> • Type can be extended, but not modified• Extensions can only appear at the end of the

type definitionMUTABLE • Type can be reordered and/or modified

February 27, 2014


Extensible Types

February 27, 2014

@Extensibility(Extenisble_EXTENSIBILITY)struct TrackData { long id; //@ID 1 long x; //@ID 2 long y; //@ID 3};

@Extensibility(Extensible_EXTENSIBILITY)struct TrackData { long id; //@ID 1 long x; //@ID 2 long z; //@ID 4 long y; //@ID 3};

@Extensibility(Extensible_EXTENSIBILITY)struct TrackData { long id; //@ID 1 long x; //@ID 2 long y; //@ID 3};

@Extensibility(Extensible_EXTENSIBILITY)struct TrackData { long id; //@ID 1 long x; //@ID 2 long y; //@ID 3 long z; //@ID 4};

Connext DDS 5.0


@Extensibility(Mutable_EXTENSIBILITY)struct TrackData { long id; //@ID 1 long x; //@ID 2 long y; //@ID 3};

@Extensibility(Mutable_EXTENSIBILITY)struct TrackData { long id; //@ID 1 long x; //@ID 2 long y; //@ID 3 long z; //@ID 4};

Connext DDS 5.1

@Extensibility(Mutable_EXTENSIBILITY)struct TrackData { long id; //@ID 1 long x; //@ID 2 long y; //@ID 3};

@Extensibility(Mutable_EXTENSIBILITY)struct TrackData { long id; //@ID 1 long x; //@ID 2 long z; //@ID 4 long y; //@ID 3};

Note: 4-byte or 12-byte per member extra overhead on the wire.

Mutable Types

February 27, 2014


XTypes: Optional Members

February 27, 2014

@Extensibility(MUTABLE_EXTENSIBILITY)struct TrackData { long id; long x; long y; long z; //@Optional};

typedef struct TrackData { DDS_Long id; DDS_Long x; DDS_Long y; DDS_Long * z;};

NULL value means there is no value because logically the value is not there

C/C++

Bandwidth overhead per memberFINAL/EXTENSIBLE MUTABLE

SET 4/12-byte HEADER + VALUE 4/12-byte HEADER + VALUE

NOT SET 4/12-byte HEADER 0-byte

Supports scenarios in which data type member value is not known or does not exist


Builtin QoS Profile

February 27, 2014

• Facilitate configuration for common scenarios and use cases• Reduce configuration effort vs. setting individual QoS

parameters

• Total of 34 QoS Profiles as part of the core library• Can be extended and expanded as new features developed

and new use cases identified


Builtin QoS Profiles

BASELINE_ROOT

BASELINE_5_0_0

BASELINE

BASELINE_5_1_0

GENERIC_COMMON

GENERIC_MONITORING

GENERIC__CONNEXT_MICRO_COMPATIBILITY

GENERIC_CONNEXT_OTHER

DDS_VENDOR_COMPATIBILITY

Baseline Generic

PERIODIC_DATA

RELIABLE_STREAMING

STREAMING

EVENT

STATUS

ALARM_EVENT

ALARM_STATUS

LAST_VALUE_CACHE

Pattern

Experimental

KEEP_LAST_RELIABLE_TRANSIENT_LOCAL

KEEP_LAST_RELIABLE_TRANSIENT

KEEP_LAST_RELIABLE_PERSISTENT

STRICT_RELIABLE_LARGE_DATA_FAST_FLOW

STRICT_RELIABLE_LARGE_DATA_MEDIUM_FLOW

STRICT_RELIABLE_LARGE_DATA_SLOW_FLOW

KEEP_LAST_RELIABLE_LARGE_DATA_FAST_FLO

W

KEEP_LAST_RELIABLE_LARGE_DATA_MEDIUM_

FLOW

KEEP_LAST_RELIABLE_LARGE_DATA_SLOW_

FLOW

STRICT_RELIABLE

BEST_EFFORT

KEEP_LAST_RELIABLE

STRICT_RELIABLE_HIGH_THROUGHPUT

STRICT_RELIIABLE_LOW_LATENCY

Basic

Durability

PARTICIPANT_LARGE_DATA

PARTICIPANT_LARGE_DATA_MONITORING

STRICT_RELIABLE_LARGE_DATA

KEEP_LAST_RELIABLE_

LARGE_DATA

AUTO_TUNING

Large Data

Performance

Large Data withFlow Control

February 27, 2014


Improved Default Transport Settings

• Improves out-of-box performance• Minimizes need for tuning

• Default QoS profile available to enable backward compatible with 5.0.0

February 27, 2014


Turbo Mode (Experimental)

• As sending rate changes, automatically adjusts QoS to maintain optimal performance

• Improves performance in burst traffic scenario– High throughput at high sending rate– Low latency at slow sending rate

• Supported by new perftest• http://community.rti.com/downloads/rti-connext-dds-perf

ormance-test

February 27, 2014

http://community.rti.com/downloads/rti-connext-dds-performance-test




Auto Throttle (Experimental)

• Improves latency and jitter in high throughput conditions• Automatically adjusts DataWriter send rate and send

window size in reliable communications using protocol feedback:– Send window occupancy to control the send rate– NACK messages to control the send window size

• Tradeoff– May increase CPU usage in sending machine (active wait)– Slightly reduces throughput for small data sizes (<10%)

February 27, 2014


Auto Throttle Performance

0 10000 20000 30000 40000 50000 600000

5000

10000

15000

20000

25000

30000

Auto Throttle disabled

Average Minimum 99%

Message Size (bytes)

Late

ncy

in M

icros

econ

ds

0 10000 20000 30000 40000 50000 600000

5000

10000

15000

20000

25000

30000

Auto Throttle enabled

Average Minimum 99%

Message Size (bytes)

Late

ncy

in M

icros

econ

ds

February 27, 2014


Recording Service: Flexible Field Selection

February 27, 2014

Pre 5.1…

What the user really cares aboutWhat we store by default, apart from user-data

25 columns of metadata



February 27, 2014

2 columns

of metadata

Now (5.1.0) …

What the user really cares aboutWhat we also store in the DB by default



Users can now select what SampleInfo fields to store with their recorded data

February 27, 2014


Admin Console Enhancements

February 27, 2014

Eclipse RCP DDS

Admin Console Infrastructure

System Control

(5.0)

System Awareness

(5.1)

Debugging

(5.1)


Admin Console Features

5.0• Physical View• System Editor• Processes View• Host View• Process Editor• Routing Service Editor• Persistence Editor• Recording Service Editor• Replay Service Editor• XML Editor

5.1• Auto Join• DDS Logical View• Health and Match Analyses• DDS QoS View• Domain Topic• DDS Data Type View• System Log• Resource Charts for Host

and Routing Service

February 27, 2014


Auto Join

February 27, 2014

Domains Joined:

6

8, 6, 80


DDS Logical ViewThis view presents a hierarchical view of the system from a DDS perspective. It shows a list of DDS domains and the topics within them.

Each entity has a badge to represent its health.

February 27, 2014


Health & System HealthThe Health View makes it simple to tell if a system entity is known to have existing problems.

February 27, 2014

9 conditions are evaluated to determine health.


Real-time Match Analyses

February 27, 2014


QoS

February 27, 2014


Topic

February 27, 2014


Topic

February 27, 2014


Topic

February 27, 2014


DDS Data Type View

February 27, 2014

Hierarchical view of a data type used by a DDS DataWriter or DataReader

The equivalent IDL is also available.


System Log

February 27, 2014

Combined log from all applications using Distributed Logger.


Routing Service: Resources Chart

February 27, 2014

Similar view for Host


Extended Process Information

February 27, 2014


Prototyper with LUA (Experimental)

XMLSettings

Prototyper (“Container”)

Lua Engine

DDS

Lua Component

February 27, 2014


CodeGen 2.0 (Experimental)• Reduce code generation time especially when dealing with

large IDL files and large number of IDL files• Ease customizations and extensions to code generation

• Usability– Apache Velocity (VLT) templates are used in place of XSL

• Performance– With small IDL files, performance is 6x faster (server mode)

– With a large IDL file (2,000 structs) CodeGen 2.0 is >10x faster

February 27, 2014


Migrating to 5.1

• No API changes required– Exception: Removed “_exp” suffix from XML

Application Creation APIs• Must regenerate code• Builtin QoS profile provided for 5.0 interoperability• Monitoring and administration topics are not

compatible with 5.0– Now leveraging XTypes for future forward compatibility

February 27, 2014


Connext DDS 5.1 Summary

• 60+ new features and enhancements to improve– Scalability– Extensibility– Adaptability and performance– Usability and debuggability

• New experimental features• 20+ new platforms

February 27, 2014


Next Steps

• Connext DDS Professional 5.1 downloads– Evaluation: http://www.rti.com/downloads/connext.html– Customer portal: http://support.rti.com– Live CD: http://community.rti.com/content/page/downloads

• Blog posts with more details at http://blogs.rti.com• Many resources at http://community.rti.com

– Examples– Case+Code– Forums– Knowledgebase

• New and improved performance test– http://community.rti.com/downloads/rti-connext-dds-performance-test

February 27, 2014

http://www.rti.com/downloads/connext.html



http://support.rti.com/

http://community.rti.com/content/page/downloads



http://blogs.rti.com/

http://community.rti.com/




Thank You!

February 27, 2014


Internet of Things (IoT)• Human speed, seconds+• Centralized• Data center or cloud• Relatively static,

administered• Fast, reliable networks• Big Data at rest

• Physics speed• Microsecondsseconds• Decentralized, disperse• Dynamic, autonomous• Disadvantaged networks• Big Data in motion• Can never go down

IT Systems: BI, ERP, Web Services

Intelligent Systems:Real-Time Monitoring and Control

February 27, 2014

Connext dds 5 1 scale intelligent systems to the internet of things

Documents

long id

realtime innovations

long x

long y

long z

extensible types connext

mutable types connext

iocp apiconnext dds