Page 1
Your systems. Working as one.
Prerequisites of network centric intelligence:
Data Distribution Bus
Intelligence Workshop, Rome, May 2012Gerardo Pardo‐Castellote, Ph.D. [[email protected] ] CTO, Real‐Time Innovations, Inc. [www.rti.com]Co‐author of DDS specification Co‐chair of the OMG Data‐Distribution SIG
Page 2
Enhance interoperability, reduce system costs and increase
capability via data‐centric system integration
© 2012 RTI • ALL RIGHTS RESERVED 2
DATA ‐> INFORMATION ‐> KNOWLEDGE + INTELLIGENCE
Page 3
RTI: Global leader in DDS
• Over 70% worldwide embedded messaging middleware market share
• First with…– DDS API (2004)– RTPS interoperability protocol (2007)
• Active in OMG standardization– Board of Directors member
– Co‐chair DDS SIG– Chair DDS standard revision committees
• Most mature solution– 15+ years of commercial availability
– Diverse range of industries: defense, finance, medical, industrial control, power
generation, communications
– 500+ commercial customers, 100+ research projects
– 350,000+ licensed copies
3© 2012 RTI • ALL RIGHTS RESERVED
Page 4
4
Challenge: More Data, More Speed, More
SourcesTRENDS:• Growing Information Volume• Lowering Decision Latency• Increasing System Availability• Accelerating technology insertion
and deployment
Next‐generation systems
needs:
• Scalability• Robustness & Availability• Performance• Security• Integration & Evolution• Interoperability
4© 2012 RTI • ALL RIGHTS RESERVED
Page 5
Integration time & cost
High
Low
Small LargeSystem Scale
The Key Challenge: Integration
Page 6
Interoperability
• Operational– The ability of systems, units, or forces to provide
services to and accept services from other systems, units, or forces, and to use the services so exchanged
to enable them to operate effectively together. (DoD Joint Publication 1‐02)
• Software System– The ability of software systems to exchange
information without loss or change, and to use the exchanged information to produce a useful result.
6© 2012 RTI • ALL RIGHTS RESERVED
Page 7
Why Do We Care?• Interoperability is a force multiplier
– System of Systems capability provide
greatly increased
effectiveness
beyond the "sum of the parts" (individual
systems and technologies)• Operator’s perspective:
– Interoperability allows use of costly
systems at their full
potential• Taxpayer perspective:
– Interoperability allows us to pay once for a capability, vs. many times, and opens the market for multiple
component providers.– Interoperability significantly reduces the largest portion of
total ownership cost ‐
operations and support.
© 2012 RTI • ALL RIGHTS RESERVED
Page 8
State Of Practice
• Recent studies have shown a growth in interoperability policy issuance in DoD
– Thousands of pages of directives, instructions, and mandates
– Numerous standards and architecture bodies in the DoD• Weak Correlation between Increased Interoperability
and Standards– Standards are necessary, but not sufficient for
interoperability• Conventional means of developing platform and
systems are complex, manpower intensive, and time consuming.
– Achieving Interoperability can increase complexity
© 2012 RTI • ALL RIGHTS RESERVED
Page 9
Approaches to Software Integration
AppApp
AppApp
AppAppAppApp
AppApp
AppApp Point‐to‐pointPoint‐to‐point
Page 10
Approaches to Software Integration
AppApp
AppApp
AppAppAppApp
AppApp
AppApp Point‐to‐pointPoint‐to‐point
AppApp
AppApp
AppAppAppApp
AppApp
AppApp Server/Broker/ESB
Server/Broker/ESB
Page 11
Approaches to Software Integration
DDS Data‐Centric BusDDS Data‐Centric Bus
AppApp AppApp AppApp AppApp AppApp AppApp
AppApp
AppApp
AppAppAppApp
AppApp
AppApp Point‐to‐pointPoint‐to‐point
AppApp
AppApp
AppAppAppApp
AppApp
AppApp Server/Broker/ESB
Server/Broker/ESB
Page 12
Levels of Conceptual Interoperability (LCIM)
Level 0No Interoperability
Level 6Conceptual Interoperability
Level 5Dynamic Interoperability
Level 4Pragmatic Interoperability
Level 3Semantic Interoperability
Level 2Syntactic Interoperability
Level 1Technical Interoperability
Stand alone systems that have no interoperability
Full assumptions and constraints of meaningful abstraction of
reality. Fully specified but independent model
Maintains state changes between systems during run time.
Includes assumptions and constraints that effect data interchange
Systems are aware of methods & procedures of other systems.
Context is understood by all participating systems
Meaning of data is exchanged through use of a common
information model. The meaning of information is shared and
unambiguously defined.
Common structure or common data format for exchanging
information. The format of the information exchange is
unambiguously defined
Communication protocol for exchanging data. Bits & Bytes are
exchanged in an unambiguous manner
Tradition
alMiddlew
are
Data‐Ce
ntric
Middlew
are
Page 13
Copyright © 2010 RTI ‐
All rights Reserved. . 13
Data‐Centric Middleware allows applications to be integrated to the Information Model
APP APP APP APP
DDS Global Data Space
DataModel
Standard Mapping(*)
Standard API
No custom mappings / code necessaryDirect support for data‐centric actions: create, dispose, read/take
Page 14
Everyday Example: Calendaring
Alternative Process #1 (message‐centric):
1.
Email: “Meeting Monday at 10:00.”
2.
Email: “Here’s dial‐in info for meeting…”
3.
Email: “Meeting moved to Tuesday”
4.
You: “Where do I have to be? When?”
5.
You: (sifting through email messages…)
14© 2012 RTI • ALL RIGHTS RESERVED
Page 15
Example: Calendaring
Alternative Process #2:1.
Calendar: (add meeting Monday at 10:00)
2.
Calendar: (add dial‐in info)3.
Calendar: (move meeting to Tuesday)
4.
You: “Where do I have to be? When?”5.
You: (check calendar. Contains
consolidated‐state)
The difference is state! The infrastructure consolidates changes and maintains it
15© 2012 RTI • ALL RIGHTS RESERVED
Page 16
DDS: Standards‐based
Data‐Centric Integration
Streaming
DataSensors Events
Real‐TimeApplications
EnterpriseApplications
Actuators
16© 2012 RTI • ALL RIGHTS RESERVED
Page 17
Web‐EnabledDDS
2012
Family of Specifications
17
DDSImplementation
Network / TCP / UDP / IP
App
DDSImplementation
App
DDSImplementation
DDS Spec
2004
DDSInteroperablity
2006
UML Profilefor DDS
2008
DDS forLw
CCM
2009
DDS X‐Types
2010
DDSSecurity
20122010
DDS‐STD‐C++DDS‐JAVA5
App
17© 2012 RTI • ALL RIGHTS RESERVED
Page 18
DDS mandated by key DoD
Programs
• UK Generic Vehicle Architecture– Mandates DDS for vehicle comm.
– Mandates DDS‐RTPS for interop.
• DISR– Mandates DDS for Pub‐Sub API– Mandates DDS‐RTPS for Interop
• Army, OSD– UCS, Unmanned Vehicle Control
• US Navy Open Architecture– Mandates DDS for Pub‐Sub
• SPAWAR NESI– Mandates DDS for Pub‐Sub SOA
18© 2012 RTI • ALL RIGHTS RESERVED
Page 19
RTI Connext
DDS Application Examples
Aegis Weapon SystemLockheed MartinRadar, weapons, displays, C2
B‐1B BomberBoeingC2, communications, weapons
Common Link Integration
Processing (CLIP)
Northrop GrummanStandards‐compliant interface
to legacy and new tactical
data links
Air Force, Navy, B‐1B and B‐52
ScanEagle
UAVBoeing
Sensors, ground station
Advanced Cockpit Ground Control
Station
Predator and SkyWarrior
UASGeneral Atomics
Telemetry data, multiple
workstations
RoboScoutBase10
Internal data bus and link to
communications center
19© 2012 RTI • ALL RIGHTS RESERVED
Page 20
RTI Connext
DDS Application Examples
Full‐immersion simulationNational Highway
Transportation Safety
Authority
Migrated from CORBA,
DCOM for performance
Air‐Traffic ManagementINDRA.Deployed inUK, Germany, SpainStandards, Performance,
Scalability
Industrial ControlSchneider ElectricVxWorks‐based PLCscommunicate via RTI‐DDS
Signal ProcessingPLATH GMBH
RTI supports modular
programming across
product line
Large TelescopesEuropean Southern
Observatory
Performance &
Scalability
1000 mirrors, 1sec loop
Radar SystemsAWACS upgrade
Evolvability,
Mainteinability, and
supportability
20© 2012 RTI • ALL RIGHTS RESERVED
Page 21
RTI Connext
DDS Application Examples
Multi‐ship simulator
FORCE Technology
Controls, simulation
display
Mobile asset tracking
Wi‐Tronix
GPS, operational status
over wireless links
Highway traffic
monitoring
City of Tokyo
Roadway sensors, roadside
kiosks, control center
Driver safety
Volkswagen
vision systems, analysis, driver
information systems
Medical imaging
NMR and MRI
Sensors, RF generators, user
interface, control
computers
Automated trading
Automated Trading Desk (ATD,
now Citigroup)
Market data feed handlers,
pricing engines, algorithmic
trading applications
21© 2012 RTI • ALL RIGHTS RESERVED
Page 22
Data‐Centric Qos‐Aware Pub‐Sub Model
Persistence
ServiceRecording
Service
Virtual, decentralized global data space
CRUD operations22© 2012 RTI • ALL RIGHTS RESERVED
Source (key) Latitude Longitude Altitude
UAV1 37.4 -122.0 500.0
UAV2 40.7 -74.0 250.0
UAV3 50.2 -0.7 2000.0
Page 23
Quality of Service (QoS)
• Aside from the actual data to be delivered, users often need to specify HOW to send it …
…
reliably (or “send and forget”)
… how much data (all data , last 5 samples, every 2 secs)… how long before data is regarded as ‘stale’
and is discarded
… how many publishers of the same data is allowed… how to ‘failover’
if an existing publisher stops sending data
… how to detect “dead”
applications
… …
• These options are controlled by formally‐defined Quality of Service (QoS)
Page 24
Real‐Time Quality of Service (QoS)
QoS PolicyDURABILITY
HISTORY
READER DATA LIFECYCLE
WRITER DATA LIFECYCLE
LIFESPAN
ENTITY FACTORY
RESOURCE LIMITS
RELIABILITY
TIME BASED FILTER
DEADLINE
CONTENT FILTERS
Volatility
User Q
oS
Delivery
PresentationRedundancy
Infrastructure
Transport
QoS Policy
USER DATA
TOPIC DATA
GROUP DATA
PARTITION
PRESENTATION
DESTINATION ORDER
OWNERSHIP
OWNERSHIP STRENGTH
LIVELINESS
LATENCY BUDGET
TRANSPORT PRIORITY
Page 25
Operational robustness and performance
© 2012 RTI • ALL RIGHTS RESERVED 25
Page 26
Architecture for the next‐generation systems
• Existing technologies are reaching robustness/performance/scalability limits
• DDS provides a fundamentally new DataBus
architecture and approach– Powerful data‐centric model
– Ultra‐scalable and robust– Fully decentralized, peer‐to‐peer, “no bottlenecks”
architecture
– Superior Wire Protocol
– Standards‐based, multi‐platform
Single‐lane trafficNo prioritization
Brokers aschoke‐points Connext
DDS Approach
26© 2012 RTI • ALL RIGHTS RESERVED
Page 27
Real‐Time Performance: U.S. Navy Analysis
NESI part 5 v3.0 pg 70
Page 28
Performance
• Reliable multicast• Fully meshed, reliable
Number of Subscribers
Orders of
magnitude faster
than IT solutions
Fastest DDS solution
28© 2012 RTI • ALL RIGHTS RESERVED
Page 29
Scalability
1 ~1000 subscribers, < 15% throughput decrease
600,000
500,000
400,000
300,000
200,000
100,000
0
Messages pe
r Second
Per Subscriber (2
00 Bytes)
0 200 400 600 800
1,000
Number of Subscribers
• Scalable Performance!• Millions of data
elements• .5m updates/sec
(batched)• 10s µs latency• 1000s consumers
per update• Orders of magnitude
more scalable
than IT solutions
29© 2012 RTI • ALL RIGHTS RESERVED
Page 30
Comparison with other technologies
DDS/GSOAP/JMS/Notification Service Comparison - Latency
0
500
1000
1500
2000
2500
4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384
Message Size (bytes)
DDS JMS Notification Service
Message Length (samples)
Adapted from Vanderbilt presentation at July 2006 OMG Workshop on RT Systems30© 2012 RTI • ALL RIGHTS RESERVED
Page 31
Joint Battle Command (Blue Force Tracker): Poor Performance, Lack of Maintainability
Mission:• Track positions of friendly and
hostile forces on the battlefield
• Design goal: 100K tracked
updates/sec
Legacy Capability:• 500K lines of code• 8 yrs to develop• 21 servers• Achieved: 20K tracked updates/sec,
reliability and uptime challenges
“This would not have been possible
with any other known technology.”
—Network Ops Center Technical Lead
Next‐Gen Capability:• 50K lines of code—order of
magnitude less
• 1 yr to develop—8x less
• 1 laptop—20x less
• Achieved: 250K+ tracked updates/sec,
no single point of failure
31© 2012 RTI • ALL RIGHTS RESERVED
Page 32
Conclusions• DDS is a mature international Standard from OMG
– Platform Neutral: Operating systems and Programming
Languages
– Deployed worldwide in Military systems and other
Demanding real‐time applications
• DDS Is mandated by DoD
for Publish‐Subscribe and
data‐distribution applications
• DDS is an ideal integration platform for Intelligent
Systems– Highly Tunable via Quality of Service (QoS)– Rich services (persistence, filtering, high‐availability)
• RTI is the Leading provider of DDS technology & Services
32© 2012 RTI • ALL RIGHTS RESERVED
Page 33
Find out more…
www.rti.com
community.rti.com
demo.rti.com
www.youtube.com/realtimeinnovations
blogs.rti.com
www.twitter.com/RealTimeInnov
www.facebook.com/RTIsoftware
www.slideshare.net/GerardoPardo
dds.omg.org
www.omg.org
©2012 RTI • ALL RIGHTS RESERVED 33
Page 34
Thank You!
34© 2012 RTI • ALL RIGHTS RESERVED