FMDP - BCC Group International · As a source-agnostic platform, FMDP focuses on cost reduction, transparency and governance while lowering the TCO. It provides an easy, low-risk

BCC Group International

White Paper

Issue 5.2 November 2015

[FMDP]GLOBAL Financial Messaging Data Platform

Open Access to Global Information

Entitlement | Flexibility | Performance | Reliability | Scalability | Resilience | Cost Reduction

Copyright © BCC Group International GmbH & Co. KG 2008 - 2016

This BCC Group document is for BCC Group customers and authorized developers, and is provided for such

information purposes only.

FMDP White Paper | November 2015 Copyright © BCC Group International GmbH & Co. KG 2008 - 2015

2

TABLE OF CONTENTS

1. Preface ............................................................................................................................................ 3

2. Management Summary .................................................................................................................. 3

3. A choice between two unsatisfactory options ............................................................................ 6

4. Overview of FMDP Approach ........................................................................................................ 7

5. FMDP Design Goals ....................................................................................................................... 8

5.1. Dedicated Hardware for Messaging ......................................................................... 8

5.2. Point-to-Point TCP/IP Protocol ................................................................................. 8

5.3. Topic-based Data Delivery ........................................................................................ 9

5.4. Hardware-based Entitlement Firewall (MECS) ......................................................... 9

5.5. Single API ..................................................................................................................10

6. FMDP Components ...................................................................................................................... 11

6.1. Feeds & Caching ......................................................................................................11

6.1.1. Broadcast Feeds .....................................................................................................11

6.1.2. On-Demand Feeds ..................................................................................................11

6.1.3. Contribution / User Publishing ...............................................................................11

6.1.4. Cache .......................................................................................................................11

6.1.5. Message Recorder ..................................................................................................11

6.2. Multi-vendor Entitlement Control Service (MECS) .................................................12

6.3. Vendor-approved Entitlement Control ....................................................................13

6.4. API .............................................................................................................................13

6.5. Testing Tools ............................................................................................................13

7. TREP-RT/EDGE/RMDS Integration ............................................................................................. 14

8. Bloomberg BPipe Feed Handler ................................................................................................. 14

9. Microsoft Excel® Integration (EMIL) .......................................................................................... 14

10. openMAMA Bridge ....................................................................................................................... 14

11. ITRS GENEOS Monitoring ........................................................................................................... 14

12. Cloud capabilities ........................................................................................................................ 14

13. Example FMDP Setup (not showing DR site) ............................................................................ 15

14. BCC Group – FinTech Made in Germany .................................................................................. 15


3

1. PREFACE

This Whitepaper describes the FMDP System and why the financial market in

particular requires a new approach to distributing messages (e.g. market data). It

provides an overview of the current market situation and describes the approach that

BCC Group has chosen to implement a new generation Message Distribution

System, an approach which keeps all the useful features of existing architectures, but

enables greater speed and simplicity.

As a source-agnostic platform, FMDP focuses on cost reduction, transparency and

governance while lowering the TCO. It provides an easy, low-risk migration away

from legacy platforms while offering royalty-free connectivity of feeds and

applications.

FMDP architecture was designed and developed by BCC Group with important

development contributions in the Entitlement arena by Royal Bank of Canada

Capital Markets (RBCCM) supported by Solace Systems. Testing and production

stability contributions (besides RBCCM production tests) were made at the premises

of German DZ BANK during a four-year period (2011-2015) of active production

support by DZ BANK market data engineers and developers who also heavily tested

FMDP Excel Integration (EMIL) and API.

2. MANAGEMENT SUMMARY

Users, specifically traders all over the world, can communicate in real time about their

problems. Most of their systems have not seen any enhancement in flexibility or

speed for over 10 years. They also have ongoing, long discussions with IT when it

comes to communication enhancements e.g. additional symbols, additional fields,

new recipients or sending values out of spreadsheets.

Spreadsheets are a good example of how difficult it is to get all vendor and customer-

owned data into a single application or, worse still, publish proprietary values. It is not

enough that each vendor requires their own Add-In, but some of the Add-Ins even

seem to intentionally conflict with each other. And the data remains in encapsulated

worlds of the transporting vendor although it is non-vendor data.

Market data vendors have been highly successful in binding their customers to their

systems by implementing terminals connected to their communication channels

(including Excel). After all the crises the financial world has undergone, users are

either connected to one communication channel or the other; in rare cases you are in

fact connected to more than one.

Basically the only “professional” communication channel that everybody can use from

their workplace still remains email. It is available to everybody but is not specifically

designed for communicating values, value groups or news in a structured, context-

sensitive way and it is neither smart nor real-time.

While the communication technology world has evolved to offer amazing

communication freedom using mobile devices and the internet, the trader world is

basically the same. Ultra-high speed ultra-high available messaging technology

burned into chips of appliances (e.g. Solace Systems message routers) have in many

cases, taken the place of overpriced and old communication architectures that require


4

operating system licenses for hundreds of servers together with messaging software

implying exorbitant license and maintenance costs.

Not so in the dealing rooms where global financial data distribution has not yet

reached that level of freedom.

Despite the fact that enterprise data feeds from external vendors (if not feeding into

their own platforms) also provide global distribution for proprietary customer data at

very high cost, they are mostly used to feed directly into applications.

Traders are restricted in content and update rate when publishing over a provider

channel and they are still trapped inside their “preferred” provider network together

with the provider’s terminal.

None of them is able to feed into a vendor agnostic platform that would be capable of

distributing everyone’s data altogether in a unified and entitled way to all applications,

including Excel.

FMDP has been developed for these very reasons, specifically to provide trading

room communication freedom.

FMDP development started 2009 with an early production version in 2011 and the

fully functional version was tested with Bloomberg and Thomson Reuters RMDS

Bridged Data in 2012. Today FMDP is a fully tested and stable, ready to use and fast

in implementing Market Data Distribution Platform for all kinds of data. FMDP

simultaneously supports thousands of users and millions of subscriptions.

Combining multiple FMDP Installations in different locations that are interconnected

allows the distribution of data globally without overloading existing communication

lines, because only subscribed data portions from the available set of data from a

distant location will be transferred.

FMDP was intended to meet the following requirements and these have been 100%

achieved:

Open, bank-owned and giving free but entitled access to all market data

providers and applications,

Designed to be able to use vendor-specific entitlement systems if requested

by the vendor or exchange, but also having an open entitlement system for all

data sources and applications that cooperates seamlessly with any vendor

entitlement system,

Designed to distribute high volumes of messages with low latency,

Easily scalable to higher capacity by just adding a piece of hardware,

Distribute and entitle real-time and reference data,

Include all components for 24/7 operation and resilience,

Designed to be “cloudable” and globally deployable.

Small footprint and low maintenance cost


5

Designed as a product, not as a project, in order to achieve street-wide

acceptance and market data community support, incl. market data vendors

FMDP meets these requirements as follows:

FMDP is hardware based (FMDP routers are Solace Systems message

routers). This optimized and high-performance hardware enhances speed by

several orders of magnitude (factor 10-1000) compared with software-based

messaging solutions and reduces the number of servers. Reducing the

number of servers means reducing system complexity and saving software

licenses and production costs.

FMDP enables vendor independence and thus more flexibility by providing a

universal interface (FMDP API) for all feeds and applications.

FMDP allows entitled distribution of any kind of data object, obviously real-

time market data messages, but also any type of serialized object type, thus

providing entitled distribution for real-time data, historical and reference data

in all formats.

As an intrinsic part of FMDP, resilient failover incl. last value caches is

provided as a stand-alone process or built into feed handlers.

FMDP provides entitlement clients for managing and monitoring the system as

well as user-to-data and user-to-application entitlement management.

A message recording service is available for all data sources using modern

NoSQL databases.

FMDP is “Cloud Ready”, which means that all components can also be run in

a private or public cloud if required.

FMDP installations in different locations can be combined into a global,

optimized distribution network.

The universality of the FMDP API offers significant simplifications in

developing market data applications for real-time, reference and historical

data.

An Excel Add-in is available for sophisticated subscription and publishing over

the platform globally.

FMDP component architecture is transparent and easy to install

FMDP architecture was designed and developed by BCC Group with

important development contributions in the entitlement arena by Royal Bank of

Canada Capital Markets (RBCCM) supported by Solace Systems.

FMDP’s Hardware-based, High-Performance Message Distribution Service was

developed to overcome the trend of ever-increasing infrastructure demands by

focusing on introducing new hardware technologies that include street-wide


6

accepted entitlement and the provision of a smooth migration path to the new

platform.

BCC Group today owns and maintains all parts of FMDP including contributions

from RBCCM.

Image 1: Current World of Distribution with Broadcast Backbone in the financial sector

3. A CHOICE BETWEEN TWO UNSATISFACTORY OPTIONS

Message distribution is a massive I/O problem that cannot be solved with faster

processors or more memory. The bottleneck is in getting data to the applications,

while supporting different service levels. In today’s installations, hundreds of servers

with hundreds of processors manage the distribution. This requires considerable

resources such as electricity, rack space and manpower to administer the

infrastructure, together with the enormous expense of the operating system and

messaging software licenses, their updates, customization and maintenance.

Altogether, in-house tailored or standard front, middle and back-office applications

can account for up to 80% of all messages, mostly internal data. Consequently there

are two options:

Buy everything from one supplier to avoid the complexity and cost of

connecting applications to different providers. Homogenous infrastructure

avoids multiple bills and keeps IT costs to a minimum. Unfortunately, banks

are then restricted to that supplier’s universe of data, infrastructure and

applications, and it is unlikely that they will cover all their information needs or

be first in adapting to new trends. Once trapped in the provider’s universe,

banks are subjected to arbitrary mark-ups dictated by the provider.

Connect applications directly to different providers, offering the freedom

to choose the best available information sources, and making it possible to

negotiate costs. However, this results in non-transparent billing, additional


7

exchange fees and high IT maintenance costs. Overall reliability and

manageability may suffer leading to unpredictable costs for support and for

further development of home-grown solutions.

Image 2: Current World of Data Delivery with direct Feed Connections to Applications

So today, companies in the financial industry have to choose. On the one hand, opt

for a complex and highly expensive single-provider solution that does not satisfy the

needs of all groups in the bank. Alternatively choose a home-grown consolidation of

feeds from different providers that forces them to build application connectors and

operate and maintain highly complex IT infrastructure and hundreds of program

sources. Even worse, it may be a mixture of both approaches.

4. OVERVIEW OF FMDP APPROACH

Consolidation of the provider scene has effectively reduced the number of market-

data platforms available. The TCO of a new solution must therefore be significantly

lower than the current installation, and must be vendor independent, i.e. it cannot be

developed or sold by a financial market data provider. There must also be a way to

migrate applications incrementally, without requiring a “big-bang” approach. So, any

new solution must be capable of coexisting with the pre-existing systems until all

applications have been moved.

FMDP handles the main task of ultra-fast I/O with innovative purpose-built hardware

instead of software-based solutions running on general-purpose servers like current

products. This addresses most of the problems mentioned above by eliminating

technical dependence on data vendors, removing the necessity for server proliferation

to meet scaling requirements, providing a single platform for all service levels, and

reducing administration tasks required to maintain the infrastructure.

As one of the most important features of a market data distribution platform, FMDP

provides entitlement as a key feature, giving customers the ability to control which

clients are permitted access to which data. An important design goal was to build a


8

system where all sources can be connected and all providers and exchanges will be

happy to allow the distribution of their data (Entitlement).

Where vendors require the use of their own Entitlement systems, the feed handlers

perform checks on them before feeding the data into FMDP.

Due to its simple architecture enabled by the use of Solace Systems appliances,

FMDP is a fast-to-install solution that includes an Entitlement controlled Pub/Sub API

along with a ready to go Excel Add-In.

5. FMDP DESIGN GOALS

5.1. Dedicated Hardware for Messaging

To address all the requirements e.g. for a new generation market data distribution

platform, FMDP is built around dedicated hardware for high speed data distribution

and filtering (FMDP-Blade) running on devices from Solace Systems

(www.solacesystems.com). This hardware is purpose-built to offer low latency, high

resilience and high throughput of streaming, topic-based data without the need for

servers, operating systems or additional messaging software.

FMDP has added special built-in features for topic-based access control – the

Entitlement Firewall.

Today, the most time and performance critical components of a data distribution

system are covered by a number of communicating servers all connected to one or

more broadcast backbones. Solace hardware has “burned” this publish/subscribe

paradigm into pure hardware using FPGA technology that, in contrast to standard

CPU processing, can speed up routing and delivery of data by 10 to 1000 times.

Using Solace hardware, all data is directly routed to the subscribers and the

broadcast layer is no longer needed, thus making latencies predictable having

removed the need for a dedicated broadcast backbone. Furthermore this approach

perfectly supports forced entitlement capabilities.

A single FMDP-Blade is capable of managing up to 20 million messages per second,

and can deliver wire-to-wire latency as low as 75 μsec. The hardware approach

makes it possible for FMDP to handle millions of subscriptions to thousands of clients

simultaneously.

5.2. Point-to-Point TCP/IP Protocol

All communication in FMDP is based on TCP/IP point-to-point services with domains

separating vendor streams from each other and different accounts with hundreds of

consumers.

The FMDP Domain capabilities of the system allow for multi-client installations in

remote data centers for hosting or cloud constellations.


9

Image 3: Old World global Distribution vs. FMDP based on Solace Hardware

5.3. Topic-based Data Delivery

All data received is transformed to a standard Solace in-memory format and placed in

the FMDP-Solace appliance for fast DMA access of subscription management and

topic routing on the FMDP-Blade.

All administrative communication, i.e. subscription requests, authentication requests

or feed connection requests are TCP/IP messages controlled by the FMDP API. This

ensures data is accessed only by entitled and authenticated participants. Data is

tagged with an internal “Product Key” derived from the provider’s unique information

about the data’s provenance, asset class etc. to ensure Entitlement capabilities.

In addition, all FMDP admin services are directly connected to the FMDP hardware,

avoiding the need for different technologies, protocols or proprietary software

components for interconnection, so all admin features also benefit from Solace’s built-

in capabilities for resilient, distributed infrastructures.

5.4. Hardware-based Entitlement Firewall (MECS)

MECS (Multi-source Entitlement Control Service) adds all components to the client’s

FMDP Domain that manage the Entitlement Firewall for consumers; i.e. for

authentication, entitlement and connectivity of feeds and applications.

Subscription attempts by the API are checked and if there is a positive response the

data is passed to the consumer.


10

Image 4: Solace Hardware based Entitlement Firewall

NOTE: It is important to note that entitlement is also checked on initial

subscription request to some vendors in the feed adapter if the vendor so

requires. This is the case in the Bloomberg and Thomson Reuters feed

handlers. The Bloomberg Feed Handler verifies user subscriptions against the

EMRS while the Thomson Reuters Feed Handler uses openDACS for user

entitlement IN ADDITION to the FMDP Entitlement. This is due to the fact that

FMDP Entitlement cannot be turned off by design.

5.5. Single API

FMDP delivers a royalty-free API that allows all types of participants to connect to the

system either feeding or consuming. MECS ensures that all data is delivered to

entitled users only while freeing the developer from complex handling of entitlement

functions. There is only one API for all communication within FMDP, i.e. Admin Tools,

Management Clients and Monitoring all use the same communication paradigm.


11

6. FMDP COMPONENTS

FMDP calls all kind of connected applications “Participants”. These are distinguished

by their role in the system and are differently supported in the FMDP API. Entitlement

is also handled differently depending on the role defined for the application. An

application has to be previously defined, incl. its role, to be able to login.

6.1. Feeds & Caching

The Feed participant is an application that publishes into the FMDP System. There

are different types of feeds as described below. All feeds can contain a last value

cache depending on usage.

6.1.1. Broadcast Feeds

Broadcast Feeds are feeds that start pumping all available data to the FMDP System

on connection to the delivering provider. Examples are the former Bloomberg BPipe,

VWD Broadcast feed and Morningstar Real-Time Broadcast feed (all TPC). FMDP

supports the connection of Broadcast Feeds and manages the cache automatically.

6.1.2. On-Demand Feeds

On-Demand Feeds deliver their data on request. As such, only data that has been

previously subscribed to is delivered. Examples are Bloomberg’s new B-Pipe and

Morningstar Real-Time on-demand feeds (TAPI), Thomson Reuters EDGE devices

as well as the RMDS or TREP-RT System interconnection (H2RBridge). FMDP allows

the connection of on-demand feeds, handling the initial subscription to the provider’s

feed on first request by a consumer, and managing the cache automatically.

Depending on data vendor restrictions on data access, the vendors’ Entitlement

Systems are used for Entitlement (e.g. openDACS or EMRS).

6.1.3. Contribution / User Publishing

FMDP treats contributors as feeds. All rules for feeds apply to contributions in the

same way, i.e. contributions must deliver Entitlement IDs and must belong to a

registered feed. MECS manages permissions to specific sources and message types.

6.1.4. Cache

FMDP includes high speed dynamic 64-bit caches for dynamic last value

management. FMDP high-speed caching technology is used for stand-alone caches

serving e.g. Contribution Data from EMIL (FMDP Excel Integration) or is included in

feed handlers. Caches include data persistence, automatic failover, lookups, cron

controlled clean-up and many more features. Caches are handled similarly to feed in

FMDP thus their role must be “Feed” type.

6.1.5. Message Recorder

FMDP optionally includes 64-bit message recorders for any kind of message type into

a NoSQL Database. This can be required for compliance reasons or for building

histories for internal data. The message recorders are normal applications in FMDP.


12

6.2. Multi-vendor Entitlement Control Service (MECS)

The Entitlement Firewall is managed by MECS. MECS is described in more detail in a

separate document. FMDP has an entitlement system in addition to the authentication

mechanism, namely the Multi-vendor Entitlement and Control System (MECS).

Both are integrated in the FMDP MECS service and rely on the FMDP database. Its

most important tables are:

Users: lists all users and applications permitted to connect to the FMDP

environment.

Products: lists all data and application products available in the FMDP

environment, e.g. feed servers, feed record types or programs (consumers or

applications).

UserEntitlement: contains the entitlement information for users and products.

DataDictionary: contains field descriptors for fields of a specific provider and

feed ID (optional).

Multiple FMDP MECS services may run concurrently for fail over and load balancing

purposes, but all must use the same database. MECS is delivered with two graphical

client applications for intuitive management of the FMDP Platform:

FMDP Manager

GUI for the management of all components and settings for FMDP

Image 5: FMDP Manager


13

FMDP Entitlement Editor

GUI for the intuitive management of all Data and Application Entitlements.

Image 6: Entitlement Editor

6.3. Vendor-Approved Entitlement Control

For sources that require their own entitlement system (e.g. Bloomberg EMRS) a

vendor approved entitlement agent is available.

6.4. API

FMDP is delivered with a royalty-free API, the same API that is used for all existing

FMDP Components. The API allows subscribing, publishing and request/response

programming for application developers while assuring Entitlement checks and

automatic monitoring without user intervention.

6.5. Testing Tools

A sophisticated set of installation testing and development support tools such as

message dumps and mock publishers are available for FMDP.


14

7. TREP-RT/EDGE/RMDS INTEGRATION

A high speed FMDP-to-TREP-RT-Bridge (H2RBridge) is available that allows all

applications connected to FMDP to receive TREP-RT Data from all active TREP-RT

sources in high-speed. Entitlement is then managed by openDACS.

8. BLOOMBERG BPIPE FEED HANDLER

For FMDP (and raw Solace Systems devices) a Bloomberg Feed Handler is available

that allows all applications connected to FMDP to receive Bloomberg Data. The

Bloomberg BPipe Feed Adapter has been certified by Bloomberg for RBCCM

worldwide locations and uses EMRS entitlement.

9. MICROSOFT EXCEL® INTEGRATION (EMIL)

A fully featured FMDP-Connector for Subscribing and Publishing in high-speed to

FMDP from and to Excel® (Excel® Multi-source Integration Layer, EMIL) is available.

10. OPENMAMA BRIDGE

FMDP is fully openMAMA connected. All openMAMA applications can immediately

use FMDP including all features such as caching and entitlement.

11. ITRS GENEOS MONITORING

ITRS GENEOS Monitoring is available for all FMDP Feed Handlers.

12. CLOUD CAPABILITIES

FMDP components can be deployed to private clouds thus making the Platform

available to different firms simultaneously if needed. However, feed adapters and

entitlement components must be installed and entitled on a client-by-client basis, but

all can use the same Solace HW. For each client the components run on different


15

FMDP-Domains. FMDP-Domains can still be configured to interconnect different

clients for exchanging client’s internal data in high speed.

13. EXAMPLE FMDP SETUP (NOT SHOWING DR SITE)

14. BCC GROUP – FINTECH MADE IN GERMANY

BCC Group is a Frankfurt-based FinTech software house specialized in the

development and delivery of high-quality software solutions for the financial industry.

For nearly two decades, we have been supporting customers in the area of Market

Data Technology, Market Data Processes and Spreadsheet Integration Technology.

BCC Group’s core competence is the development of solutions based on financial

market data feeds (Bloomberg, Thomson Reuters, Morningstar, etc.) and Solace

Systems appliances. This includes full Excel Integration (EMIL) for publishing and

subscribing to market data. BCC Group also is a development partner to Solace

Systems and was the first partner company distributing Solace Systems technology in

Europe.

BCC Group has developed FMDP based on Solace messaging technology for the

high-speed distribution of bank owned, permission controlled financial market data.

BCC Group also developed the first Bloomberg Feed Handler with full EMRS control

connected to Solace devices serving customers based all over the world. BCC Group

maintains strong development partner relationships with Solace Systems,

Morningstar, Bloomberg, Thomson Reuters, Software AG, Microsoft, mongoDB, ORC

Software and others.


16

BCC Group International GmbH & Co. KG

Taunusanlage 1

60329 Frankfurt am Main, Germany

Telephone: +49 69 50 50 60 – 704

Fax: +49 69 50 50 60 – 520

Email: [email protected]

Website: http://bccgi.com

mailto:[email protected]

FMDP - BCC Group International · As a source-agnostic platform, FMDP focuses on cost reduction, transparency and governance while lowering the TCO. It provides an easy, low-risk

Documents