Competitive edgewithmongod bandpentaho_2014sep_v3[1]

Gaining a Competitive Edge in FS with MongoDB and Pentaho

Matt Kalan Business Architect, Financial Services at MongoDB [email protected] @matthewkalan

Bo Borland Vice President, Field Technical Sales at Pentaho [email protected] @boborland

2

•  Financial Services Industry Drivers •  Traditional and Desired User Scenarios •  Data Management Requirements

•  MongoDB Capabilities •  Pentaho Capabilities

•  Pentaho BA Demo - Analyzing MongoDB data •  Pentaho DI Demo - Blending Disparate Data •  Questions

Agenda

3

FS Industry Transformation Drivers of change

•  Lost revenue (fees, prop trading)

•  Better risk management

•  Regulatory change and uncertainty

•  New competitors

•  Emerging markets opportunities

•  Proliferation of channels

•  Globally distributed operations

•  Faster market movements

Requirements

•  New products and new markets

•  Increase wallet share

•  Agility to respond to competitors & regulators

•  Firm-wide, cross-silo reporting

•  Cost savings

•  Cross-channel and global integration

•  Intraday decision support

•  Operational efficiencies

4

How to Address

•  Maximize customer engagement

•  Enable cross-silo regulatory and operational reporting

•  Leverage automation and tools for analytics and notifications

•  Based on agile, comprehensive, and timely data management

How to Respond to Transformation Requirements

•  New products and new markets

•  Increase wallet share

•  Agility to respond to competitors & regulators

•  Firm-wide, cross-silo reporting

•  Cost savings

•  Cross-channel and global integration

•  Intraday decision support

•  Operational efficiencies

5

Traditional Interactions

Customer  

Investment  Advisor  

Phone/email/IM  

Investmen

t  An

alysis  

Has  minimal  customer  intelligence  

Mostly  just  publishing  out  informa>on  

Fundamentals  

Pricing  

News  

1.  Decide  to  check-­‐in  aDer  a  quarter  

3.  Calls  client  

4.  Client  having  a  baby  and  looking  for  a  new  house  

Customer  has  rela>vely  limited  value  from  advisor  

2.  Review  porKolio,  research,  and  past  correspondence  

5.  Already  got  mortgage  and  set  up  ESA  6.  Wants  to  talk  again  in  3  months  

6

Desired Interactions

Customer  

Investment  Advisor  

Phone/email/IM  

Investmen

t  An

alysis  

Much  greater  and  frequent  customer  intel  

Richer  and  relevant  informa>on  and  engagement  

Fundamentals  

Pricing  

News  

Customers  benefit  from  high  value  app(s)  and  more  relevant  advice  

Twi8er  

Blogs/RSS  

Facebook  

Central  Bank  info  

Data  Analysis  

Pa8ern  analysis  

1.  Uses  online  savings  guide  w/  1  child  

2.  No>fica>on  

5.  Timely  call  to  check-­‐in  3.  Uses  mobile  app  to  research  Tesla  6.  Client  is  having  a  

baby  and  wants  a  mortgage  

4.  No>fica>on  

7.  You  suggest  an  ESA  and  mortgage  proposal  

8.  You  also  discuss  Tesla  and  ba[ery  technology  

Single  view  of  Customer  

How to Manage All This Data

8

RDBMSs not engineered for these modern applications

Data Types

•  Unstructured data

•  JSON & Digital

•  Polymorphic data

Volume of Data

•  Petabytes of data

•  Trillions of records

•  Millions of queries per second

Agile Development

•  Iterative

•  Short development cycles

•  New workloads

New Architectures

•  Horizontal scaling

•  Commodity servers

•  Cloud computing Single Views

•  Disparate data

•  Intraday

•  Cross-channel/silo

•  Global

9

symbol: “TSLA”, type: ”news”, headline: “Tesla…”, url: ”http://...”, }

{symbol: “TSLA”, eps: -1.11}

tweet: “Nice car…”, type: ”tweet”}

{symbol: “TSLA”, type: “fundamental”, mktCap: 34.93, eps: -1.11}

{symbol: “TSLA”, type: “price”, bid: 280.31, offer: 280.51, date: 2014-08-23, bidQty: 300, offerQty: 100}

Investment and Market Data

custID: 1000, type: ”mResearch”, symbol: “TSLA”, sector: ”Auto”, ...}

{custID: 1000, type: ”call”, ....}

custID: 1000, type: ”researchPaper”, doc: ”AutoOverview”, ...}

{custID: 1000, type: “email”, date: 2014-09-14, subject: “Tesla”}

{custID: 1000, type: “savingApp”, income: 200000, mthSvngs: 10000, date: 2014-08-15, numChild: 1, offerQty: 100}

Customer Activity Data

Many shapes of investment and customer data

10

Differently shaped data are spread across many systems

… Bank  mobile  app  

Website  app  

Wealth  Mgmt  App  

Banking  CRM  app  

Investment  Banking  CRM  app  

ONE COMMON MODEL CustID | Activity ID | Date | Type | 100s or 1000s fields mostly agreed up front

11

Need to aggregate it in one dynamic database

… Bank  mobile  app  

Website  app  

Wealth  Mgmt  App  

Banking  CRM  app  

Investment  Banking  CRM  app  

COMMON FIELDS CustomerID | Activity ID | Type…

DYNAMIC FIELDS Can vary from record to record

Single  View  

12

Easy Horizontal Scaling Required

•  No impact to application

•  Minimal impact to operations

•  Elastic capacity as you need it

•  Automatic balancing

ApplicaGon  

One  Logical  Database  

Primary  

ParGGon  1  

Primary  

ParGGon  2  

Primary  

ParGGon  N  

…

13

Rich Querying & Indexes Required

Objects + Rich Querying Multiple Fields •  Select John’s holdings

•  Select everyone holding MSFT

Geospatial •  Find the nearest branch right now

Text Search •  Find all customers that mention China in their call activity (for a new product)

Aggregation •  Calculate the value of John’s portfolio •  Show holdings by customer

Map Reduce •  For those that hold greater than 50,000

shares of each sector, what is the next largest sector they hold?

{ ! customer_id: 100,! customer_name: ‘John Smith’ !

as_of_date: 2014-06-11,! last_updated_location: ! [45.123, 47.232],! phone: [‘212-555-1212’, ! ‘917-111-2222, …]! holdings: [ !

{ symbol: “MSFT”,! quantity: 10000, … },! { symbol: “IBM”,! quantity: 20000, … }, …! ]!}!

14

MongoDB capabilities

ApplicaGon  

Driver  

Mongos  

Primary  

Secondary  

Secondary  

Shard  1  

Primary  

Secondary  

Secondary  

Shard  2  

… Primary  

Secondary  

Secondary  

Shard  N  

db.customer.insert({…})!db.customer.find({ ! name: ”John Smith”})!

1. Dynamic  Document  Schema  

  { name: “John Smith”,! date: “2013-08-01”),! address: “10 3rd St.”,! phone: [! { home: 1234567890},! { mobile: 1234568138} ]! }!

2.  Na>ve  language  drivers  

4.  High  performance  - Data  locality  - Indexes  - RAM  

3.  High  availability  - Replica  sets  - Strong    Consistency  

5.  Horizontal  scalability  - Sharding