CON2161 Big Data in Financial Services: Technologies, Use Cases and Implications Jim Acker Global Solution Manager for Big Data Industry Business Unit, Financial Services
Mar 28, 2015
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1
CON2161Big Data in Financial Services: Technologies, Use Cases and ImplicationsJim AckerGlobal Solution Manager for Big DataIndustry Business Unit, Financial Services
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2
Understanding the DriversExecutives frustrated with their data gathering and distribution systems
Executives’ Biggest Data Management Gripes:*
#1
#2
#3
#4
#5
Don’t have the right systems in place to gather the
information we need (38%)
Can’t give our business managers access to the
information they need; need to rely on IT (36%)
Systems are not designed to meet the specific needs
of our industry (29%)
Can’t make sense of the information we have and
translate it into actionable insight (25%)
Information is no longer timely by the time it makes it
to our business managers (24%)
* Source: Oracle Overload to Impact Study 2012
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
695,000 Status updates
510,040 Comments 2,000,000
Search Queries
204,166,667 Emails
571NewWebsites
The data problem just got a lot biggerLeveraging untapped data for commercial gain
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4
The Big Data Opportunity
Big Data: Techniques and Technologies that Enable Enterprises to Effectively and Economically Analyze All of their Data
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5
Big Data is ALL DataUnstructured, Semi-Structure and Structured
There is always structure. But its not formally definedor anticipated.Social Media, RSS feeds, Videos, DOCs, PDFs, Graphics
Semi-Structured. Does not conform to DB tables, butstill contains tags or semantic elements.Emails, log files, machine generated content
What is the main difference in this data?
Volume, Velocity, Variety, Value
These Characteristics Challenge your Existing Architecture
and your Thought Processes
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6
Contrast in Big Data ModelsDemands a new holistic look into data architecture
SQL RDBMSSchema on
Write
Relational DB
HDFSSchema on
ReadMap-Reduce
Distributed File System
No / Minimal Data Model ExplicitExtreme Scale Scale Large Scale
Batch / slow – getting faster Processing Real time and batchMinimal Security Robust
Flexibility and time to value Advantages Optimized and familiar
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7
Pulling it ALL Together for Business Value
Create value from the full range of data sources– Its about using ALL your data
– No more sampling
Value First– Let the data drive the questions, or …
– Test a hypothesis against all your data
Still Need Information Management– Once you find value, incorporate IM
– Big Data is NOT a Silo
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8
A Word of CautionGartner Hype Cycle for Big Data
You are Here
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9
Big Data in Financial Services
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10
ALLDATA
DiscoverAnalyze
PlanPredict
BETTERDECISIONS FASTER ACTION
Big Data is About Analytics
ACQUIRE
ORGANIZE
10Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11
Big Data Use Cases Today
Finding and Monetizing Unknown Relationships
Correlating Diverse Data Sets
Drive OpportunityReduce Cost
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12
Big Data Solutions for Financial ServicesTwo main patterns for how customers are using Big Data
IT Optimization
Big Data Analytics
• ETL and batch processing • Extended Data Warehouse• Mainframe offloading • Archiving
• Customer 360 • Omni-channel CX• Cross-selling / Geo-fencing • Payment Analytics• AML / Anti-Fraud • Risk Management• Pricing Management • Compute Offload (VAR)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13
IT Optimization
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14
Big Data Usage PatternETL and Batch Processing Workloads on Hadoop
Integrate
SQL
SQL
NoSQL
• Scalable• Flexible• Cost
Effective
DW & BI
Analytics
Web
Mainframe
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15
Regions Bank
Objectives
Meet ever evolving regulatory requirements Consolidate existing deposit, loan and
customer databases
Solution
Big Data Appliance and Exadata ODS for single, reliable, cleansed data source
ODS is single landing zone and archival repository for internal, external, structured, semi-structured, and unstructured data
Results & Benefits• Faster access to all their data• Reduced IT costs by eliminating duplicate
data stores
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16
Thomson Reuters
Objectives
Maximize cross-sell opportunities Lower cost and complexity
Solution
Economically capture all customer activity Testing 50M events/sec ingest rates into
the Oracle Big Data Appliance Feeds Exadata EDW for customer
profitability & segmentation analysis
Rick KingChief Operating Officer for TechnologyThomson Reuters
“Oracle's engineered systems… are geared toward high performance big data delivery - and that is exactly the type of work we do”
BDA Exadata Exalytics
EDWSandbox & DR
Event Capture & Store
Interactive Analytics
Research Applications
Upsell/Cross Sell
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17
Big Data Usage PatternExpand Data Warehouse with Granular Data Store
MartsData Warehouse
Σ Σ
BusinessIntelligence
Archiving
• Online• Scalable• Flexible• Cost
Effective
Data Factory
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18
End-to-end business information environment that provides accurate, transparent and timely information to shareholders, regulators and management
Objectives
Tier 1 Global BankNew Information Management Architecture
Results & BenefitsReduce complexity and risk of changesReduce cost of operation Increased stability & performance
Results & BenefitsReduce complexity and risk of changesReduce cost of operation Increased stability & performance
Solution
7 Exadata Racks 16 Node Hadoop Cluster – 33TB Oracle Loader for Hadoop (pending)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19
Big Data Analytics
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20
Ad-hoc
Big Data Usage PatternScale-out Information Discovery
• Online• Scalable• Flexible• Cost
Effective
Data FactoryContinuous On-Demand
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21
Enable customers to learn about stocks and increase buying confidence
Cultivate the advisor-client relationship online and acquire smaller clients
Objectives
Credit SuisseIncreased sales through instant access to information
Results & Benefits Incremental sales for Bank based on this
application for 5 years. Improved customer relationships
Results & Benefits Incremental sales for Bank based on this
application for 5 years. Improved customer relationships
Solution Information Discovery on pooled research
data sets in multiple unstructured formats Oracle powers their internal application that
advisors utilize to quickly find information on financial metrics
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22
Big Data Usage PatternInstant Responses based on Historical Analysis
BusinessIntelligence
• Online• Scalable• Flexible• Cost
Effective
Integrate
Event Decisions
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23
NoSQL for Fraud ScoringFinancial Services coordinated theft prevention
Objectives
Solution
Combine data sources for complex scoring Detect, alert analyst with low latency Handle burst seasonal transaction volumes
Oracle Coherence cluster for real time transaction object management
Oracle NoSQL Database for fraud model and customer profile management
Oracle Database for statistics and fraud modeling-related data
Application Data Ingestion
Tra
nsa
ctio
n A
uth
oriz
atio
nP
roce
sso
r
NoSQL DB Driver
Results & Benefits Simple data model, flexible transactions Scalable, Low Latency data management Easy configuration and administration Enterprise Support
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24
Real-time Location-Based OffersTier 1 Global Bank
Objectives
Customer profile enrichment with Big Data Capture credit card POS and merchant data with
event processor Determine geo location of POS and nearby bank
wholesale customers Leverage real-time decision engine to generate
offer to mobile device
Solution
Increase revenue through real-time, location based offers
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.25
Tier 1 Global BankOffer Workflow
Capture credit card
transactions & identify customer location
Derive next best offer
using customer
information and
propensity
Evaluate offers
based on customer location
Make offer through mobile text message
Locate and identify customer Select next best offer
Identify next best offers Make offer
Analyze customer acceptance/rejection
Enrich propensity based on acceptance/rejection
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.26
ATM MACHINE
POS DEVICE
SMART PHONE APP
Event Capture and Co-relation
Temporal cache based customer identification
REAL TIME EVENT CAPTURE
TCPIPIF
XX
ML
Routing
Integration adapters
Rules
Mapping
Real-time/Near Time, Batch
DATA TRANSPORT LAYER
XML
IFXT
CP
IP
Real time decision
Real time intervention – click to chat, click to call
Adaptive self- learning
Near real-time analysis and dashboarding
INTELLIGENT INTERVENTION PLATFORM
WEBSERVICES MQ
NE
XT
B
ES
T
AC
TIO
N
EXECUTION
Near time/Batch for acceptance/rejection data
Near time/Batch to performmodel update
FACTORY
BANK REPOSITORIES
Client profile, historical transactions, Good life data, segment info, profit info, risk info, Opt-in
information etc.
KEY VALUE PAIRS
Map information, social networks, device logs, smart app interfaces etc.
STAGING
Structured, Non-structured, Semi-structured
MapReduce + NLP Derived outputs- intent, segment, enhanced customer masteringETL/Real-Time
Statistical modeling – Propensity, segments etc.
Natural language processing
Intent and semantic inference
Advanced model free visualization
DATA VISUALIZATION LAYER
DATA PROCESSING LAYER
DATA STORAGE LAYER
System ArchitectureOracle Big Data at Work
LEGEND
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27
Product Roadmap
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28
Engineering the Oracle Big Data Solution
Stream Acquire – Organize – Analyze
In-D
ata
ba
se
A
na
lyti
cs
DataWarehouse
Oracle Advanced Analytics
OracleDatabase
Oracle BI Enterprise Edition
Oracle Real-TimeDecisions
Endeca Information Discovery
Decide
Oracle Event Processing
Apache Flume A
pp
lic
ati
on
s
Oracle NoSQL
Database
Cloudera Hadoop
Oracle R Distribution
Oracle Big Data Connectors
Oracle DataIntegrator
Unified Analytics APIs
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.29
Why Make Big Data a Divided World?
VS
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.30
Goal: Unified Data Analytics Environment
VS
•Real-Time Analytics
•Thousands of Users
•Secure and Available
•All Data On-line and Ready to Use
•Large Scale Systems
•Cost Effective
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31
Unified Data Analytics EnvironmentUnified Analytics API
SQL R MR
Unified Analytics Processing Platform
Hadoop RDBMS
IB
Management Framework and Tools
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.32
Analyze Data across your Oracle Systems
SQL Analytics on ALL data Expand the data pool for
analytics leveraging Hadoop
Stream Hadoop resident data
through Big Data Connectors
for SQL processing
Use the full power of Oracle
SQL on all data
Or use Oracle Loader for
Hadoop to integrate data in
Oracle Database
SQL
Hadoop Oracle Database
IB
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.33
Analyze Data across your Oracle SystemsR Analytics on ALL data
Expand the data pool for
analytics leveraging Hadoop
Improve scalability and
performance for R without
changes to your programs
Dynamically leverage Hadoop
through Big Data Connectors
to execute R analytics
R
Hadoop Oracle Database
IB
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34
Unified Data Analytics Environment
Real-Time Analytics
Thousands of Users
Secure and Available
All Data On-line and Ready to Use
Large Scale Systems
Cost Effective
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35
Unified Big Data Environment
VS&
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.36
Oracle Big Data Solution
Stream Acquire – Organize – Analyze
In-D
ata
ba
se
A
na
lyti
cs
DataWarehouse
Oracle Advanced Analytics
OracleDatabase
Oracle BI Enterprise Edition
Oracle Real-TimeDecisions
Endeca Information Discovery
Decide
Oracle Event Processing
Apache Flume A
pp
lic
ati
on
s
Oracle NoSQL
Database
Cloudera Hadoop
Oracle R Distribution
Oracle Big Data Connectors
Oracle DataIntegrator
• Complete
• Integrated
• Scalable
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.37
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.38