8/13/2019 Big Data Big Expectations - Dennis Faucher
1/18
But First, Something Fun
1. Pull Out Your Phone
2. Open Your Texting App
3. Prepare to Send a Text to 22333
4. Here are the Possible Text Responses 223-33
http://www.polleverywhere.com/multiple_choice_polls/xNwWZxlNGRbxN2Whttp://www.polleverywhere.com/multiple_choice_polls/xNwWZxlNGRbxN2W8/13/2019 Big Data Big Expectations - Dennis Faucher
2/18
Big Data - Big Expectations
8/13/2019 Big Data Big Expectations - Dennis Faucher
3/18
Todays Agenda
Big Reality: Real Solutions
Big Data Framework
Real Customer Examples
Why
How
What
8/13/2019 Big Data Big Expectations - Dennis Faucher
4/18
What Makes it Big Data?
VOLUME VELOCITY VARIETYVALU
VERAC
SOCIAL
BLOG
SMART
METER
10110010
00100110
10101110
01010010
Data that cannot be turned into business value fast eno
8/13/2019 Big Data Big Expectations - Dennis Faucher
5/18
Where Does Big Data Come From?
Structured: aka "Process-mediated": Examples: ERP, CRM , POS
Characteristics: transactional, referential, relational, Traditio
managed
Semi-Structured: aka "Machine-generated Examples: XML, JSON, Network Logs, Sensor Data
Characteristics: Well suited to computer processing but ma
Volume and accumulation, often too large for EDW
Unstructured: aka "Human-sourced": Examples: CDR, Doctor's Notes, Social Media, Audio, Vide
fields
Characteristics: subjective record of personal experiences,
required to realize value
Process Mediated
Data
This is yourdata.This is placeholdertext
forwhateverbest representsthe structured and unstructured
data you want to query. tis, justo epellentesquemetus, et sollicitudin
diam lectus eu sapien. Cum sociisnatoque penatibus et magnis disparturient montes, nascetur ridiculu
s Do40% more ofthis rhon muscus aliquam, massa mauris por
ta tortor, vel porta odio ligula aelit. Sed laoreet, lectus ac dapibusplacerat, nulla turpis lacin
ia lectus, eget hendrerit nibh mauris vel ligula. Sed a
nisl dolor, a tincidunt leo. ab
Machine Generated
Data
Human Sourced
Data
8/13/2019 Big Data Big Expectations - Dennis Faucher
6/18
Where is the Value?
Gain Market Share (Social Scrapes)
Identify Cost Savings (Streaming Fraud Detection, Doctors Notes)
Customer Retention (Call Detail Records) Real Time Business Automation (High Freq. Trading, Smart Utility
Business Process Re-invention (Personalized Risk vs Statistical Av
Modernization and Competitive Technological Advantage
8/13/2019 Big Data Big Expectations - Dennis Faucher
7/18
How Does it Get to the User?
Hadoop
(Traditional)
EDW
NoSQL
Analytics
Mart
In-Memory
Process Mediated
Data
DataVirtualization
BI/Analytics
This isyour data.Thisisplaceholdertext
forwhatever bestrepresents
thest ructuredandunst ructured
datayouwant to query. t is, justo e
pellentesquemetus, etsollicitudin
diamlectus eusapien. Cumsociis
natoquepenatibus etmagnis dis
parturientmontes,nascetur ridiculu
s Do40%moreofthis rhonmuscus aliquam,massamauris por
tatortor,velportaodio l igulaa
elit.Sedlaoreet, lectus ac dapibus
placerat,nulla turpis lacin
ialectus, egethendrerit n ibhmauris velligula. Sed a
nisldolor, atincidunt leo.ab
Machine Generated
Data
Human Sourced
Data
Da
taIntegration
Disparate, non-performing Integrated, performance-optimized
8/13/2019 Big Data Big Expectations - Dennis Faucher
8/18
Where Do We Put It?
(Traditional)
EDW
Analytics
MartHadoopNoSQL
Human
Information
Java /Open Source VeSQL, Windows, Linux
8/13/2019 Big Data Big Expectations - Dennis Faucher
9/18
A Few Customer Examples
8/13/2019 Big Data Big Expectations - Dennis Faucher
10/18
Volume - Financial Analysis
1T Rows
8/13/2019 Big Data Big Expectations - Dennis Faucher
11/18
Executive Management /
Plant Management
ISA-95 Asset Hierarchical Data ModelMaintenance
Reliability
Operations
Engineering
Manual In
Capital Proj
Supply Ch
Finance
Health Safety &Environment
Outputs:Hundreds Standard KPIsReporting with Drill Down
Root Cause AnalysisRole Based DashboardsManagement of Change
Multiple Site Role upCross Functional
Collaboration Platform
SAPJDEOracle
Aspen EDOScheduling
PrimaveraMS ProjectSharepoint
IntergraphAspenTechAutoCADBentleyAveva
MeridiumCapstoneDB / Excel
IP21OSI PiHoneywellYokogawa
Siemens
MaximoSAPPMOraclePassportDataStream
DBExcel
SAPESS
Variety & Veracity
V i t & V it
8/13/2019 Big Data Big Expectations - Dennis Faucher
12/18
Variety & Veracity
8/13/2019 Big Data Big Expectations - Dennis Faucher
13/18
Using ALL the DataVolume & VarietyeBay
I was thinking if we created a n
engine with added functionality
me find that obscure R2-D2 ac
faster and easier?
Bob, what if we could answer
these questions What didsomeone buy?, What did they
bid on it? Its also Where were
they at the time? Its also Who
influenced them within their
social circle? All that data is
amassed. cool idea?
We can build a Big Datas infrastructure challen
Hadoop cluster consisted of 400 nodes and Two
this will allow us to bring the processing to where
the need for time-consuming data t
Larger Index than Voyage
More descriptions, history & metada
100 engineers, all new codebase,
8/13/2019 Big Data Big Expectations - Dennis Faucher
14/18
Structured Data
VarietyFinancial Structured and Unstructured Data
Select Customers with < 150K in Assets pull
demographics
From a database get me all matches from the CRM and Call Detail Records that match the query
From unstructured sources get me all matches for calls, chat, email that were positive for the structured results
Unstruct
Reference check 21 image DB
Clickstream data from banking w
Meaning based positive commen
Columnar
Pull < total 30% of net worth from Check 21 Image database
Customers who conducted net worth
report from our banking web site
10,015,664,356,165 rows (10 trillion)22B-43B rows loaded daily
20node x86 cluster
1.794petabytes raw data
7:1 compression
8/13/2019 Big Data Big Expectations - Dennis Faucher
15/18
Meter Data Management (MDM) System that 1) Customer segmentation 2) Anomalyvolume
Velocity - Smart Meters
8/13/2019 Big Data Big Expectations - Dennis Faucher
16/18
Can we find any relationship between FDA recall actions & conversations within th
universe?
Variety - Public Health
48 million people get
128,000 are hospitaliz
and 3000 die each yefoodborne diseases
In 2009
Peanut butterrecall costproducers
$1,000,000,000
How to avoid unintended economic consequences
actions?
What where people talking about two weeks before the
How did they feel?
Where were they?
What were they eating?
8/13/2019 Big Data Big Expectations - Dennis Faucher
17/18
How Do I Start?
Data Analytics Gap Analysis
Vendor Agnostic Assessment of
Current State and Go-Forward Strategy
Benchmark your Data ManagementInfrastructure against industry
Identify Critical Business andTechnology Gaps
Matrix/SWOT of product capabilitiesand environmental applicability
Strategic and Technical Execution andDeployment Planning
Proof of Value
Business Value Assess
Technical Assess. of ta
System Design and Arc
POC Installation/configu
Integration Testing
Performance Benchmar
System Tuning/Optimiz
Performance Evaluation
ROI Assessment
8/13/2019 Big Data Big Expectations - Dennis Faucher
18/18
Thank You