R:evolution 2014 - Martin Willcox

Big$Data$Then$And$Now$Presenta(on*to*R:Evolu(on*Conference*!*5th*March*2014*!*Mar(n*Willcox,*Director*Big*Data*CoE*(Interna(onal)*

The$technology$industry$thrives$on$hype:$“Big$Data”$is$hot$right$now$

“Big$Data”$has$recently$overtaken$“cloud$compu>ng”$as$the$most$hyped$expression$in$IT.$

We$are$currently$bombarded$with$“facts”$and$opinions…$

“Unprecedented*data*growth…*that*con1nues,*regardless*of*budget*constraints”*Ten*Trends*&*Technologies*To*Impact*IT*Over*The*Next*5*Years*

David*Cappucio,*Research*VP*(Gartner),*January*9th*2013*

…on$both$sides$of$the$“Big$Data”$debate$

“Big*Data*is*bullshit…*it’s*really*just*data.”*Harper*Reed,*CTO*Obama*For*America*

Spoiler:$both$sides$are$half$right$

Yes*it’s*a*big*deal…* …no*it’s*not*unprecedented*

Big$Data,$circa$1986$Deployment*of*EPoS*systems*in*the*late*80s*revolu(onises*Retail*

•  Enormous$(by$the$standards$of$the$day)$Teradata$system$enables$WalMart$to$capture$store*/*SKU*/*day*level*aggregated$data$across$all$its$stores$in$North$America;$

•  The*rest*is*Retail*history…*

Big Data Big Data “…really$we$got$big$by$replacing$inventory$with$informa>on…”$–$Sam*Walton,*Founder,*WalMart*

From*transac1ons*I*to*interac1ons:*the*three*new*waves*of*Big*Data*

Analysis$of$clickstream$data$enables$Amazon$and$eBay$to$achieve$“mass$customisa>on”$of$their$webSsites.$

Analysis$of$social$/$interac>on$data$enables$Amazon,$Apple$and$LinkedIn$to$go$social$(“people$who$like$what$you$like$also$like…”)$

Increasing$instrumenta>on$is$now$leading$to$the$emergence$and$op>misa>on$of$“the$Internet$of$Things”.$

People*interac1ng*with*

things*

People*interac1ng*with*

people*

Things*interac1ng*with*

things*

*(1)**

*(2)**

*(3)**

These*trends*are*real*and*accelera1ng*–*but*are*they*about*“more”,*or*“different”?*

“Big$Data”$are$oVen$“unstructured”$and$difficult$to$store$and$analyse$in$tradi>onal$database$technologies…$

I didn’t say Bill was ugly. I didn’t say Bill was ugly. I didn’t say Bill was ugly. I didn’t say Bill was ugly. I didn’t say Bill was ugly. I didn’t say Bill was ugly.

…now$“new”$informa>on$management$strategies,$Analy>cs$and$suppor>ng$technologies$are$enabling$us$to$extend$Enterprise$Analy>cs$

Structured(data(

Mul,-structured(data(

Non-tradi,onal((,me-series(/(path(/(graph)(analy,cs(

Coun,ng(things(and(sta,s,cal(analy,cs(

Business$Intelligence$&$Analy>cs$

Capture,*Store,*Refine*

Explora1on*&*Discovery*

Take$home$lesson$#1$

“Big*Data”*aren’t*just*“lots*more*data”;*“big”*oZen*means*“different”.*

The$corollary$of$Moore’s$Law$Simple*compu(ng*devices*are*now*incredibly*inexpensive*

An*iPad2*would*have*stayed*on*the*list*of*the*world’s*most*powerful*supercomputers*through*1994.*

13 06/03/2014 Teradata Confidential

I$CAN$SENSE$YOUR$MOVEMENT$&$UNDERSTAND$YOUR$BEHAVIOR$

Shopping cart will track the consumer’s every move


I$KNOW$WHO$YOU$ARE$FACIAL$RECOGNITION$

Retailers are testing new facial recognition technology


I$KNOW$YOU$AND$CAN$ENGAGE$VIA$MOBILE$

Mobile enabling you to engage in the Store to find the items you like


I$KNOW$WHAT$INTERESTS$YOU$

Improving on-shelf availability with cameras


Facebook-enabled coat hanger tracks the number of ”likes”

I$CAN$UNDERSTAND$IF$YOU$ARE$SOCIALLY$INFLUENCED$

Photo © C&A Brazil/DDB Brazil


The*(smart)*machines*are*coming,*bearing*data.**We*will*soon*be*able*to*measure*anything*–*and*everything.*

New$sources$of$data$follow$the$same$trajectory$From*byZproduct*to*raw*material;*from*BI*to*CI*

“We*are*used*to*the*idea*of*deploying*new*

technology*to*improve*produc(vity*and*

efficiency...*But*data*are*no*longer*merely*the**byZproduct*of*process*improvement,*they*are*becoming*the*raw*

material*of*business.”$

“Hot$right$now”$in$Retail$Big$Data$Discovery$Analy>cs$

Mass$personalisa>on$/$collabora>ve$filtering$ Social$/$sen>ment$analysis$

Marke>ng$ahribu>on$/$PPC$analy>cs$ Golden$path$/$pathStoSchurn$analy>cs$


Tradi>onal$BI:$what$is$the$answer$to$the$ques>on?$Discovery$&$Explora>on:$what$are$the$interes>ng$ques>ons?$

“Capture only what’s needed”

IT delivers a platform for storing, refining, and

analyzing all data sources Business explores data for questions worth answering

Big Data Analytics Multi-structured & Iterative Analysis

IT structures the data to answer those questions

Business determines what questions to ask

Classic BI Structured & Repeatable Analysis

“Capture in case it’s needed”


EDW Model V Big Data Discovery EDW Model

Highly Planned & Controlled Slow Release Schedule •  3x releases 2 years High Central funding cost Low Risk / High Success

Discovery Model

Small Iterative Projects •  40+ Discoveries / 2 years Low cost per project •  $20k-$50k per project BAU funded initiatives •  $Project funded •  $Central discovery team /

BIU for free thinking High Risk / High Fail •  Iterates to a new project

$5m

$5m

$5m

3 Releases Over 2 years

Release 1

Release 2

Release 3

40+ Projects Over 2 years


EDW Model V Big Data Discovery EDW Model

•  EDW projects must succeed

•  Successful Discoveries productionised as part of release schedule

Discovery Model

•  Many projects “fail”

•  Failure is accepted as part of the process and leads to new innovations and iterative projects

•  Successful projects are often productionised on the EDW for execution

$5m

$5m

$5m

3 Releases Over 2 years

Release 1

Release 2

Release 3

40+ Projects Over 2 years

Successful Project Failed Project


Enabling*innova1on*means*embracing*risk,*“failing*fast”*–*and*moving*on.*

And$finally…$

Good*technology*is*necessary,*but*not*sufficient;*organisa1on*and*culture*maber*more.*@Willcoxmnk*

R:evolution 2014 - Martin Willcox

Technology