3 джозеп курто превращаем вашу организацию в big data компанию

Transforming your organization into a Big

Data companyBIG DATA | Digital October 2015

Josep Curto | Professor, IE Business School | CEO, Delfos Research

2

Agenda

Digital disruption (or why Big Data is one of the key strategies for your organization)

Business Models (The multiple personalities of Big Data)

Implementing Big Data (The moment of truth)

3

Digital Disruption

4

Context

5

Have you ever wonder….

Why Amazon, Apple, Google, Facebook and Netflix are so successful?

6

They understand one major thing

7

The real you!

8

In particular,…

Your digital footprint

9

Let’s think for a moment

Every object, person and organization has a digital information halo

10

And the data…

… is becoming more robust, valuable and complex

11

This is the new normal

Business changes

Technology changes

12

Based on…

S (Social)

M (Mobile)

A (Analytics)

C (Cloud)

helping to create

personal customer

experiences

13

Challenges

14

Challenge 1: Capture

Source: Bitly http://blog.bitly.com/post/9887686919/you-just-shared-a-link-how-long-will-people-pay

Faster data, reduce lifespan

15

Challenge 2: Storage

0.8$ZB$

*Ze#abyte:)1,000,000,000,000,000,000,000)bytes)(#1#trillion#gigabytes)#

2009$

2020$

40##Ze5abytes*#

Source: IDC IDC Digital Universe Study, 2012, Sponsored by EMC

More data! what is relevant?

16

Challenge 3: Analisis

The more sources we have, the more complex to extract the value

Source: IDCIDC Digital Universe Study, 2012, Sponsored by EMC

2011: 50.07 Tb/s

2012: 86.54 Tb/s

Data in transit: 856 Tb/s

17

Challenge 4: Visualization

New data sources and formats and increased volume need new communication techniques

18

Challenge 5: IT Dependence

Technology is deeply embedded into our business processes

19

Challenge 6: Create a new culture

• Attitude • Habits • Knowledge

20

Big Data

What is big data

Volume Velocity Variety

22

Veloc

ity Variety

Volume

· Batch· Near Real Time· Real Time· Streaming

· Structured· Semi-structured· Unstructured

Volume + Variety

Volume + Velocity

Velocity + Variety

· Terabyte· Petabyte· Exabyte· Zettabyte· Yottabyte

Volume + Velocity + Variety

23

More Vs

The most important V

25

Big Data is not new

KB

Files

Statistics

COBOL

GB

Tables

OLAP Cubes

SQL

TB

Semi-structured

Apps

XML

PB

Dynamic Variety

Mahout (& other)

NoSQL

Big

Data

Analítica

Language

60s 80 - 96 97 - 07 07 - ?

26

3Vs are not enough

Veloc

ity Variety

Volume

· Batch· Near Real Time· Real Time· Streaming

· Structured· Semi-structured· Unstructured

Volume + Variety

Volume + Velocity

Velocity + Variety

· Terabyte· Petabyte· Exabyte· Zettabyte· Yottabyte

Volume + Velocity + Variety

• Horizontal scalability • Relational constraints

27

Relational Constraints

Non Relational Schema-free

Distributed 4 Types: Key-value, column-oriented, graph, document

Relational Data is normalised and

static Relational schema

Data in one repository

28

Horizontal Scalability

What is big data (II)

Batch Processing

(Volume)

Stream Processing

(Velocity)

NoSQL (Variety)

30

Big Data Layers

31

Hadoop is the most famous technology

32

Big Data

RDBMS

Data Visualization

Predictive Analytics

Hadoop +

Content Search & Analytics

In-memory Streaming Technologies

Object & Graph Databases

It is not the only one

33

We are moving from…

Information Systems

Data

Mobile Data

Machine Data

Social Media Data

Audio, video, text

Stream Data

Sou

rces

Corporate Information Factory / Data Warehousing

Stor

age

&

Pro

cess

ing

Information Management

Data Governance Master Data ManagementD

ata

Man

agem

ent

Ana

lysi

s

Analytics Operational Intelligence

Business Intelligence

34

to a new architecture

Information Systems

Data

Mobile Data

Machine Data

Social Media Data

Audio, video, text

Stream Data

Sou

rces

Corporate Information

Factory / Data WarehousingSt

orag

e &

P

roce

ssin

g

Information Management

Data Governance Master Data ManagementD

ata

Man

agem

ent

Ana

lysi

s

Analytics Operational Intelligence

Business Intelligence

Big Data

NoSQL In-memory MPP HPC

Data Products

35

Big Data Market

36

The Big Data market is growing

Source: IDCIDC Worldwide Big Data Technology and Services 2010 – 2015 Forecast, March 2013

0

10

20

30

40

2011 2012 2013 2013 2015 2016 2017

32.4

25.7

20.4

16.1

12.69.8

7.4

37

Hadoop is leading the way

1991 1992 1994 1995 1997 2000

Project emerge

Community creation

Code is available,

community grows

First companies

Ecosystem emergence Mainstream,

M&A Starts

2006 2007 2008 2009 2012 2015

38

Many tecnologies

Source: 451ResearchDatabase Landscape Map - February 2014

39

From data to business

Source: IDC

40

The number of companies is growing

Source: Delfos Research Worldwide Data Market 1836 - 2015

41

But our expectations are high

Fuente: GartnerHype Cycle for Big Data,2012

Fuente: GartnerGartner HypeCycle 2013

42

New needs

43

A new organization

44

Our data needs

Available

Accesible

Quality

Right time

Security

Information

45

Which technology is required?

Data Market

BD

BA

BI

DM

DT

46

Which capacities do we need?

47

How do I have to compete?

Decoding the digital footprint

48

Combining different factors

Amplificator Interface

Algorithms Data

Business model

49

We need to compete on innovation

Wow moments

Great virtual design

Great physical design

50

Business Models

51

Data-driven business models

52

How to generate value

53

Optimization ≠ innovation

54

Data-driven Business Models

Source: University of CambridgeA Taxonomy of Data-driven Business Models used by Start-up Firms

Data-driven Business

ModelKey Activities

Customer Segment

Revenue Model

Key Resources Cost Structure

55

Data

Synthesizing the different sources leads to the taxonomy

14

Data Sources

Internal existing data

Self-generated Data

External

Acquired Data

Customer provided

Free available

Open Data

Social Media data

Web Crawled Data

Source: University of CambridgeA Taxonomy of Data-driven Business Models used by Start-up Firms

56

ActivitiesDimension: Activities

15

Key Activity

Data Generation Crowdsourcing

Tracking & Other Data Acquisition

Processing

Aggregation

Analytics

descriptive

predictive

prescriptive Visualization

Distribution

Source: University of CambridgeA Taxonomy of Data-driven Business Models used by Start-up Firms

57

Value proposition

Dimension: Offering

16

Offering

Data

Information/Knowledge

Non-Data Product/Service

Source: University of CambridgeA Taxonomy of Data-driven Business Models used by Start-up Firms

58

Revenue modelDimension: Revenue Model

17

Revenue Model

Asset Sale

Lending/Renting/Leasing

Licensing

Usage fee

Subscription fee

Advertising

Source: University of CambridgeA Taxonomy of Data-driven Business Models used by Start-up Firms

59

Customers

Dimension: Target Customer

18

Target Customer B2B

B2C

Source: University of CambridgeA Taxonomy of Data-driven Business Models used by Start-up Firms

60

Some types are emergingThe 6 BM types are characterised by the key activities and key data sources

29

Type F

Type A Type D

Type E Type B

Type C

Aggregation Analytics Data generation Fr

ee

ava

ilabl

e Cu

stom

er

prov

ided

Tr

acke

d &

ge

nera

ted

Key activity

Key

Data

Sou

rce

6 significant Business Model types were identified

28

Type B: “Analytics-as-a-Service”

Type C: “Data generation & Analytics”

Type D: “Free Data Knowledge Discovery”

Type A: “Free Data Collector & Aggregator”

Type E: “Data Aggregation-as-a-Service”

Type F: “Multi-Source data mashup and analysis”

Source: University of CambridgeA Taxonomy of Data-driven Business Models used by Start-up Firms

61

One example: CBInsights

Data-driven Business Model

Data Processing, Aggregation &

Analysis

Investors, media

Subscription fee

External Companies Data

Platform, data scientists

62

Some examples

63

Big Data around the world

Credit Suisse Netflix

TescoWalMart

General Motors

Disney Metlife

Apple

Caesars Entertainment SpotifyHouston Rockets NFL

64

The dark side of Big Data

65

The dark force is strong in your company

Google knows about you more than you think

A graph to rule them allNSA

66

Where is the privacy?

67

Even your TV watches you!

68

Responsibilities

Integrity Confidentiality Authenticity Availability

69

Big Data limits

Internal limits

Skills Culture Integration

70

External limits

Regulation Data ≠ Correlation

Models ≠ Reality

71

72

Implementing Big Data

73

Implementing Big Data

Many reasons to fail

IMPLEMENTATION FAILURE

COST

OPERATIONAL STRATEGIC

Lack of top management commitment

Unavailability of subject matters

experts

Unavailability of key users for

UTA

Poor quality of testing

Poor Knowledge

Transfer

Cost Overrun

Unrealistic ROI

TECHNOLOGY PEOPLE

Poor Data Quality

Over Customization

Inadequate data sources knowledge

Poor IT infrastructure

Poor ETL Quality

Poor BI product selection

User resistance To change

High rotation of Project team members

Inadequate resources

Poor user involvement

TACTIC

Inadequate training and education

Non-empowered decision-makers

Poor departmental

alignment

Inappropriate timing to go live

Poor communication

Unrealistic expectations

Inadequate functional

requirements

Inadequate project team composition

Poor project management

Unrealistic project scheduling Ineffective organizational

change management

74

How to start

Framing the problem

• Identify business needs • Conceptualise business opportunities • Determine Big Data type • Define Big Data Strategy

Pilot and beyond

• Develop model • Identify data set • Build / Buy / Subscribe to big data

architecture • Create pilot • Scale pilot

Communication

• Comunicate outcome • Identify next steps

79

Your future big data project

80

Data Strategies

81

What we want

TimeTime to action

Lost

val

ue

Data latency

Analysis latency

Decision-making latency

Business Event

Data ready for analysis

Information

Action

Business Value

82

When we need themBI (mature) BA (mature) Big Data

(emerging)

Tools Query, reporting, OLAP, alerts Forecasting, regression and modeling and/ or BI

Machine learning, visualization

FocusWhat happened, how many, how often, what is the problem, what

action is needed

Why is this happening, what if these trends continue, what will happen

next, what is the best that can happen

Capture, storing and analyzing data: all

Use Reactive Proactive / Predictive / Prescriptive All / none

Types of data

structured Structured / semi-structured All

Data Complexity

Low Low / medium High

Scope Management Processes Vertical / processes

83

Becoming data-driven

84

Traditional Knowledge, Literacy and Skills

Computer Literacy

Analytic Proficiency

Data Proficiency

Operational Proficiency

Total Information Proficiency

Building traditional capabilities and skills

Mastering technology

Automating clerical work

Reengineering business processes

Building ubiquitous knowledge bases

Optimizing all decisions

We want to create digital competencies

85

We need to measure the value (McDonald 2004)

Valu

e C

reat

ed

Overall Success of the Initiative

Implementation Success

• On- time • On-budget

User Success• User adoption • User Satisfaction • Data Problems

Operational Success• Productivity

Improvements • Process efficiency and

effectiveness • Key Performance

Indicators

Business Success• Return on Investment • Economic Value Add • Revenue increases • Cost Savings • Customer / Corporate

profits • Enables Business

Strategy and Competitive Advantage

• Create a formal, continuos process for measuring success and value generated

• Identify and measure results of each project phase• Establish realistic goals and expectations based on

capability / maturity

86

We need to become data-driven

Source: Thomas H. Davepnort

Josep Curto @josepcurto

[email protected]