MongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, Cloudera

1

I o T E U R O P E A N C I T Y T O U R

S T U T T G A R T

2

Hadoop and the Future of Data Management Bernard Doering Regional Sales Director, Central Europe

33

Leading the Way in Data ManagementPowered by Hadoop2008CLOUDERA FOUNDED BY MIKE OLSONAMR AWADALLAH &JEFF HAMMERBACHER

2009HADOOP CREATOR

DOUG CUTTING JOINS CLOUDERA

2009CLOUDERA RELEASES CDH THE FIRST COMMERCIAL APACHE HADOOP DISTRIBUTION

2010CLOUDERA MANAGER:

FIRST MANAGEMENT APPLICATION FOR

HADOOP

2011CLOUDERA REACHES 100 PRODUCTION CUSTOMERS

2011CLOUDERA UNIVERSITY

EXPANDS TO 140 COUNTRIES

2012CLOUDERA ENTERPRISE 4THE STANDARD FOR HADOOP IN THE ENTERPRISE

2012CLOUDERA CONNECT

REACHES 300 PARTNERS

2014THE ENTERPRISEDATA HUBLAUNCHED

2013CLOUDERA IMPALACLOUDERA NAVIGATORCLOUDERA SEARCH

2013TOM REILLY JOINS AS CEO

OVER 800 PARTNERS IN CLOUDERA CONNECT

CDHCloudera Manager

CLOUDERA ENTERPRISE

4ASK BIGGER QUESTIONS

ENTERPRISEDATA HUB

Intel Confidential4

Big Deal: Cloudera + IntelIntel invests $740M in Cloudera As Intel’s largest data center venture capital investment, which represents

Intel’s commitment to Internet of Things and Big Data Supports Cloudera’s ability to remain independent

Intel & Cloudera drive innovation through open source Accelerate evolution of Hadoop by joining forces on foundational

technologies Enable open source developers to innovate in and on top of the Hadoop

platform

Intel enables CDH to run best on Intel Architecture – performance optimisation Enables Cloudera to make best use of Intel data center technologies Provides datacenter infrastructure for Cloudera development &

benchmarking at scale

Intel Confidential5

Big Goal: Converge on one open source platform

• Most stable, compatible, and mature Hadoop distribution

• Leading SQL functionality & performance (Impala)

• Deepest management and governance capabilities

• 150 Hadoop developers• 100 open source committers

• The only distribution with performance and security enhanced from the silicon up

• Leading security capabilities including encryption, access control, and auditing

• 50 Hadoop developers and 12 committers

• Long-standing committment to open source with 1000 developers working on Linux, KVM, Xen, Java, OpenStack, Hadoop

6

Data drives innovation – Internet of Things

INTELLIGENT CLOUD

Richer data to analyze

2.8 Zettabytes of data generated WW

in 20121

SMART CLIENTS

Richer user experiences

Richer data from devices

INTELLIGENT THINGS

Sources: (1) IDC Digital Universe 2020, (2) IDC

40 Zettabytes of data will be generated

WW in 20201

7

Big Data is All Data and All Paradigms

Transactional & Application Data

Machine Data Social Data

• Volume • Structured• Throughput

• Velocity • Semi-structured • Ingestion

• Variety• Highly unstructured • Veracity

Enterprise Content

• Variety• Highly unstructured• Volume

88

Which one of these people is likely to be carrying a bomb?

Do you have any liquids in your carry-on?

99

Is it possible to set rates based on actual risk for each particular house?

How big is your house? What are comparable insurance claims rates?

1010

Which new technologies actually improve patient health?

What’s our budget for new equipment?

1111

Can we correlate manufacturing data with customer satisfaction?

Can a robot weld this car better than a person?

12 ©2014 Cloudera, Inc. All rights reserved.12

Expanding Data Requires A New Approach

1980sBring Data to Compute

NowBring Compute to Data

Relative size & complexity

DataInformation-centric

businesses use all data:

Multi-structured, internal & external data

of all types

Compute

Compute

Compute

Process-centric businesses use:

• Structured data mainly• Internal data only• “Important” data only

Compute

Compute

Compute

Data

Data

Data

Data

14 ©2014 Cloudera, Inc. All rights reserved.

The Old Way: Moving Data to ComputeHuge Investment in Specialized Systems that Treat Data as a Commodity

SERVERSMARTSEDWS DOCUMENTS STORAGE SEARCH ARCHIVE

ERP, CRM, RDBMS, MACHINES FILES, IMAGES, VIDEOS, LOGS, CLICKSTREAMS EXTERNAL DATA SOURCES

Major ChallengesMissing Data• Leaving data behind• Risk and compliance• High cost of storage

Complex Architecture• Many special-purpose systems• Moving data around• No complete views

Cost of Analytics• Existing systems strained• No agility• “BI backlog”

Time to Data• Up-front modeling• Transforms slow• Transforms lose data

15 ©2014 Cloudera, Inc. All rights reserved.

The Old Way: Siloed Business FunctionsLack of Coordination Increases Opportunity Costs and Decreases Data Availability

TRANSACTIONALRISKMARKETING LENDING CREDIT CARDS INVESTMENT

CUSTOMER DATATRANSACTIONS MARKET DATA RESEARCHLOGS

BACK OFFICE

Major Challenges

Poor Visibility

Inefficiency

Extreme Cost

Complexity

16 ©2014 Cloudera, Inc. All rights reserved.

The New Way: Bringing Compute to DataMaximize Benefit from All Your Data for Mission-Critical Jobs and Innovation

SERVERS MARTS EDWS DOCUMENTS STORAGE SEARCH ARCHIVE

ERP, CRM, RDBMS, MACHINES FILES, IMAGES, VIDEOS, LOGS, CLICKSTREAMS EXTERNAL DATA SOURCES

Major BenefitsActive Compliance Archive• Full fidelity original data• Indefinite time, any source• Lowest cost storage

Diverse Analytic Platform• Bring applications to data• Combine different workloads on

common data (i.e. SQL + Search)• True analytic agility

Self-Service Exploratory BI• Simple search + BI tools• “Schema on read” agility• Reduce BI user backlog requests

Persistent Storage• One source of data for all analytics• Persist state of transformed data• Significantly faster & cheaper

18

Data ScienceExplorationETLAcceleration

Operational Efficiency Information Advantage

CheapStorage

Business IT

Your Journey to Achieve Full Potential

©2014 Cloudera, Inc. All Rights Reserved.

EDWOptimization

Consolidation 360° View

Advance from Strategy to ROI with Best Practices and Peak Performance

19 ©2014 Cloudera, Inc. All rights reserved.19

From Hadoop to an Enterprise Data Hub

Open SourceScalableFlexibleCost-Effective

✔

Managed ✖Open Architecture ✖Secure and Governed ✖

✔

✔

✔

3RD PARTYAPPS

STORAGE FOR ANY TYPE OF DATAUNIFIED, ELASTIC, RESILIENT, SECURE

CLOUDERA’S ENTERPRISE DATA HUB

BATCHPROCESSING

MAPREDUCE

ANALYTICSQL

IMPALA

SEARCHENGINE

SOLR

MACHINELEARNING

SPARK

STREAMPROCESSINGSPARK STREAMING

WORKLOAD MANAGEMENT YARN

FILESYSTEMHDFS

ONLINE NOSQLHBASE

DATAM

ANAG

EMEN

TCLO

UD

ERA NAVIG

ATOR

SYSTEMM

ANAG

EMEN

TCLO

UD

ERA MAN

AGER

SENTRY, SECURE

20

WEB/MOBILE APPLICATIONS

ONLINE SERVING SYSTEM

ENTERPRISE DATA WAREHOUSE

ENTERPRISE REPORTINGBI / ANALYTICSMACHINE

LEARNINGCONVERGED

APPLICATIONSCLOUDERA MANAGER

META DATA / ETL TOOLS

ENTERPRISE DATA HUB

©2014 Cloudera, Inc. All Rights Reserved.

The Modern Information ArchitectureData Architects System Operators Engineers Data Scientists Analysts Business Users

Customers & End Users

SYS LOGS WEB LOGS FILES RDBMS

21

Customer Success Across Industries

Financial &Business Services

Telecom & Technology

Healthcare &Life Sciences

Media &Information

Retail &Consumer

Energy & Public Sector

©2014 Cloudera, Inc. All rights reserved.

22

BI and Analytics Partners

Enabling The App Store of Big Data

SI, Cloud, MSP Partners

Database Partners

Resellers

Data Integration PartnersHardware Partners

©2014 Cloudera, Inc. All rights reserved.

Intel Confidential

Partnership

● Combine the rich data from MongoDB with other data sources in Cloudera

● Leverage data from Cloudera in operational apps on MongoDB

24

Example - Storage Archive

eCommerce App Storage Archive

● Clicks● Behavior● Etc.

MongoDB Connector for Hadoop

● Profile Data● Product Catalog● Clicks● Etc.

25

MongoDB Connector for Hadoop

Example - ETL

eCommerce App ETLData

Warehouse

● Existing Reporting● Clicks● Behavior● Etc.

● Profile Data● Product Catalog● Clicks● Etc.

26

Example - Recommendation Analysis

eCommerce App Analysis

● CTR Analysis● Patterns

● Better recommendations in real-time

● Profile Data● Product Catalog● Clicks● Etc.

MongoDB Connector for Hadoop

27

Operational Analytical

28

Thank you!Bernard [email protected]. +49 172 692 9837

28

29

I o T E U R O P E A N C I T Y T O U R

S T U T T G A R T