Keynote Address at 2013 CloudCon: Future of Big Data by Richard McDougall (Infrastructure CTO, VMware)

© 2009 VMware Inc. All rights reserved

Big Data: Now and in the Future

Is Your Cloud Ready for Big Data?

Richard McDougall

CTO, Application and Storage Services

2

Not Just for the Web Giants – The Intelligent Enterprise

3

Real-time analysis allows instant understanding of

market dynamics.

Retailers can have intimate understanding of their

customers needs.

Market Segment Analysis Personalized Customer Targeting`

4

The Emerging Pattern of Big Data Systems: Retail Example

Real-TimeStreams

Exa-scale Data Store

Parallel DataProcessing

Real-TimeProcessing

MachineLearning

Data Science

Cloud Infrastructure

5

Storage: Plan for Peta-scale Data Storage and Processing

2000 2003 2006 2009 2012 20150.01

0.1

1

10

100

1000

Online Apps

AnalyticsPB ofData

Analytics Rapidly Outgrows Traditional Data Size by 100x

6

Unprecedented Scale

“Data transparency, amplified by Social Networks

generates data at a scale never seen before”

- The Human Face of Big Data

We are creating an Exabyte of data every minute in 2013

Yottabyte by 2030

7

A single GE Jet Engine produces

10 Terabytes of data in one hour – 90 Petabytes per year.

Enabling early detection of faults, common mode failures, product engineering feedback.

Post Mortem Proactively Maintained Connected Product

8

The Emerging Pattern of Big Data Systems: Manufacturing

Exa-scale Data Store

Parallel DataProcessing

Real-TimeProcessing Machine

Learning

Data Science

Cloud Infrastructure

Real-TimeSensor

Analytics SupportProduct

Engineering

9

Cloud Infrastructure Supports Mixed Big Data Workloads

MachineLearning HadoopReal-Time

Analytics

Cloud Infrastructure

MachineLearning

Hadoop

Real-TimeAnalytics

Management

Network/Security

Storage/Availability

Compute

10

Software-defined Datacenter: Compute

Agility / Rapid deployment

Lower Capex

Isolation for resource control and security

1

2

3

Operational efficiency4

Management

The Core Values of Virtualization Apply to Big Data

Network/Security

Storage/Availability

Compute

11

Strong Isolation between Workloads is Key

Hungry Workload 1

Reckless Workload 2

NosyWorkload 3

Cloud Infrastructure

12

Management

Software-defined Datacenter: Storage

Requirements of Next Generation Storage

Network/Security

Storage/Availability

Compute

10x lower cost of storage

Handle explosive data growth

Support a variety ofapplication types

1

2

3

Solve the privacy andsecurity issues4

13

Software-defined Storage Enables Fundamental Economics

0.5 1 2 4 8 16 32 64 128 $-

$0.50

$1.00

$1.50

$2.00

$2.50

$3.00

$3.50

$4.00

$4.50

$5.00

$5.50

Cost per GB

Petabytes Deployed

TraditionalSAN/NAS

DistributedObject

StorageHDFSMAPRCEPH

Scale-out NASIsilon, NTAP

14

Management

Software-defined Datacenter: Network and Security

Automate secure network provisioning

Network & Security Requirements for Big Data

Network/Security

Storage/Availability

Compute

3

New high bandwidth network designs

1

Leverage Software-defined network security

2

15

Big Data Requires Extreme Bandwidth

16

Summary

17

Customers Winning from Consolidated Big Data Platforms

“Dedicated hardware makes no sense”

“Software-defined Datacenter enables rapid deployment multiple tenants and labs”

“Our mixed workloads include Hadoop, Database, ETL and

App-servers”

“Any performance penalties are minor”Management

Network/Security

Storage/Availability

Compute

18

Cloud Infrastructure is Ready for Big Data – Are you?

Cloud Infrastructure

19

Q&A