Top Banner
a presentation at the UNITED NATIONS STATISTICAL COMMISSION by DR. MATT WOOD introducing BIG DATA ANALYTICS
82

Matt Wood, Chief Data Scientist, Amazon Web Services

Feb 14, 2017

Download

Documents

vuthu
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Matt Wood, Chief Data Scientist, Amazon Web Services

a presentation at the UNITED NATIONS STATISTICAL COMMISSION

by

DR. MATT WOOD

introducing

BIG DATA ANALYTICS

Page 2: Matt Wood, Chief Data Scientist, Amazon Web Services

Hello.

Page 3: Matt Wood, Chief Data Scientist, Amazon Web Services

Thank you.

Page 4: Matt Wood, Chief Data Scientist, Amazon Web Services

IData, data everywhere

Page 5: Matt Wood, Chief Data Scientist, Amazon Web Services

I IIData, data everywhere

Data timeline

Page 6: Matt Wood, Chief Data Scientist, Amazon Web Services

I II IIIData

securityData, data everywhere

Data timeline

Page 7: Matt Wood, Chief Data Scientist, Amazon Web Services

I II III IVData

movementData, data everywhere

Data security

Data timeline

Page 8: Matt Wood, Chief Data Scientist, Amazon Web Services

I II III IVData

movementData, data everywhere

Data security

Data timeline

0.Amazon web

Services

Page 9: Matt Wood, Chief Data Scientist, Amazon Web Services

Compute, storage & databases.

Page 10: Matt Wood, Chief Data Scientist, Amazon Web Services

Retail Merchantservices

Web services

Page 11: Matt Wood, Chief Data Scientist, Amazon Web Services

Blinding flash of the obvious.

Page 12: Matt Wood, Chief Data Scientist, Amazon Web Services

Available.

Page 13: Matt Wood, Chief Data Scientist, Amazon Web Services

Low cost.

Page 14: Matt Wood, Chief Data Scientist, Amazon Web Services

Flexible.

Page 15: Matt Wood, Chief Data Scientist, Amazon Web Services

1.3 trillion objects835k peak requests/second

Page 16: Matt Wood, Chief Data Scientist, Amazon Web Services

300 government agencies.1,500 educational institutions.

Page 17: Matt Wood, Chief Data Scientist, Amazon Web Services

Data, data everywhereI

Page 18: Matt Wood, Chief Data Scientist, Amazon Web Services

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

Page 19: Matt Wood, Chief Data Scientist, Amazon Web Services

Cost of data generation is falling.

Page 20: Matt Wood, Chief Data Scientist, Amazon Web Services

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

lower cost,increased throughput

Page 21: Matt Wood, Chief Data Scientist, Amazon Web Services

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

highly constrained

Page 22: Matt Wood, Chief Data Scientist, Amazon Web Services

Gap.

Page 23: Matt Wood, Chief Data Scientist, Amazon Web Services

1990 2000 2010 2020

The Data Analysis Gap

Enterprise Data Data in Warehouse

Gartner: User Survey Analysis: Key Trends Shaping the Future of Data Center Infrastructure Through 2011 IDC: Worldwide Business Analytics Software 2012–2016 Forecast and 2011 Vendor Shares

Generated data

Available for analysis

Data volume

Page 24: Matt Wood, Chief Data Scientist, Amazon Web Services

Utility.

Page 25: Matt Wood, Chief Data Scientist, Amazon Web Services

Remove constraints.

Page 26: Matt Wood, Chief Data Scientist, Amazon Web Services

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

highly constrained

Page 27: Matt Wood, Chief Data Scientist, Amazon Web Services

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

Page 28: Matt Wood, Chief Data Scientist, Amazon Web Services

Close the gap.

Page 29: Matt Wood, Chief Data Scientist, Amazon Web Services

Technologies and techniques for working productively with data, at any scale.

Page 30: Matt Wood, Chief Data Scientist, Amazon Web Services

Data timelineII

Page 31: Matt Wood, Chief Data Scientist, Amazon Web Services

Lots of data.Lots of users.Lots of uses.

Lots of locations.

Page 32: Matt Wood, Chief Data Scientist, Amazon Web Services

Cost.

Page 33: Matt Wood, Chief Data Scientist, Amazon Web Services

Multipliers.

Page 34: Matt Wood, Chief Data Scientist, Amazon Web Services

Generation challenge.

Page 35: Matt Wood, Chief Data Scientist, Amazon Web Services

Analytics challenge.

Page 36: Matt Wood, Chief Data Scientist, Amazon Web Services

Co-evolution.

Page 37: Matt Wood, Chief Data Scientist, Amazon Web Services

Co-evolution.

software

Page 38: Matt Wood, Chief Data Scientist, Amazon Web Services

Co-evolution.

software

utility computing

Page 39: Matt Wood, Chief Data Scientist, Amazon Web Services

Hadoop.

Page 40: Matt Wood, Chief Data Scientist, Amazon Web Services

Availability challenge.

Page 41: Matt Wood, Chief Data Scientist, Amazon Web Services

Beautiful and unique.

Page 42: Matt Wood, Chief Data Scientist, Amazon Web Services

Snowflake Statistics

Page 43: Matt Wood, Chief Data Scientist, Amazon Web Services

Data has gravity.

Page 44: Matt Wood, Chief Data Scientist, Amazon Web Services

Move data to users.

Page 45: Matt Wood, Chief Data Scientist, Amazon Web Services

Move data to users.X

Page 46: Matt Wood, Chief Data Scientist, Amazon Web Services

Move tools to data.

Page 47: Matt Wood, Chief Data Scientist, Amazon Web Services

Place data where it can be easily consumed.

Page 48: Matt Wood, Chief Data Scientist, Amazon Web Services
Page 49: Matt Wood, Chief Data Scientist, Amazon Web Services
Page 50: Matt Wood, Chief Data Scientist, Amazon Web Services
Page 51: Matt Wood, Chief Data Scientist, Amazon Web Services
Page 52: Matt Wood, Chief Data Scientist, Amazon Web Services
Page 53: Matt Wood, Chief Data Scientist, Amazon Web Services

Reusable environment.

Page 54: Matt Wood, Chief Data Scientist, Amazon Web Services

Always more people outside your team, than within it.

Page 55: Matt Wood, Chief Data Scientist, Amazon Web Services

Technologies and techniques for working productively with data, at any scale.

Page 56: Matt Wood, Chief Data Scientist, Amazon Web Services

Data security.III

Page 57: Matt Wood, Chief Data Scientist, Amazon Web Services

Security is our number one priority.

Page 58: Matt Wood, Chief Data Scientist, Amazon Web Services

Shared responsibility.

Page 59: Matt Wood, Chief Data Scientist, Amazon Web Services
Page 60: Matt Wood, Chief Data Scientist, Amazon Web Services

Choose your region.

Page 61: Matt Wood, Chief Data Scientist, Amazon Web Services

Availability zones.

Page 62: Matt Wood, Chief Data Scientist, Amazon Web Services

ITAR

FIPS 140-2

MPAAISO 27001

SOC 2 ISAE 3402 PCI DSS

HIPAA

FISMA Moderate

Page 63: Matt Wood, Chief Data Scientist, Amazon Web Services
Page 64: Matt Wood, Chief Data Scientist, Amazon Web Services

Virtual Private Cloud.

Page 65: Matt Wood, Chief Data Scientist, Amazon Web Services

Network isolated environment.

Page 66: Matt Wood, Chief Data Scientist, Amazon Web Services

Data movement.IV

Page 67: Matt Wood, Chief Data Scientist, Amazon Web Services

“How do I get my data into the cloud?”

Page 68: Matt Wood, Chief Data Scientist, Amazon Web Services

Generated and stored in the AWS cloud.

Page 69: Matt Wood, Chief Data Scientist, Amazon Web Services

Inbound transfer if free.

Page 70: Matt Wood, Chief Data Scientist, Amazon Web Services

Multipart upload.

Page 71: Matt Wood, Chief Data Scientist, Amazon Web Services

Physical media.

Page 72: Matt Wood, Chief Data Scientist, Amazon Web Services

AWS Direct Connect.

Page 73: Matt Wood, Chief Data Scientist, Amazon Web Services

1Gbps or 10Gbps

Page 74: Matt Wood, Chief Data Scientist, Amazon Web Services

Built in AZ replication.

Page 75: Matt Wood, Chief Data Scientist, Amazon Web Services

Regional replication.

Page 76: Matt Wood, Chief Data Scientist, Amazon Web Services

aws.amazon.com

Page 77: Matt Wood, Chief Data Scientist, Amazon Web Services

IData, data everywhere

Page 78: Matt Wood, Chief Data Scientist, Amazon Web Services

I IIData, data everywhere

Data timeline

Page 79: Matt Wood, Chief Data Scientist, Amazon Web Services

I II IIIData

securityData, data everywhere

Data timeline

Page 80: Matt Wood, Chief Data Scientist, Amazon Web Services

I II III IVData

movementData, data everywhere

Data security

Data timeline

Page 81: Matt Wood, Chief Data Scientist, Amazon Web Services

Thank you.