Are You Ready?
Adam Wray
CEO – Basho Technologies
FIREFOX iKETTLE
FITBIT
GOOGLE GLASS
APPLE WATCH
SMART THINGS
SENSORS IN AIRPLANE ENGINES
“The Internet of Things is all about catching things in progress rather than waiting until after the fact to analyze the data,” says Paul Maritz, CEO of Pivotal
*Source IDC
THE DATA EXPLOSION
2013
4.4 ZB2020
44 ZB
If the Digital Universe were
represented by the memory in a stack of tablets…
WHAT IS DRIVING THE
Gaining insights from previously
unconnected data
Connecting people for work and play
Innovation of connected devices
DATA EXPLOSION?
80 MILLION PEOPLE1300 MESSAGES A SECOND
24 Hours a Day / 365 Days a Year
Information is requested and amended more
than 2.6 BILLION times a year
Has enabled over 42 MILLION Summary
Care Records to be created and stored
Has transmitted over 1.3 BILLION
prescription messages
NHS Data Spine
Smart Thermostat Market is expected to grow
from $146.9 million in revenue in 2014 to $2.3
billion by 2023.
-Navigant
THE CONNECTED HOME
Store historical date for analytics
Store User Schedules
Sessions Storage for Connected
Users
1.5 TRILLION RECORDS PER DAY
400% Year Over Year Increase
Ability to Monitor entire IT
environment from Single Portal
Array of Real-Time Statistics
and Insight
Centralize & Correlate Events,
Alerts, and Notifications
Cloud-based management tools and data
analytics usually only accessible to larger
companies.
Millions of transactions each day.
POS FOR OVER 20K SMALL
BUSINESSES
Quick Service
Retail
Restaurant & Bar
20 TERABYTES OF DATA PER DAY
BILLIONS OF MOBILE DEVICES
10 BILLION data transactions a
day – 150,000 a second – Apple
Forecasting 2.8 BILLION locations
around the world
Generates 4GB OF DATA every
secondWe’re focusing on
helping people make
better decisions with
the weather.
WEATHER FORECAST
PREDICTS WALMART’S SALES
Ideal BERRY weather turns
out to be low wind with
temperatures below 80
degrees.
People are more likely to
eat STEAK when it's warm
out with higher winds but
no rain, but not if it gets too
hot.
WHAT ARE THE IMPLICATIONS?
Enterprises must choose: Modernize or Sink!
• Adopt technologies to successfully manage data generated
by new data sources and consumed by users accessing
complex data types from around the world, 24 hours a day.
• Organizations must make plans to handle data ingestion,
data storage, and analytics in order to bring value to the
business.
• Increasingly look to distributed systems to help manage the
influx of traffic, and ease current operational challenges.
DISTRIBUTED WORKLOADS
App App App App
Virtualization
Server
App
Aggregation
Server Server Server Server
Client-Server Era:
SMALL APPS
BIG SERVERS
ONE LOCATION
Cloud Era:
BIG APPS
SMALL SERVERS
MANY LOCATIONS
Everything works
at small scale
What happens when
something goes wrong
The customer
experience matters
WHY DISTRIBUTED SYSTEMS?
Scale out, up and down predictably
and linearly
Survive server, network or data
center failures
Data locality enables data operations
close to end-users
DISTRIBUTED SYSTEMS
DEVELOPERS OPERATIONS CUSTOMERS SALES
Challenge 1 – Isolation of Data
The growing hype surrounding data lakes is causing substantial confusion in the information management space
Gartner
Challenge 2 – Consistency of Data
To Be or Not To Be
Consistent?
Understand the
question…
C A
PX
The CAP TheoremDr Eric Brewer
Describes the trade-offs involved in distributed systems
Challenge 3 – Data Gravity
Apps
Services
Lower Latency
&
Higher Bandwidth
Growth
Over Time Data
“Perhaps the biggest challenge is that the IoT has the potential to generate orders of magnitude more data than any other source in existence today. So, in the world of the IoT we will test the limits of ‘big.’”
Bill Franks, Chief Analytics Officer for Teradata
Putting Challenges In Perspective
DISTRIBUTED SYSTEMS AND NoSQL
• Most powerful tool in distributed systems is NoSQL
• Efficient with active workloads
• More cost effective and scalable than RDBMS
• Key Value most flexible and preferred
Usage in web applications for blogs
Session management
Chat applications
Social network management
• Others – Document, Graph, Big Table
• Riak most operationally efficientMarket is Expected to Reach
$4.2 Billion, Globally,
by 2020 - Allied Market Research
Two Sigma's vision has been to develop
technological innovations that intelligently
analyze the world's data to consistently
deliver value for their clients.
MANAGING $18B WITH TRADE DATA
Modeling teams work
with vast data sets –
from corporate
earnings to news and
weather reports.
Seek to synthesize
quantitative and
qualitative information
sources to generate
new scientific insights
with systematic
precision.
Wall Street Journal Front Page April 1, 2015
5TB per Day in Real-Time Growing > 50% per Year
Over 3K Customers, 5K Appliances, 100K+ Data Sources
on Customer Networks
SECURITY AS A SERVICE