Charting the Path to Intelligent Operations with Machine Learning
Atakan Cetinsoy VP - Predictive Applications
21st Century Megatrends
As the world population is headed to 10 billion:
• Intensifying scramble for scarce resources
• Growing urbanization and diversity
• Social media and the shifting balance of power
SUSTAINABILITY
PRODUCTIVITY
ENGAGEMENT
Utility Industry Trends
• Evolving energy portfolio
• Transition to distributed generation schemes
• Efficiency as a “New” energy resource
• Growing smart meter infrastructure
• Dynamic pricing and demand response
The Connected World
We’re here!
SOURCE: Cisco
The Industrial Internet
SOURCE: General Electric
• Hypothetical 1% efficiency gain via IoT technology.
Savi
ngs
(in B
illion
s U
SD)
Sensor Data and Predictive Apps
SOURCE: Forrester
SOURCE: Joseph Sirosh
Case Study: Digital Cows
SOURCE: Fujitsu.com
IoT Time Series Data
Sensor Time +7 +35 +50 BLOB
101 15:00 N/A N/A N/A {…}
102 15:00 N/A N/A N/A {…}
102 15:01 N/A N/A N/A {…}
103 15:01 11 20 N/A {…}
103 15:02 N/A N/A 33 {…}
1 Minute Time Window
Offset in Seconds
• Wide row structure with possibly 1000s of measurements
• 100M to 1 billion data points per second can be processed!
• Compacted into BLOB format stored as a single value
SOURCE: MapR
Big Data or Big Hype?• Data that is
• Too big to fit on a single server
• Too unstructured to fit into rows and columns
• Too continuos to fit into an EDW
• “Size matters” but actionable insights take the prize.
Data Driven Decision Making
Evolution of Analytics
Attribute Traditional Analytics Analytics 2.0
Data Type Rows and Columns Unstructured
Volume Up to TBs Up to PBs
Flow Static Pool Continuos
Technology EDW + SQL Open Source + Machine Learning
Analysis Descriptive, Hypothesis-based
Predictive, Machine Learned
Purpose Internal Decision Support
Data-driven Products/Services
SOURCE: Thomas H. Davenport
Includes everything in Traditional Analytics plus the following.
Machine Learning?
• “Machine Learning is the field of study that gives computers the ability to learn without being explicitly programmed.” — Prof. Arthur Samuel
The Need for Machine Learning• Can you find any pattern in this tiny data set?
• Now imagine millions of rows and thousands of columns of it!
The Need for Data-driven Decisions
• Human intuition is poor
• Human judgement is biased
• Human reasoning is causal and not statistical
• Machine Learning is a tool to help people make smarter, unbiased, more effective data-driven decisions.
What is a Data Scientist?
Industry Subject-matter Expertise
Computer Science and/or Hacking Skills
Math and Statistics Knowledge
Machine Learning
Traditional Research
Data Science
SOURCE: Drew Conway
Future of Machine Learning
• “Machine Learning is becoming a new abstraction layer of the computing infrastructure.”
Tushar Chandra, Principal Engineer — Google Research
BigMLAn end-to-end machine learning platform that is
• Builds interpretable machine learning models that address the vast majority of predictive tasks.
• Accessible to the entire organization to make data-driven decisions.
• Provides a public API so that application developers can build predictive applications.
• Cloud-born solution that provides instant access and instant scale.
CONSUMABLE
PROGRAMMABLE
SCALABLE
Predictive Modeling Best Practices• Business objective and
predictive model alignment
• Proof of concept based on sampled data
• Model validation with proper accuracy measures
• Transparent vs. “Black Box” algorithms
Interpretable Predictive Models
Model Variable Contribution
Model Evaluation
Predictive Apps for Utilities• Operational
• Accurate and Granular Load Forecasting
• Network Outage Predictions
• System Failure Predictions
• Demand Response Optimization
• Marketing
• Customer Churn Prediction
• Pricing Response Prediction
• Energy Efficiency
• Household Level Predictive Analytics