Smart Meter Data Analytic Using Hadoop Omkar Nibandhe and Abhishek Korpe students
Jan 27, 2015
Smart Meter Data Analytic Using Hadoop
Omkar Nibandhe and Abhishek Korpestudents
SMART METER DATA ANALYTICS
(SMDA)USING HADOOP
By :
Omkar Nibandhe ( Student )
Abhishek Korpe ( Student )
03-0
4-20
14SM
DA u
sing
HAD
OO
P
2
Why Smart Meter Data Analytics ?
03-0
4-20
14SM
DA u
sing
HAD
OO
P
3
What are SMART METERS ?
MIF
• Track and store the amount of energy used.• Send the collected data to the Energy
Distribution company server at regular time intervals.
House/Industry
Smart Meter Server
03-0
4-20
14SM
DA u
sing
HAD
OO
P
4
Advantages
• Service Provider• Demand-Response• Time of use tariff• Load Profile Analysis • Theft Detection• Billing Accuracy
• Customers• Usage Pattern• Billing Accuracy• Convenience in change
of service provider
03-0
4-20
14SM
DA u
sing
HAD
OO
P
5
Confronting the data deluge
• Rate Generation – 15 minutes.• For single meter – 3000 readings/month (approx ).
• For 1 million meters – 36 Billion readings/year (approx).
• Annual Growth – 13% ( 2010 – 2015 ).
• Total Shipment – 460.9 Million Smart Meters.
Source: Build smart metering solutions with IBM Informix TimeSeries
03-0
4-20
14SM
DA u
sing
HAD
OO
P
6
Capitalizing on the unique value of Hadoop
Solution ?
• Reducing data load times.• Improving query performance.• Massive Scalability.
03-0
4-20
14SM
DA u
sing
HAD
OO
P
7
Demand - Response
03-0
4-20
14SM
DA u
sing
HAD
OO
P
8
Time of use Tariff
03-0
4-20
14SM
DA u
sing
HAD
OO
P
9
Load Profile Analysis
Using hadoopUsing hadoop
load
time
03-0
4-20
14SM
DA u
sing
HAD
OO
P
10
Data Flow Diagram
Predictive Analysis
( FLUME )
Load Profile Analysis
03-0
4-20
14SM
DA u
sing
HAD
OO
P
11
Hadoop Cluster in LAB 412
Masters
Slaves
Slave1
Slave7 Slave8 Slave9 Slave11
Slave6Slave5Slave4Slave3Slave2
Slave10
Slave19Slave18Slave17Slave16Slave15Slave14Slave13
Slave12
Slave20 Slave21
03-0
4-20
14SM
DA u
sing
HAD
OO
P
12
LAB 412 (MESCOE Pune, India)
03-0
4-20
14SM
DA u
sing
HAD
OO
P
13
SMDA - NameNode
03-0
4-20
14SM
DA u
sing
HAD
OO
P
14
SMDA - SecondaryNameNode
03-0
4-20
14SM
DA u
sing
HAD
OO
P
15
SMDA - JobTracker
03-0
4-20
14SM
DA u
sing
HAD
OO
P
16
SMDA – Input ( .MIF )
03-0
4-20
14SM
DA u
sing
HAD
OO
P
17
1392 19501 0.157
SMDA - Output
03-0
4-20
14SM
DA u
sing
HAD
OO
P
18
Test Job 1
03-0
4-20
14SM
DA u
sing
HAD
OO
P
19
Test Job 2 ( Combiner )
03-0
4-20
14SM
DA u
sing
HAD
OO
P
20
Future Scope• Analysis of -• Customer segmentation.• Customer behavior.• Meter ping commands. • Outage management.• Power quality.• Extending data point(s) : weather, geographical location, family
consumption, etc.
03-0
4-20
14SM
DA u
sing
HAD
OO
P
21
Thanks to ….• Irish Social Science Data Archive ( ISSDA ).
• Rahul Khinvasara – Director, zCon Solutions Pvt. Ltd.
• Modern Education Society’s College of Engineering (MESCOE).• Prof. Balaji Bodkhe – Guide, MESCOE.• Prof. N. Shaikh – Head of Computer Department, MESCOE.• Prof. A. Hake – Vice Principal, MESCOE.• Prof. P. Raut – Administrative Head, MESCOE.
03-0
4-20
14SM
DA u
sing
HAD
OO
P
22
Any Suggestions ??
03-0
4-20
14SM
DA u
sing
HAD
OO
P
23
Thank You.
03-0
4-20
14SM
DA u
sing
HAD
OO
P
24
Any Questions?
03-0
4-20
14SM
DA u
sing
HAD
OO
P
25