Transcript
Time series Forecasting
Presented at: Big Data Analy2cs Programme NIT Srinagar (15 March 2017) Presenter: Haroon Rashid haroonr@iiitd.ac.in
Help Yourself!
2
Contents
� What is forecas2ng � Why we need it
� Forecas2ng approaches � Forecas2ng evalua2on � Forecas2ng applica2ons
3
Forecasting: Case Scenario - I
500
1000
Aug 28 Aug 29 Aug 30 Aug 31 Sep 01Timestamp
Powe
r (W
) variableTrain
Forecast
4
Forecasting: Case Scenario - II
5
Find carbon footprint for Toyota Matrix with capacity of 1.8 (L), and mileages of 25 (C) and 31 (H)?
Sta2s2cs of 4 cylinder cars
Table: hWps://www.otexts.org/fpp/1/4
Forecasting Types
� Time series Forecas2ng � Data collected at regular intervals of 2me � e.g., Weather, electricity forecas2ng
� Cross-‐Sec2onal Forecas2ng � Data collected at single point in 2me � e.g., Carbon emission, disease predic2on
6
Time series Forecas2ng (Energy)
Assumptions
1. Historical informa2on is available 2. Past paWerns will con2nue in the future
7
100
200
300
400
Aug 03 Aug 04 Aug 05 Aug 06Timestamp
Powe
r (W
)
500
1000
Aug 27 Aug 28 Aug 29 Aug 30Timestamp
Powe
r (W
)
Forecasting Horizon
1. Short-‐term forecas2ng: � Hours to few days ahead
2. Medium-‐term forecas2ng: � Few days to months ahead
3. Long-‐term forecas2ng: � Months to years ahead
8
I. Short term decision II. Long term investment
9
Why electricity forecasting
2
3
4
5
0 20 40 60 80Months (Approx. 6 years)
Pow
er (g
W)
Sol: Backup Generators 1. Renewables (Solar) 2. Diesel Generators 3. Power plants
Img: hWp://www.installeronline.co.uk/brownout-‐one-‐coming/
Forecasting steps
1. Problem defini2on � Purpose (demand supply, Fault detec2on) � Factors (Weather, occupancy, day type)
2. Informa2on gathering � Sense [Sensors: smart meters, temperature, PIR] � Retrieve [Z-‐wave, Bluetooth, Wi-‐Fi, GSM] � Store [Mongo DB, MySQL]
3. Preliminary analysis � Visualiza2ons [R, Python, Matlab, Plotly]
4. Choosing & Fieng models 5. Model evalua2on
10
3. Preliminary analysis
0.5
1.0
0.00.51.01.52.0
0255075
100125
0246
Apartm
ent_1A
partment_2
HVAC
Chiller
StreetLights
Aug 09 Aug 10 Aug 11 Aug 12 Aug 13 Aug 14 Aug 15
Powe
r(KW
)
11
Sunday Saturday
3. Preliminary analysis
500
1000
1500
2000
2012−05−01 2012−11−01 2013−05−01 2013−11−01 2014−05−01 2014−11−01Timestamp
Powe
r (wa
tts)
12
3. Preliminary analysis: Seasonality, trend
13
data
seasonal
trend
remainder
2 4 6 8 10 12
200
400
600
−40
0
40
200
300
400
500
−50−250255075
Time
4. Model fitting: Averaging approach
500
1000
1500
2000
Aug 30 Sep 01 Sep 03 Sep 05 Sep 07 Sep 09Timestamp
Ener
gy (W
)
variableActual
Forecast
Average last 6, RMSE = 251.44
14
Yt+1|1...t =(Y1 + Y2 + ...+ Yt)
length(t)
4. Model fitting: Naïve approach
500
1000
1500
2000
Aug 30 Sep 01 Sep 03 Sep 05 Sep 07 Sep 09Timestamp
Ener
gy (W
)
variableActual
Forecast
Naive, RMSE = 246.14
15
Yt+1|t = Yt
4. Model fitting: Vertical approach
16
0.1
0.2
0.3
0.4
0 5 10 15 20 25Hour of the Day
0.1
0.2
0.3
0.4
0 5 10 15 20 25Hour of the Day
Powe
r (kW
)
Day291623
Forecast of Day 30, ‘15
Weighted Forecasting
5. Model evaluation: Prediction accuracy
Root mean square error (RMSE): Lower is beWer
17
RMSE =
vuut 1
n
nX
i=1
(yi � yi)2n = values
yi = Actual values
yi = Forecast values
y = [713, 711, 652, 522]
y = [751, 713, 711, 652]
RMSE = 73
5 .Model evaluation: Residual diagnostics
−1000
0
1000
Aug 30 Sep 01 Sep 03 Sep 05 Sep 07 Sep 09Timestamp
Res
idua
ls (W
)
Naive − Residuals
18
Demo1
Summary - I
1. Time series forecas2ng 2. Steps in forecas2ng 3. Forecas2ng models: Naïve, averaging 4. Model evalua2on 5. Demonstra2on in R
19
Line fitting
20
●
●
●
●
●
●
●
●
●
●
10
20
30
1 2 3 4 5 6 7 8 9 10X
YLine Fitting
Y = mX + C
Slope = 3, Intercept = 0 Approx.
Linear Regression
●
● ●
●
●
●
●
●●
●
●
●
●
●
●
200
400
600
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Hour of a day
Powe
r (W
)
21
Linear Regression
●
● ●
●
●●
●
●●
●
●
●
●
●
●
200
400
600
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Hour of a day
Powe
r (W
)
Intercept:764.2 Slope:−45.6
22
y = �0 + �1x+ ✏
power = �0 + �1(dayhour) + ✏
Least squares
●
● ●
●
●●
●
●●
●
●
●
●
●
●
200
400
600
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15Hour of a day
Powe
r (W
)
Intercept:764.2 Slope:−45.6
23
NX
i=1
✏
2i =
NX
i=1
(yi � �0 � �1xi)2
Error
Minimize
Evaluation
1. Standard error: Lower is beWer
2. Goodness of fit: Higher is beWer
24
ei = yi � yi
se =
vuut 1
N � 2
NX
i=1
e2i
R2 =
P(yi � y)2P(yi � y)2
R_squared: hWps://goo.gl/Xm5gUd
Evaluation
25
Regression: Scenario II
●● ●
●
●●
●
● ●●
●
●
●
●
● ●●
●
●● ●
●
●
●
●
●●
●
●●
●
●● ●
●●
●
●
●●
●●
●
●
●
●
● ●●
●
● ●●
●
●
●
●
●
● ●●
●
● ● ● ● ●
●
● ●
●
●
500
1000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24Hour of a day
Powe
r (W
)
26
Regression: Scenario II
27
●● ●
●
●●
●
● ●●
●
●
●
●
● ●●
●
●● ●
●
●
●
●
●●
●
●●
●
●● ●
●●
●
●
●●
●●
●
●
●
●
● ●●
●
● ●●
●
●
●
●
●
● ●●
●● ● ● ● ●
●
● ●
●
●
500
1000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24Hour of a day
Powe
r (W
)
Residual error:297.01
y = �0 + �1x+ ✏
Polynomial regression
●● ●
●
●●
●
● ●●
●
●
●
●
● ●●
●
●● ●
●
●
●
●
●●
●
●●
●
●● ●
●●
●
●
●●
●●
●
●
●
●
● ●●
●
● ●●
●
●
●
●
●
● ●●
●● ● ● ● ●
●
● ●
●
●
500
1000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24Hour of a day
Powe
r (W
)
Residual error:174.48
28
y = �0 + �1x+ �1x2 + ...+ �1x
p + ✏
Spline regression
29
●● ●
●
●●
●
● ●●
●
●
●
●
● ●●
●
●● ●
●
●
●
●
●●
●
●●
●
●● ●
●●
●
●
●●
●●
●
●
●
●
● ●●
●
● ●●
●
●
●
●
●
● ●●
●
● ● ● ● ●
●
● ●
●
●
500
1000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24Hour of a day
Powe
r (W
)
Multiple regression
−1
0
1
2
3
Aug 28 Aug 29 Aug 30 Aug 31 Sep 01Timestamp
Powe
r (W
) variableTrain
Forecast
Actual
Residual Test RMSE:1.05
30
Demo 2
Summary - II
1. Regression (Line fieng) 1. Linear 2. Non-‐Linear (Cousin : Splines) 3. Mul2ple
2. Demonstra2on
31
Auto-regressive (AR) model
� Auto-‐regressive model: � Models future values as a func2on of recent past sequen2al values
� Representa2on: An AR model with past p values is denoted as AR(p)
32
Yt = f(Yt�1, Yt�2, ..., Yt�p, ✏t)
Yt = �0 + �1Yt�1 + �2Yt�2 + ...+ �pYt�p + ✏t
Moving Average (MA) model
33
Yt = �0 + ✏t + �1✏t�1 + �2✏t�2 + ...+ �q✏t�q
Yt = f(✏t, ✏t�1, ✏t�2, ..., ✏t�q)
• Moving average model: • Models future values as a func2on of recent
past sequen2al error terms
• Representa2on: An MA model with past q values is denoted as MA(q)
AR MA model
34
Yt = f(✏t, ✏t�1, ✏t�2, ..., ✏t�q, Yt�1, Yt�2, ..., Yt�p)
Yt = �0 + �1Yt�1 + �2Yt�2 + ...+ �pYt�p+
✏t + �1✏t�1 + �2✏t�2 + ...+ �q✏t�q
• Auto regressive Moving Average (ARMA) model: • Models future values as a func2on of recent
past sequen2al values and error terms
• Representa2on: ARMA(p, q) model
Stationary vs. Non stationary time-series
200
300
400
500
0 20 40 60 80Timestamp
Beer
produ
ction (
mega
litres
)
35
400
450
500
550
600
0 20 40 60 80Timestamp
Beer
produ
ction (
mega
litres)
−100
−50
0
50
100
0 20 40 60 80Timestamp
Beer
produ
ction
(mL)
Non-‐sta2onary Non-‐sta2onary
Sta2onary
Differencing (Non-stat. -> Stat.)
36
−100
−50
0
50
100
0 20 40 60 80Timestamp
Beer
pro
duct
ion
(mL)
200
300
400
500
0 20 40 60 80Timestamp
Beer
pro
duct
ion
(mL)
Yt = Yt � Yt�1
Differencing order (d): Number of 2mes differencing is done
ARIMA model
37
ARIMA is defined by a tuple (p, d, q)
Auto-‐Regressive Integrated Moving Average
AR I MA
Yt = �0 + ✏t + �1✏t�1 + �2✏t�2 + ...+ �q✏t�q
Yt = �0 + �1Yt�1 + �2Yt�2 + ...+ �pYt�p + ✏tYt = Yt � Yt�1
[Order p] [Order d]
[Order q]
ACF/PACF plots
1. Auto-‐Correla2on Func2on (ACF) Plot: � Correla2on coefficients of 2me-‐series at different lags � Defines q order of MA model
2. Par2al Auto-‐correla2on Func2on (PACF) Plot:
� Par2al correla2on coefficients of 2me series at different lags
� Defines p order of AR model
38
ACF/PACF plots
39
500
1000
Aug 29 Aug 30 Aug 31 Sep 01Timestamp
Powe
r (W
)
−0.25
0.00
0.25
0.50
5 10 15Lag
PACF
−0.25
0.00
0.25
0.50
5 10 15Lag
ACF
Data
PACF plot
ACF plot
Model evaluation
� AIC/BIC � Residual errors
40
Demo 3
Seasonality
41
data
seasonal
trend
remainder
2 4 6 8 10 12
200
400
600
−40
0
40
200
300
400
500
−50−250255075
Time
SARIMA (Seasonal ARIMA)
� SARIMA(p,d,q) (P,D,Q):
� Order (P,D,Q) handles the seasonality part
42
Complete Forecasting pseudocode
1. Visualize 2me-‐series data 2. If data is noisy
� Apply averaging or naïve model
3. If data is not sta2onary � First, sta2onarize data using differencing � Next, apply any 2me series model
4. If data is already sta2onary � Apply any 2me series model
43
Challenges : Outliers
44
0.5
1.0
0 5 10 15 20 25Hour of the Day
Powe
r (kW
)
Day26272829
Anomalous usage
0.25
0.50
0.75
1.00
1.25
0 5 10 15 20 25Hour of the Day
ForecastActual
Day 30
Challenges: Domain Knowledge
45
100
200
300
400
0 5 10 15 20 25Hour of the Day
Powe
r
Day291623
250
500
750
0 5 10 15 20 25Hour of the Day
Powe
r (w)
ForecastActual
1
500
1000
0 5 10 15 20 25Hour of the Day
Powe
r (w)
Day26272829
250
500
750
1000
1250
0 5 10 15 20 25Hour of the Day
Powe
r (w)
ForecastActual
2
Application: Anomaly Detection
46
References
1. Book: Forecas2ng principles and prac2ce, hWps://www.otexts.org/fpp
2. Understanding seasonality and trend with code: hWps://anomaly.io/seasonal-‐trend-‐decomposi2on-‐in-‐r/
3. ARIMA ordering: hWps://people.duke.edu/~rnau/411arim3.htm
4. Time series Forecas2ng theory: hWps://www.youtube.com/watch?v=Aw77aMLj9uM
5. Book: Applied predic2ve modeling by kuhn et al. 6. Book: An introduc2on to sta2s2cal learning by
Gareth et al.
47
Annexure
48
Multidimensional Scaling (MDS)
0
300
600
900
0 5 10 15 20 25Hour of the day
Powe
r (wa
tts)
Day1Day2Day3Day4
49
●
●
●
●
−1500 −1000 −500 0 500 1000 1500
−600
−200
020
060
0
MDS Dimension−1
MD
S D
imen
sion−2
Day1
Day2
Day3
Day4
Dissimilarity/Distance Matrix
Apply MDS
Day1 Day2 Day3 Day4
Day1 0000 2789 1194 2699
Day2 2789 0000 2516 0254
Day3 1194 2516 0000 2371
Day4 2699 0254 2371 0000
dist(dayx
, dayy
) =qP
n=24i=1 (dayi
x
� dayiy
)2
Regression coefficient
�1 =
PNi=1(yi � y)(xi � x)PN
i=1(xi � x)2
�0 = y � �1x
50
Forecasting band
51
200
204
208
0 5 10 15 20 25Hour of the day
Powe
r (wa
tts)
Actual_UsageForecast
top related