Designing and aggregating experts for energy demand forecasting

DESIGNING AND AGGREGATING EXPERTS FOR ENERGY DEMAND FORECASTING

Yannig Goude EDF R&DGeorges Oppenheim UPEM & Paris 11Pierre Gaillard EDF R&D, HEC Paris-CNRSGilles Stoltz HEC Paris-CNRS

SESO 2014 International Thematic Week“Smart Energy and Stochastic Optimization''

June 23 to 27, 2014

| 2

INDUSTRIAL CHALLENGES

Smart grids More and more « real time » data (ex: linky, 1million meters in 2016) Demand response (new tariffs, real time pricing…) New communication tools with customers (webservice, on-line reporting….)

Renewables energy development A more and more probabilistic context

Opening of the electricity market: Losses/gains of customers

Sensors data: Production/consumption sites Smart home, internet of things

New usages/tariffs: Electric cars Heat pumps, smart phones, battery charge, computers, flat screens…. Demand response, special tariffs (time varying…)

| 3

STATISTICAL CHALLENGES Large scale data sets

Parallelizing statistical algorithms Complex data analysing: heteregonous spatial/temporal sampling, different sources/nature of data

Adaptivity Non-parametric models, fonctional data analysis Model selection, data driven penalty…

Sequential estimation Break detection On-line update, sequential data treatment (data flow, connection to big data) Aggregation with on-line weigths

Multi-scale models Multi-horizon models Multi level data on the grid

Data mining of time series

Large scale simulations Simulation platform, parallel processing

| 4

CONTRIBUTIONS

Large scale data sets GAM parallel processing EDF R&D/IBM simulation platform

Adaptivity GAM models, automatic GAM selection functional data analysis (CLR: curve linear regression, KWF: kernel wavelet fonctional)

Sequential learning: Adaptive GAM Combining forecasts

Spatio temporal/multi-scale models, complex data « Downscaling » electricity consumption: link INSEE (socio-demographic, census) data to local electricity

consumption (meters, grid data) and meteo data EDF R&D/IBM simulation platform

| 5

LOAD FORECASTING

Electricity consumption is the main entry for optimizing the production units

| 6

ELECTRICITY CONSUMPTION DATA

Trend

Yearly, Weekly, Daily cycles

| 7

ELECTRICITY CONSUMPTION DATA

Meteorological events

Special days

| 8

GAM (GENERALIZED ADDITIVE MODELS)

A good trade-off complexity/adaptivity

Publications Application on load forecasting

• A. Pierrot and Y. Goude, Short-Term Electricity Load Forecasting With Generalized Additive Models Proceedings of ISAP power, pp 593-600, 2011.

• R. Nédellec, J. Cugliari and Y. Goude, GEFCom2012: Electricity Load Forecasting and Backcasting with Semi-Parametric Models, International Journal of Forecasting , 2014, 30, 375 - 381.

GAM « parallel »: BAM (Big Additive Models)• S.N. Wood, Goude, Y. and S. Shaw, Generalized additive models for large datasets, Journal of Royal Statistical

Society-C, 2014.

Adaptive GAM (forgetting factor)• A. Ba, M. Sinn, Y. Goude and P. Pompey, Adaptive Learning of Smoothing Functions: Application to Electricity Load

Forecasting Advances in Neural Information Processing Systems 25, 2012, 2519-2527.

| 9

GAM

| 10

GEFCOM COMPETITION

20 substations on the US grid 11 temperature series hourly data from january 2004 to june 2008 9 weeks to predict : 8 from 2005 to 2006, and the one following the train set (no temperature forecast available)105 teams

?

One issue : no localisation information

http://www.kaggle.com/c/global-energy-forecasting-competition-2012-load-forecasting

| 11

Nedellec, R.; Cugliari, J. & Goude, Y.GEFCom2012: Electric load forecasting and backcasting with semi-parametric modelsInternational Journal of Forecasting , 2014, 30, 375 - 381

GEFCOM COMPETITION

| 12

GEFCOM COMPETITION

| 13

CURVE LINEAR REGRESSION

Regressing curves on curves Dimension reduction, SVD of cov(Y,X) , selection with penalised model selection Scale to big data sets (SVD+linear regression)

Publications Application on electricity load forecasting

• H. Cho, Y. Goude, X. Brossat & Q. Yao, Modeling and Forecasting Daily Electricity Load Curves: A Hybrid Approach Journal of the American Statistical Association, 2013, 108, 7-21.

• Cho, H.; Goude, Y.; Brossat, X. & Yao, Q, Modelling and forecasting daily electricity load using curve linear regressionsubmitted to Lecture Notes in Statistics: Modeling and Stochastic Learning for Forecasting in High Dimension .

Clusturing functional data• H. Cho, Y. Goude, X. Brossat & Q. Yao, Clusturing for curve linear regression, technical report, 2013.

| 14

CURVE LINEAR REGRESSION

| 15

OTHER MODELS

Random forest: a popular machine learning method for classification/regression• Breiman, L., . Random Forests, Machine Learning, 45 (1), 2001.

KWF (Kernel Wavelet Functional): another approach for functional data forecasts• See: Antoniadis, A., Brossat, X., Cugliari, J., Poggi, J., Clustering functional data using wavelets. In: Proceedings of

the Nineteenth International Conference on Computational Statistics(COMPSTAT), 2010.• Antoniadis, A., Paparoditis, E., Sapatinas, T., A functional wavelet–kernel approach for time series prediction.

Journal of the Royal Statistical Society: Series B 68(5), 837–857, 2006.

Temperature

Bank Holiday

Lag Load

<6°C

>6°C

yes

no

>55GW

<55GW

http://luc.devroye.org/BRUCE/brucepics.html

| 16

SEQUENTIAL AGREGATION OF EXPERTS

| 17

SEQUENTIAL AGREGATION OF EXPERTS

| 18

SEQUENTIAL AGREGATION OF EXPERTS

Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press (2006)

| 19

EXPONENTIALLY WEIGHTED AVERAGE FORECASTER (EWA)

| 20

EXPONENTIATED GRADIENT FORECASTER (EG)

| 21

OTHER ALGORITHMS

Theoretical calibrationWorks well in practice

Gaillard, P., Stoltz, G., van Erven, T.: A second-order bound with excess losses (2014).ArXiv:1402.2044

| 22

APPLICATION ON LOAD FORECASTING

Publications• M. Devaine, P. Gaillard, Y. Goude & G. Stoltz, Forecasting electricity consumption by aggregating specialized experts

- A review of the sequential aggregation of specialized experts, with an application to Slovakian and French country-wide one-day-ahead (half-)hourly predictions Machine Learning, 2013, 90, 231-260.

• Gaillard, P. & Goude, Y., Forecasting electricity consumption by aggregating experts; how to design a good set of experts to appear in Lecture Notes in Statistics: Modeling and Stochastic Learning for Forecasting in High Dimension, 2013.

initial « heterogenous » experts: GAM Kernel Wavelet Functional Curve Linear Regression Random Forest

Designing a set of experts from the original ones: 4 « home made » tricks Bagging: 60 experts Boosting:Boosting: trained on such that performs well

45 experts Specializing: focus on cold/warm days, some periods of the year… 24 experts Time scaling: MD with GAM, ST with the 3 initial experts

| 23

combining

Designing experts

COMBINING FORECASTS

| 24

ANOTHER DATA SET: HEAT DEMAND

| 25

Forecasting methods: Industrial implementation on the way (national, substations, cogeneration central in Poland: 30%

better with GAM) CLR: improve automatic clusturing, forecasting the clusters (HMM)

Combining: publication of the R package OPERA (Online Prediction through ExpeRts Aggregation) coming soon application on other data sets derive probabilistic forecasts from a set of experts

Probabilistic forecasts

PERSPECTIVES