Artificial Intelligence & Energy · Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran,

Artificial Intelligence & Energy

Bertrand Cornélusse, Raphaël Fonteneau

1

TheSmartgridsteamispartoftheMontefioreResearchUnitoftheULg,containsaround15researchersandisheadedbyPr.DamienErnst

http://blogs.ulg.ac.be/damien-ernst2

OurvisionofArtificialIntelligence

3

Roadmap

Artificial intelligence

OptimizationMachine learning

Reinforcement learning

4

Roadmap

Machine learning

5

Machinelearningisaboutextracting{patterns,knowledge,information}fromdata

Clusterimages

SIRICortona OK Google

Convertvoicesignalintosentences

Makeon-linerecommandations

Recognizepatternsinimages

Interpretsentences

Google photos

6

Machinelearningstudiesandbuildsalgorithmsthatlearnfromandmakepredictionsondata

SupervisedLearninginanutshell:

Imagineyouhaveasetofdata {(x1, y1), (x2, y2), …, (xn, yn)}

represented by black points on thefigure.

To be able to estimate the value of anoutputy for any inputx, You “train” aMachineLearningalgorithmusingthesedata.Youobtaintheblueline.

Thequalityoftheestimatedependsondataquality/quantity:withmorepoints,e.g. the black circles, you would forinstancegettheredcurve.

7

x

y

Recentadvancesinmachinelearning

Machinelearningalgorithmshaverecentlyshownimpressiveresults,inparticularwhen input data are images: this has led to the identification of a subfield ofMachineLearningcalledDeepLearning.

The term “deep” refers to the fact that those learning architectures, mainlyArtificialNeuralNetworks,aremadeofseverallayers.

Zoom on a neuron

8

Deepneuralnetworkarchitectures

Source: http://www.ais.uni-bonn.de/deep_learning/images/Convolutional_NN.jpg

9

Wait…ANNarenotnew,right?

ANNdatebacktothesixties.TrainingANNwasnotaneasytaskuntilrecently.Recentprogressistwofold:

• Smart(er)trainingapproaches• GPUcalculus

10

Optimization

Artificial intelligence

Roadmap

Machine learning

Reinforcement learning

11

Fromsupervisedlearningtoreinforcementlearning

Supervisedlearningtechniques(inparticular(deep)convolutionalnetworks)maybeusedas a block in a more complex structure, inparticular in Dynamic Programming (DP) orModelPredictiveControl(MPC)schemes.

This connects to reinforcement learning, anarea of machine learning originally inspiredby behaviorist psychology, concerned withhowsoftwareagentsoughttotakeactionsinan environment so as to maximize somenotionofcumulativereward.

Deepreinforcementlearningcombinesdeeplearning with reinforcement learning (and,consequently,inDP/MPCschemes).

12

Agent

Environment

ActionReward

PlayingAtariwithdeepreinforcementlearning

13

At Google Deepmind At ULg

Human-levelcontrolthroughdeepreinforcementlearning.Nature,2015.VolodymyrMnih,KorayKavukcuoglu,DavidSilver,AndreiA.Rusu,JoelVeness,MarcG.Bellemare,AlexGraves,MartinRiedmiller,AndreasK.Fidjeland,GeorgOstrovski,StigPetersenCharlesBeattie,AmirSadik,IoannisAntonoglou,HelenKing,DharshanKumaran,DaanWierstra,ShaneLegg&DemisHassabis

Breakingnews

Recentbreakthroughs in the fieldofAI for thegameofGOhavebeendonebyGoogleDeepmind.

These results have been obtained by combining Deep Convolutional NetworkswithMonteCarloTreeSearchtechniques.

Theresultingagent,AlphaGo,achieved99.8%winningrateagainstotherGOAI,anddefeatedtheEuropeanGochampionby5gamesto0.

14

MasteringthegameofGowithdeepneuralnetworksandtreesearch.Nature,2016.DavidSilver,AjaHuang,ChrisJ.Maddison,ArthurGuez,LaurentSifre,GeorgevandenDriessche,JulianSchrittwieser,IoannisAntonoglou,VedaPanneershelvam,MarcLanctot,SanderDieleman,DominikGrewe,JohnNham,NalKalchbrenner,IlyaSutskever,TimothyLillicrap,MadeleineLeach,KorayKavukcuoglu,ThoreGraepel&DemisHassabis

Wanttoknowmore?

Googleislaunchinganewdeeplearningcourse(incollaborationwithUdacity):https://www.udacity.com/course/deep-learning--ud730

YoumayalsobeinterestedinNVidiaDeepLearningcourse:https://developer.nvidia.com/deep-learning-courses

OrevenStanfordMoocaboutMachineLearning:https://www.coursera.org/learn/machine-learning

15

Optimization

Artificial intelligence

Roadmap

Machine learning

Reinforcement learning

16

Optimization: decide the values that some variables can take,underasetofcontraints,soastomaximizeanobjective.

A long tradition of numerical solutions and theoretical analysis.Givenassumptionsonmodels,onecaneventuallygetguaranteesaboutsolutions.

Howisoptimizationconnectedtomachinelearning?Learningproblemscanbecastedasoptimizationproblems

Howismachinelearningconnectedtooptimization?Machine learning actually solves some (or part of) optimizationproblems(e.g:RL,ortuningofanalgo,orproxytoanalgo)

Machinelearningistightlycoupledtooptimization

17

Machinelearningistightlycoupledtooptimization

Anillustrationofthesimplexalgorithm. The simplexalgorithmwasinventedbyG.Dantzig. Itdatesback to thesecondworldwar.

This can be used to solvemany practical optimizationproblems.

18

set NUTRIENT ordered; set FOOD ordered;

param cost {FOOD} >= 0; param minNutrient {NUTRIENT} >= 0; param maxNutrient {i in NUTRIENT} >= minNutrient[i]; param amount {NUTRIENT,FOOD} >= 0;

# Variables var Buy {j in FOOD} integer;

# Objective minimize Total_Cost: sum {j in FOOD} cost[j] * Buy[j];

(or minimize nutrient_amount {i in NUTRIENT}: sum {j in FOOD} amount[i,j] * Buy[j];)

# Constraints subject to Diet {i in NUTRIENT}: minNutrient[i] <= sum {j in FOOD} amount[i,j] * Buy[j] <= maxNutrient[i];

Optimizationreliesonananalyticalmodel...

Example: Building the lunch menu, a first application of AI for energy ;)

19

set NUTRIENT ordered; set FOOD ordered;

param cost {FOOD} >= 0; param minNutrient {NUTRIENT} >= 0; param maxNutrient {i in NUTRIENT} >= minNutrient[i]; param amount {NUTRIENT,FOOD} >= 0;

# Variables var Buy {j in FOOD} integer;

# Objective minimize Total_Cost: sum {j in FOOD} cost[j] * Buy[j];

(or minimize nutrient_amount {i in NUTRIENT}: sum {j in FOOD} amount[i,j] * Buy[j];)

# Constraints subject to Diet {i in NUTRIENT}: minNutrient[i] <= sum {j in FOOD} amount[i,j] * Buy[j] <= maxNutrient[i];

Optimizationreliesonananalyticalmodel...

Example: Building the lunch menu, a first application of AI for energy ;)

+ Data

Your lunch menu

20

Optimizationreliesonananalyticalmodel,machinelearningmaynot

+

+

+

21

OptimizationandMachinelearninghavedifferentaims

Intheoptimizationworld,amethodtargetsoneproblemclass,orevenaninstanceofaproblem,andatheoryisobsessedbyoptimality(canIproveitmathematically?)andefficiency(canIcomputeitefficiently?)

Machinelearningisfocusedonstatisticalsignificance(reachinga trade off between overfitting and “misrepresentation”),replicability to other problems with few adaptation, andinterpretabilityofresults

22

Roadmap

Artificial intelligence

OptimizationMachine learning

Reinforcement learning

23

Energeticapplications

24

Worldenergyconsumptionoutlookaround2010

Fossil fuels Nuclear Renewables

Biomass heatSolar hotwaterGeothermal heatHydropowerEthanolBiodieselBiomass electricityWind powerGeothermal electricitySolar PV powerSolar CSPOcean powerSources: IEA

25

OptimizationhasplentyofapplicationsintheEnergyindustry

Electricalpowersystems:• Productionplanning:unitcommitment• Managinggridconstraints:optimalpowerflow

Oilandgasindustry:• Wheretodig?Inwhichsequence?

Logisticsandtransportation:• VehicleRoutingProblems

Industrialprocesses:• Reductionordisplacementofenergyconsumption

26

Example:Day-aheadelectricitypricesinEuropearedeterminedbyEuphemia

EUPHEMIA is the market coupling algorithm for European Power exchanges, implemented and developed in-house by N-SIDE, a spin-off of UCL and ULg

Used daily by Power Exchanges to fix pan-EU day-ahead electricity prices in 19 EU countries.

Computing market prices & volumes by: • coupling national markets • maximizing total economical welfare • optimizing network capacity utilization • modeling complex economical constraints

Extension to whole Europe in progress

http://energy.n-side.com/day-ahead/

27

EvolutionoftheenergysystemGlobal Grid(s) versus Microgrids

Prof. Damien Ernst - University of LiègeELIA Stakeholders͛ days

28

Fromdecentralization…

29

Fromdecentralizationtocentralization

30

Fromdecentralizationtocentralization,andback

31

WhyarewenowtalkingaboutAI,andnotjustaboutoptimization?

Wearenowtryingtooptimizemoreandmorelocally,becauserenewable energy sources are distributed, data is ubiquitousandcomputationpoweraswell.

However,theratio“gain/(timetospendforgatheringthedataand solving the problem)” is way smaller than for largecentralizedprojects.

AI offers the possibility to automate the data gathering,modelingandoptimizationstages.Forinstance,learnfromthehabitsofusersofahouse,proposesomecarpoolingoptions,correlateallthiswithcalendarevents.

32

Rethinkingtheoperationofdistributionsystems

Activenetworkmanagement.Smart modulation of generation sources, loads and storages so as to operatesafelytheelectricalnetworkwithouthavingtorelyonsignificant investments ininfrastructure.

GREDORproject.Redesigning in an integrated way the whole decision chain used for managingdistributionnetworksinordertoperformactivenetworkmanagementoptimally(i.e.,maximisationofsocialwelfare).

www.gredor.be

33

Empoweringconsumersanddistributedgeneration

Microgrids aremodern, localized, small-scalegrids, contrary to the traditional, centralizedelectricitygrid(macrogrid).

Some microgrids can operate disconnectedfrom the centralized grid and operateautonomously, strengthen grid resilience andhelpmitigategriddisturbances.

Optimizing the sizing and the operation of amicrogrid requires both optimization and AItechniques.

34

35Smart lighting

Smart homes

Smart mobility

SmartCities

Smart sensors

Wireless communication

UrbanAgriculture

36

So,whyusingaPVpanelasagoban?

37

ReferencesHowtodiscountdeepreinforcementlearning:towardsnewdynamicstrategies.V.François-Lavet,R.Fonteneau,D.Ernst.DeepReinforcementLearningWorkshop,NIPS2015.http://arxiv.org/abs/1512.02011

Benchmarkingforbayesianreinforcementlearning.M.Castronovo,D.Ernst,A.Couëtoux,R.Fonteneau,http://arxiv.org/pdf/1509.04064.pdf

ImitativeLearningforOnlinePlanninginMicrogrids.S.Aittahar,V.François-Lavet,S.Lodeweyckx,D.Ernst,R.Fonteneau.DataAnalyticsforRenewableEnergyIntegration,Volume9518oftheseriesLectureNotesinComputerSciencepp1-15,2015.

Theglobalgrid.S.Chatzivasileiadis,D.Ernst,G.Andersson.RenewableEnergy,Volume57,September2013,Pages372–383.

GlobalGrid(s)versusMicrogrids.Ernst,D.http://hdl.handle.net/2268/188217

TheGREDORproject.Redesigningthedecisionchainformanagingdistributionnetworks.Ernst,D.http://hdl.handle.net/2268/188487

Activenetworkmanagementforelectricaldistributionsystems:problemformulationandbenchmark.Gemine,Q.,Ernst,D.,&Cornélusse,B.(2014).arXivpreprintarXiv:1405.2806.

DSIMA:Atestbedforthequantitativeanalysisofinteractionmodelswithindistributionnetworks.Mathieu,S.,Louveaux,Q.,Ernst,D.,&Cornélusse,B.(2016).SustainableEnergy,GridsandNetworks,5,78-93.

ActiveManagementofLow-VoltageNetworksforMitigatingOvervoltagesDuetoPhotovoltaicUnits.Olivier,F.,Aristidou,P.,Ernst,D.,VanCutsem,T.IEEETransactionsonSmartGrid,2016.

Supervisedlearningofintra-dailyrecoursestrategiesforgenerationmanagementunderuncertainties.Cornélusse,B.,Vignal,G.,Defourny,B.,&Wehenkel,L.(2009,June).InPowerTech,2009IEEEBucharest(pp.1-8).IEEE.

38