1 Nested Dynamic Programming (nDP) algorithm for multipurpose reservoir optimization Blagoj Delipetrev Andreja Jonoski Dimitri P. Solomatine HIC, NYC August 17 – 21, 2014
1
Nested Dynamic Programming (nDP) algorithm
for multipurpose reservoir optimizationBlagoj Delipetrev
Andreja JonoskiDimitri P. Solomatine
HIC, NYC August 17 – 21, 2014
Overview nested Dynamic programming algorithm
The nDP algorithm is built from two algorithms: 1) dynamic programming (DP) and 2) nested optimization algorithm implemented with Simplex and quadratic Knapsack.
The novel idea is to include a nested optimization algorithm into the DP transition that lowers the starting problem dimension and alleviates the DP curse of dimensionality.
The nDP can solve multi-objective optimization problems, without significantly increasing the algorithm complexity and the computational expenses.
Computationally, the nDP is very efficient and it can handle dense and irregular variable discretization
It is coded in Java as a prototype application and has been successfully tested with eight objectives at the Knezevo reservoir, located in the Republic of Macedonia.
2
3
Introduction
Optimal reservoir operation methods include:Dynamic programming (DP)Stochastic dynamic programming (SDP)
HIC, NYC August 17 – 21, 2014
“curse of dimensionality” “curse of modelling”
Successive approximations, incremental dynamic programming and differential dynamic programming
the computational complexity with the state – decision space dimension
Dynamic programming
4
ttttt erqss 1 Reservoir model
111,,min ttttttt sVassgsVBellman equation
(for stages t=T-1,T-2,…1)
20
15
10
5
20
15
10
5
20
15
10
5
20
15
10
5
20
15
10
5
20
15
10
5
Inflow
Release
20
15
10
5
20
15
10
5
HIC, NYC August 17 – 21, 2014
5
Research question
HIC, NYC August 17 – 21, 2014
How to enhance DP algorithm and develop new algorithm (methodology) that is flexible with including additional objectives like cities water demand, agriculture water demand, ecology water demand, hydro power production and etc., alleviate as much as possible the curse of dimensionality and computational cost?
Hydropower
production
Minimum and
maximum
reservoir targets
6
Main ideanested Dynamic programming (nDP)
HIC, NYC August 17 – 21, 2014
One step of the DP algorithm. One step of the nDP algorithm
rt
St
qt
g(st,st+1,at)+minV(St+1)
St+1
rt
Nested optimal allocation algorithm
minV(St+1)
r1t
U1t
r2t rnt…
U2t
Unt
rt
st
qt
g(st,st+1,at)+minV(st+1)
st+1
7
Nested optimal allocation algorithmsSimplex – Quadratic Knapsack
HIC, NYC August 17 – 21, 2014
n
iititit rdw
1
min
tnttt rrrr ...21
ntnttttt drdrdr ..,, 2211
0..,,, 21 ntttt dddr
n
iititit rdw
1
2min
Simplex Quadratic Knapsack
nDP demonstration Zletovica river basin
8
Zletovica river basin is in the eastern part of the Republic of Macedonia.
HIC, NYC August 17 – 21, 2014
9
Objective function
HIC, NYC August 17 – 21, 2014
28
11,, it
iittttt Dwassg
The optimization problem has eight objectives and six decision variables.
weighted sum of squared deviations over the entire time horizon
,
,0
t1t1
1t1 hdifdh
hdifD
tt
tt
tttt
tt2 ,
,0FlifFlFlif
D t
where d1t and d2t are the minimum and maximum level targets and ht is the reservoir level height at time step t
Objective function
Water demand users1) the towns of Zletovo and Probishtip (one intake), d3t, 2) the upper agricultural zone, d4t, 3) the towns of Shtip and Sveti Nikole (one intake), d5t, 4) the lower agricultural zone, d6t, 5) the minimum environmental flow, d7t.
10
itititit
ititit rdifrd
rdifD
, ,0
where ht is the hydropower demand and pt is the hydropower production
tttt
tt pdifpd
pifD
88
8t8 ,
d ,0
HIC, NYC August 17 – 21, 2014
11
Results
HIC, NYC August 17 – 21, 2014
Experiments w1 w2 w3 w4 w5 w6 w7 w8nDP-L1 and nDP-Q1
25000 25000 0.2 0.1 0.2 0.1 0.26 0.04
nDP-L2 and nDP-Q2
25000 25000 0.15 0.1 0.25 0.1 0.26 0.04
• 55-year monthly data (1951-2005), with 660 time-steps.• The reservoir operation volume 23 million m3 is discretized in 73 equal
levels (300 103 m3 each). • The minimum reservoir level target was set at 1021.5 [amsl],• and the maximum reservoir level target at 1060 [amsl].• The water supply, irrigation, and hydropower are set to their monthly
demands
nDP-L1 optimal reservoir level in 55 years
12
Resuts
HIC, NYC August 17 – 21, 2014
13
Results
HIC, NYC August 17 – 21, 2014
Experiments w1 w2 w3 w4 w5 w6 w7 w8nDP-L1 and nDP-Q1
25000 25000 0.2 0.1 0.2 0.1 0.26 0.04
nDP-L2 and nDP-Q2
25000 25000 0.15 0.1 0.25 0.1 0.26 0.04
Comparison of nDP with other DP algorithms
14
Classical DP model of reservoir operation
aggregated water demand (AWD) DP algorithmttttt wwwwww
t 76543'
3
tttttt dddddd 76543'3
7
3
22'3
'3
'
3
iititittt rdwRdw
t
HIC, NYC August 17 – 21, 2014
Conclusions
The nDP algorithm has the following advantages: It effectively alleviates the curse of dimensionality in optimal
reservoir operation. It has better optimization capabilities compared to the DP
aggregated water demand approach and can solve problems that are more complex where the DP aggregated water demand approach is not feasible.
Computationally, it is very efficient and runs fast on standard personal computers. The presented case study optimization was executed in less than five minutes.
The algorithm allows for employing dense and variable discretization on the reservoir volume and release.
It supports using a variable weight at each time step for every objective function.
15
HIC, NYC August 17 – 21, 2014
Conclusion It provides a framework for including many objectives in the nested
optimization algorithm without a significant change in the source code or an increase in the computational expenses.
Different optimization algorithms can be used in the nesting for water allocation, however, since nested optimization has to be repeated multiple times (for each transition of DP) the algorithm used for this purpose needs to be fast.
The method presented can be applied to stochastic dynamic programming, reinforcement learning and other similar algorithms.
16
The nDP main idea is to include nested optimization algorithm inside the DP transition, which lowers the problem dimension. With this method, it is possible to solve optimization problems that are currently unsolvable with classical methods, rapidly decrease the optimization time and improve the result that was demonstrated in this paper.
HIC, NYC August 17 – 21, 2014
17
Email [email protected]
[email protected], NYC August 17 – 21, 2014
rt
St
qt
g(st,st+1,at)+minV(St+1)
St+1
rt
Nested optimal allocation algorithm
minV(St+1)
r1t
U1t
r2t rnt…
U2t
Unt
https://github.com/deblagoj/DP-3Objectives