Probabilistic Constrained Optimization on Flow Networks

Probabilistic Constrained Optimization on Flow Net-works

Martin Gugat, Jens Lang, Elisa Strauch, Michael SchusterFAU Erlangen-Nürnberg, TU Darmstadt18.02.2020

Motivation

Martin Gugat, Jens Lang, Elisa Strauch, Michael Schuster · FAU Erlangen-Nürnberg, TU Darmstadt · Probabilistic Constrained Optimization on Flow Networks 2

Motivation

Consider the optimization problem

min f (x , ξ)

s.t. P(g(x , ξ) ≤ 0) ≥ α

with objective function f , constraint g, decision vector x , random variable ξ (withprobability distribution and density function) and probability level α.

How to compute this probability?


Motivation


min f (x , ξ)

s.t. P(g(x , ξ) ≤ 0) ≥ α

with objective function f , constraint g, decision vector x , random variable ξ (withprobability distribution and density function) and probability level α.

How to compute this probability?


Spheric Radial Decomposition

Theorem: Spheric radial decomposition (SRD)

Let ξ ∼ N (0,R) be the n-dimensional standardGaussian distribution with zero mean andpositive definite correlation matrix R. Then, forany Borel measurable subset M ⊆ Rn it holdsthat

P(ξ ∈ M) =

∫Sn−1

µχr ≥ 0|rLv ∈ Mdµη(v),

where Sn−1 is the (n − 1)-dimensional sphere inRn, µη is the uniform distribution on Sn−1, µχdenotes the χ-distribution with n degrees offreedom and L is such that R = LLT .

[e.g. Van Ackooij, Henrion: Gradient formulae for nonlinear probabilistic constraints with Gaussian andGaussian-like distributions, SIAM J. Optim. (2014)]


Kernel Density Estimator

Definition: Kernel density estimator (KDE)

Let Y = y1, · · · , yN be independent andidentically distributed samples of the randomvariable Y , which has a absolutely continuousdistribution function with probability densityfunction %. Let K be a kernel function. Then, thekernel density estimator %N corresponding to thebandwidth h ∈ (0,∞) is defined as

%N(z) =1

Nh

N∑i=1

K(z − yi

h

).

[e.g. Parzen: On Estimation of a Probability Density Function and Mode, Ann. Math. Stat. (1962)]


Kernel Density Estimator


Outline

1) A stationary setting: Gas transport

1.1) A single pipe1.2) Necessary optimality conditions

2) A dynamic setting: Contamination of water

2.1) Dynamic probabilistic constraints2.2) A single pipe2.3) Necessary optimality conditions


A stationary setting: Gas transport

1) A stationary setting: Gas transport


Gas transport in a single pipe

For x ∈ [0, L], consider the stationary, semilinear, isothermal Euler equations

qx = 0

c2px = − λ

2D(RST )2 |q|q

p

with pressure p(x), flow q(x), sound speed in the gas c ∈ R, pipe friction coefficientλ ∈ R+ and pipe diameter D ∈ R+.

Let boundary conditionsp(0) = p0

q(L) = bD

be given.



Solution of the Semi-Linear IIE for Horizontal Pipes

Using the Ideal Gas-Equation pe(x) = RSTρe(x), a solution of the upper IIE is givenby:

q(x) = bD

p(x) =

√p2

0 − (RST )2 λ

a2Dq(x)|q(x)|x

Definition: Set of feasible loads

The setM := b ∈ R≥0 | p(L; b) ∈ [pmin, pmax] .

is called the set of feasible loads



Solution of the Semi-Linear IIE for Horizontal Pipes

Using the Ideal Gas-Equation pe(x) = RSTρe(x), a solution of the upper IIE is givenby:

q(x) = bD

p(x) =

√p2

0 − (RST )2 λ

a2Dq(x)|q(x)|x


The setM := b ∈ R≥0 | p(L; b) ∈ [pmin, pmax] .

is called the set of feasible loads



Motivated by applications, the gas demand is random. For mean value µ > 0, astandard deviation σ > 0 and a suitable probability space (Ω,A,P), let a Gaussiandistributed random variable

ξb ∼ N (µ, σ)

be given. We setb := ξb(ω)

for ω ∈ Ω.

How to compute P(b ∈ M) ?



Motivated by applications, the gas demand is random. For mean value µ > 0, astandard deviation σ > 0 and a suitable probability space (Ω,A,P), let a Gaussiandistributed random variable

ξb ∼ N (µ, σ)

be given. We setb := ξb(ω)

for ω ∈ Ω.

How to compute P(b ∈ M) ?


Gas transport in a single pipe (SRD)

1. Rewrite the feasible set:

b ∈ M ⇔ (pmin)2 ≤ p20 − φ | b | b ≤ (pmax)2 with φ =

λ

c2D(RST )2L.

2. Sample the sphere Sn−1 = S0 = −1, 1 and for every v ∈ Sn−1 set

bv(r ) = rσv + µ.

3. Define the regular range:

Rv ,reg := r ≥ 0 | bv(r ) ≥ 0.4. Compute the one dimensional sets

Mv = r ∈ Rv ,reg | bv(r ) ∈ M =

s⋃j=1

[av ,j , av ,j ].

5. Compute the probability

P(b ∈ M) ≈ 12

∑v∈−1,1

s∑j=1

Fχ(av ,j)−Fχ(av ,j),

where Fχ is the cumulative distribution function of the χ-distribution.





λ

c2D(RST )2L.


bv(r ) = rσv + µ.



Mv = r ∈ Rv ,reg | bv(r ) ∈ M =

s⋃j=1

[av ,j , av ,j ].


P(b ∈ M) ≈ 12

∑v∈−1,1

s∑j=1







λ

c2D(RST )2L.


bv(r ) = rσv + µ.


Rv ,reg := r ≥ 0 | bv(r ) ≥ 0.

4. Compute the one dimensional sets

Mv = r ∈ Rv ,reg | bv(r ) ∈ M =

s⋃j=1

[av ,j , av ,j ].


P(b ∈ M) ≈ 12

∑v∈−1,1

s∑j=1







λ

c2D(RST )2L.


bv(r ) = rσv + µ.



Mv = r ∈ Rv ,reg | bv(r ) ∈ M =

s⋃j=1

[av ,j , av ,j ].


P(b ∈ M) ≈ 12

∑v∈−1,1

s∑j=1







λ

c2D(RST )2L.


bv(r ) = rσv + µ.



Mv = r ∈ Rv ,reg | bv(r ) ∈ M =

s⋃j=1

[av ,j , av ,j ].


P(b ∈ M) ≈ 12

∑v∈−1,1

s∑j=1


where Fχ is the cumulative distribution function of the χ-distribution.Martin Gugat, Jens Lang, Elisa Strauch, Michael Schuster · FAU Erlangen-Nürnberg, TU Darmstadt · Probabilistic Constrained Optimization on Flow Networks 13

Gas transport in a single pipe (KDE)

Let B = bS,1, · · · , bS,N ⊆ R≥0 be independent and identically distributed samplesbS,i := ξb(ωi) (i = 1, · · · ,N, ξb ∼ N (µ, σ)).

Let PB = pS,1, · · · , pS,N ⊆ R be the pressures pS,i = p(L, bS,i) at the end of thepipe for the different loads bS,i ∈ B.

With a Gaussian kernel function

K (t) =1√2π

exp

(−1

2t2)

and a bandwidth h ∈ R+, we get an approximation of the probability density functionof the pressure

%p,N(z) =1

Nh

N∑i=1

1√2π

exp

(−1

2

(z − pS,i

h

)).

It follows

P(b ∈ M) ≈∫ pmax

pmin

%p,N(z)dz.






K (t) =1√2π

exp

(−1

2t2)


%p,N(z) =1

Nh

N∑i=1

1√2π

exp

(−1

2

(z − pS,i

h

)).

It follows


pmin

%p,N(z)dz.






K (t) =1√2π

exp

(−1

2t2)


%p,N(z) =1

Nh

N∑i=1

1√2π

exp

(−1

2

(z − pS,i

h

)).

It follows


pmin

%p,N(z)dz.



Choice of bandwidth

A good heuristic choice for the bandwidth is

h = 1.06σN5√

N.

[Gramacki: Nonparametric Kernel Density Estimation and Its Computational Aspects; Springer (2018)Turlach: Bandwidth Selection in Kernel Density Estimation: A Review ; Technical Report (1999)]

L∞-convergence

If the probability density function %p is uniformly continuous, then NadarayasTheorem guarantees

supz|%p(z)− %p,N|

N→∞−−−→ 0 P-almost surely.

[Nadaraya: On Non-Parametric Estimates of Density Functions and Regression Curves; Theory Probab.Appl. (1965)]



Choice of bandwidth

A good heuristic choice for the bandwidth is

h = 1.06σN5√

N.

[Gramacki: Nonparametric Kernel Density Estimation and Its Computational Aspects; Springer (2018)Turlach: Bandwidth Selection in Kernel Density Estimation: A Review ; Technical Report (1999)]

L∞-convergence

If the probability density function %p is uniformly continuous, then NadarayasTheorem guarantees

supz|%p(z)− %p,N|


[Nadaraya: On Non-Parametric Estimates of Density Functions and Regression Curves; Theory Probab.Appl. (1965)]



L1-convergence

If the distribution of the pressure is absolute continuous with probability densityfunction %p, then

‖%p − %p,N‖L1N→∞−−−→ 0 P-almost surely.

[Devroye, Gyorfi: Nonparametric density estimation: the L1 view ; Wiley Ser. Probab. Stat. (1985)]

L1-convergence II

If the distribution of the pressure is absolute continuous with probability densityfunction %p, then from Scheffé’s lemma it follows

|P(b ∈ M)− PN(b ∈ M)| ≤ 12‖%p − %p,N‖L1





L1-convergence

If the distribution of the pressure is absolute continuous with probability densityfunction %p, then

‖%p − %p,N‖L1N→∞−−−→ 0 P-almost surely.


L1-convergence II

If the distribution of the pressure is absolute continuous with probability densityfunction %p, then from Scheffé’s lemma it follows

|P(b ∈ M)− PN(b ∈ M)| ≤ 12‖%p − %p,N‖L1





Example: We choose the following values:

p0 pmin pmax µ σ φ

60 40 60 4 0.5 100Table: Values for the example with one edge.

⇒ M = [0,√

20]

Test 1 Test 2 Test 3 Test 4 Test 5 Test 6 Test 7 Test 8MC 82.99% 82.88% 82.83% 82.95% 83.09% 82.77% 82.83% 82.58%

KDE 82.83% 82.75% 82.69% 82.74% 82.91% 82.58% 82.70% 82.43%

SRD 82.75%

Table: Results for the example with one edge.


Necessary optimality conditions: stationary states

Let B = bS,1, · · · , bS,N ⊆ Rn≥0 be independent and identically distributed samples

bS,i := ξb(ωi) (i = 1, · · · ,N, ξb ∼ N (µ,Σ)).

Let PB = pS,1, · · · , pS,N ⊆ Rn be the pressures with pS,ij = pj(L, bS,i), where

pj(L, bS,i) is the pressure at the end of pipe j (j = 1, · · · , n) for the load bS,i ∈ B.

Then for bandwidths hj (j = 1, · · · , n), we get an approximation of the probabilitydensity function of the pressure

%p,N(z) =1

N∏n

j=1 hj

N∑i=1

n∏j=1

1√2π

exp

(−1

2

(zj − pj(bS,i)

hj

)2)

and we can compute the probability

PN(b ∈ M) =

∫Pmaxmin

%p,N(z)dz.



∫Pmaxmin

%p,N(z)dz =

∫Pmaxmin

1N∏n

j=1 hj

N∑i=1

n∏j=1

K

(zj − pS,i

j

hj

)dz

=1

N∏n

j=1 hj

N∑i=1

∫ pmax1

pmin1

· · ·∫ pmax

n

pminn

n∏j=1

K

(zj − pS,i

j

hj

)dz1 · · · dzn

=1

N∏n

j=1 hj

N∑i=1

n∏j=1

∫ pmaxj

pminj

K

(zj − pS,i

j

hj

)dzj

=1N

N∑i=1

n∏j=1

∫ ϕi,j(pmaxj )

ϕi,j(pminj )

1√π

exp(−τ2

i ,j

)dτi ,j

=1N

12n

N∑i=1

n∏j=1

[erf(ϕi ,j(pmax

j ))− erf(ϕi ,j(pminj ))

]



∫Pmaxmin

%p,N(z)dz =

∫Pmaxmin

1N∏n

j=1 hj

N∑i=1

n∏j=1

K

(zj − pS,i

j

hj

)dz

=1

N∏n

j=1 hj

N∑i=1

∫ pmax1

pmin1

· · ·∫ pmax

n

pminn

n∏j=1

K

(zj − pS,i

j

hj

)dz1 · · · dzn

=1

N∏n

j=1 hj

N∑i=1

n∏j=1

∫ pmaxj

pminj

K

(zj − pS,i

j

hj

)dzj

=1N

N∑i=1

n∏j=1

∫ ϕi,j(pmaxj )

ϕi,j(pminj )

1√π

exp(−τ2

i ,j

)dτi ,j

=1N

12n

N∑i=1

n∏j=1

[erf(ϕi ,j(pmax


]



∫Pmaxmin

%p,N(z)dz =

∫Pmaxmin

1N∏n

j=1 hj

N∑i=1

n∏j=1

K

(zj − pS,i

j

hj

)dz

=1

N∏n

j=1 hj

N∑i=1

∫ pmax1

pmin1

· · ·∫ pmax

n

pminn

n∏j=1

K

(zj − pS,i

j

hj

)dz1 · · · dzn

=1

N∏n

j=1 hj

N∑i=1

n∏j=1

∫ pmaxj

pminj

K

(zj − pS,i

j

hj

)dzj

=1N

N∑i=1

n∏j=1

∫ ϕi,j(pmaxj )

ϕi,j(pminj )

1√π

exp(−τ2

i ,j

)dτi ,j

=1N

12n

N∑i=1

n∏j=1

[erf(ϕi ,j(pmax


]



∫Pmaxmin

%p,N(z)dz =

∫Pmaxmin

1N∏n

j=1 hj

N∑i=1

n∏j=1

K

(zj − pS,i

j

hj

)dz

=1

N∏n

j=1 hj

N∑i=1

∫ pmax1

pmin1

· · ·∫ pmax

n

pminn

n∏j=1

K

(zj − pS,i

j

hj

)dz1 · · · dzn

=1

N∏n

j=1 hj

N∑i=1

n∏j=1

∫ pmaxj

pminj

K

(zj − pS,i

j

hj

)dzj

=1N

N∑i=1

n∏j=1

∫ ϕi,j(pmaxj )

ϕi,j(pminj )

1√π

exp(−τ2

i ,j

)dτi ,j

=1N

12n

N∑i=1

n∏j=1

[erf(ϕi ,j(pmax


]




(?)

min f (pmax)

s.t. gα(pmax) := α− P(b ∈ M(pmax)) ≤ 0.

∂gα(pmax)

∂pmaxk

= − 1N

12n

N∑i=1

n∏j = 1j 6= k

[erf(ϕi ,j(pmax


]·√

2√πhk

exp(−ϕ2

i ,k(pmaxk ))< 0

Necessary optimality conditions

Let p∗,max ∈ Rn be a optimal solution of (?). Then there exists a multiplier µ∗ ≥ 0, s.t.

∇f (p∗,max) + µ∗∇gα = 0gα(p∗,max) ≤ 0µ∗gα(p∗,max) = 0.




(?)

min f (pmax)

s.t. gα(pmax) := α− P(b ∈ M(pmax)) ≤ 0.

∂gα(pmax)

∂pmaxk

= − 1N

12n

N∑i=1

n∏j = 1j 6= k

[erf(ϕi ,j(pmax


]·√

2√πhk

exp(−ϕ2

i ,k(pmaxk ))< 0


Let p∗,max ∈ Rn be a optimal solution of (?). Then there exists a multiplier µ∗ ≥ 0, s.t.

∇f (p∗,max) + µ∗∇gα = 0gα(p∗,max) ≤ 0µ∗gα(p∗,max) = 0.


A dynamic setting: Contamination of water

2) A dynamic setting: Contamination of water


Dynamic probabilistic constraints

How to add a time dependence to a probabilistic constraint?

P( b ∈ M(t) ∀t ∈ [0,T ] ) ≥ α

P( b ∈ M(t) ) ≥ α ∀t ∈ [0,T ]

1T

∫ T

0P( b ∈ M(t) ) dt ≥ α




P( b ∈ M(t) ∀t ∈ [0,T ] ) ≥ α

P( b ∈ M(t) ) ≥ α ∀t ∈ [0,T ]

1T

∫ T

0P( b ∈ M(t) ) dt ≥ α




P( b ∈ M(t) ∀t ∈ [0,T ] ) ≥ α

P( b ∈ M(t) ) ≥ α ∀t ∈ [0,T ]

1T

∫ T

0P( b ∈ M(t) ) dt ≥ α



How to get time dependent random boundary data?

Let bD(t) ∈ L2([0,T ]) be given. Define

ψm(t) :=

√2√T

sin

((π2

+ mπ) t

T

)and a0

m :=

∫ T

0bD(t)ψm(t)dt

Then it is

bD(t) =

∞∑m=0

a0mψm(t).

Consider Gaussian distributed random variablesξam ∼ N (µ, σ). For am = ξam(ω), (ω ∈ Ω) considerthe random boundary data

b(t) =

∞∑m=0

ama0mψm(t) ∈ L2([0,T ]) P-a.s.

[Farshbaf-Shaker, Gugat, Heitsch, Henrion: Optimal Neumann Boundary Control of a Vibrating String withUncertain Initial Data and Probabilistic Terminal Constraints (submitted 2019)]



How to get time dependent random boundary data?Let bD(t) ∈ L2([0,T ]) be given. Define

ψm(t) :=

√2√T

sin

((π2

+ mπ) t

T

)and a0

m :=

∫ T

0bD(t)ψm(t)dt

Then it is

bD(t) =

∞∑m=0

a0mψm(t).


b(t) =

∞∑m=0





How to get time dependent random boundary data?Let bD(t) ∈ L2([0,T ]) be given. Define

ψm(t) :=

√2√T

sin

((π2

+ mπ) t

T

)and a0

m :=

∫ T

0bD(t)ψm(t)dt

Then it is

bD(t) =

∞∑m=0

a0mψm(t).


b(t) =

∞∑m=0




Contamination of water in a single pipe

For (t , x) ∈ [0,T ]× [0, L] and constants d < 0, m ≤ 0, we consider the deterministicscalar linear PDE with initial condition and boundary condition

(?)

rt(t , x) + drx(t , x) = mr (t , x),

r (0, x) = r0(x),

r (t , L) = b(t).

Assume C0-compatibility between the initial and the boundary condition. Thisequation e.g. models the flow of contamination in water along a pipe or in a network.

Solution of (?)

A solution of (?) is given by

r (t , x) =

exp(mt) r0(x − dt) if x ≤ L + dt ,exp(mx−L

d

)b(t − x−L

d ) if x > L + dt .



For (t , x) ∈ [0,T ]× [0, L] and constants d < 0, m ≤ 0, we consider the deterministicscalar linear PDE with initial condition and boundary condition

(?)

rt(t , x) + drx(t , x) = mr (t , x),

r (0, x) = r0(x),

r (t , L) = b(t).

Assume C0-compatibility between the initial and the boundary condition. Thisequation e.g. models the flow of contamination in water along a pipe or in a network.

Solution of (?)

A solution of (?) is given by

r (t , x) =

exp(mt) r0(x − dt) if x ≤ L + dt ,exp(mx−L

d

)b(t − x−L

d ) if x > L + dt .




For t∗ ∈ [0,T ], the set

M(t∗) := b ∈ L2([0,T ]);R≥0) | r (t∗, 0) ∈ [rmin, rmax]

is called the set of feasible loads.

Our aim is to compute the probability

P( b ∈ M(t∗) ).


Contamination of water in a single pipe (SRD)

0. Estimate the mean µ(t) and the variance σ(t) for the random boundary data

1. Rewrite the set of feasible loads:

b ∈ M(t∗) ⇔ rmin ≤ r (t∗, 0) ≤ rmax.

2. Sample the sphere Sn−1 and set bv(r , t) = rL(t)v + mu(t).

3. Define the regular range Rv ,reg := r ≥ 0|bv(r , t) ≥ 0.

4. Compute the one dimensional sets Mv(t∗) = r ∈ Rv ,reg|bv(r , t) ∈ M(t∗).

5. Compute the probability P(b ∈ M) ≈ 12

∑v∈−1,1

∑sj=1Fχ(av ,j)−Fχ(av ,j).


Contamination of water in a single pipe (KDE)

Let A = (a1,m)m≥0, · · · , (aN,m)m≥0 be a sampling of N independent and identicallydistributed sequences.

Let BA(t) = bS,1(t), · · · , bS,N(t) be the corresponding sampling of randomboundary functions.

Let R(t) = r (t , 0, bS,1), · · · , r (t , 0, bS,n) be the sampling of solutions at x = 0corresponding to the boundary functions in B.

With a Gaussian kernel function and a bandwidth h+ ∈ R+, we get an approximationof the probability density function of the pressure

%r ,t∗,N(z) =1

Nh

N∑i=1

K(

z − r (t∗, 0, bS,i)

h

)and we can compute the probability

P ( b ∈ M(t∗) ) ≈∫ rmax

rmin

%r ,t∗,N(z) dz.



How to compute P ( b ∈ M(t) ∀t ∈ [0,T ] )?

⇒ Minimal and Maximal value must be feasible

Define the valuesr i := min

t∈[0,T ]r (t , 0, bS,i) (i = 1, · · · ,N),

r i := maxt∈[0,T ]

r (t , 0, bS,i) (i = 1, · · · ,N).

For bandwidths h1, h2 we get an approximation of the probability density function ofthe maximal and minimal pressure

%r ,N(z) =1

Nh1h2

N∑i=1

K(

z1 − r i

h1

)K(

z2 − r i

h2


P ( b ∈ M(t) ∀t ∈ [0,T ] ) ≈∫[rmin,rmax]×[rmin,rmax]

%r ,N(z) dz.






t∈[0,T ]r (t , 0, bS,i) (i = 1, · · · ,N),

r i := maxt∈[0,T ]

r (t , 0, bS,i) (i = 1, · · · ,N).


%r ,N(z) =1

Nh1h2

N∑i=1

K(

z1 − r i

h1

)K(

z2 − r i

h2



%r ,N(z) dz.






t∈[0,T ]r (t , 0, bS,i) (i = 1, · · · ,N),

r i := maxt∈[0,T ]

r (t , 0, bS,i) (i = 1, · · · ,N).


%r ,N(z) =1

Nh1h2

N∑i=1

K(

z1 − r i

h1

)K(

z2 − r i

h2



%r ,N(z) dz.



Example: We choose the following values:rmin0 rmax

0 µ σ d m L T2 6 1 0.25 −5 −1 1 4

Table: Values for the dynamic example.

Further: 101 time discretization points, 30 terms in the Fourier series.


KDE 74.26% 74.31% 74.35% 74.11% 74.04% 74.32% 74.27% 74.27%

Table: Results for the dynamic example with one edge



Example: We choose the following values:rmin0 rmax

0 µ σ d m L T2 6 1 0.25 −5 −1 1 4

Table: Values for the dynamic example.

Further: 101 time discretization points, 30 terms in the Fourier series.


KDE 74.26% 74.31% 74.35% 74.11% 74.04% 74.32% 74.27% 74.27%

Table: Results for the dynamic example with one edge


Necessary optimality conditions: dynamics


Let r ∗,max ∈ R be a solution of the optimization problem

min f (rmax)

s.t. gα(rmax) = α− P(b ∈ M(t) ∀t ∈ [0,T ] ) ≤ 0.

Then there exist a multiplier µ∗ ≥ 0, s.t.

f ′(r ∗,max) + µ∗g′α(r ∗,max) = 0gα(r ∗,max) ≤ 0µ∗gα(r ∗,max) = 0.


References

C. Gotzes, H. Heitsch, R. Henrion, R. SchultzOn the quantification of nomination feasibility in stationary gas networks with random loadMath. Meth. Oper. Res. 84 (2016)

M. Gugat, M. SchusterStationary Gas Networks with Compressor Control and Random Loads: Optimization with ProbabilisticConstraintsMathematical Problems in Engineering (2018)

M. GugatContamination Source Determination in Water Distribution NetworksSIAM J. Appl. Math. (2012)

W. Härdle, A. Werwatz, M. Müller, S. SperlichNonparametric and Semiparametric Models; Springer (2004)

A. PrékopaStochastic Programming; Springer (1995)

Thank you for your attention!


Probabilistic Constrained Optimization on Flow Networks

Documents