-
Nonlinear Programming Formulation ofChance-Constraints
Andreas Wächterjoint with Alejandra Peña-Ordieres and Jim
Luedtke
Department of Industrial Engineering and Management
SciencesNorthwestern University
Evanston, IL, [email protected]
US & Mexico Workshop on Optimization and Its
ApplicationsHuatulco, Mexico
January 9, 2018
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Problem Statement
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
f (x) : Rn → Rc(x , ξ) : Rn × Ξ→ Rm
I Random variable ξ with support ΞI Assume f (x) and c(x , ξ)
are sufficiently smooth for all ξ ∈ ΞI X ⊆ Rn: Captures additional
constraints
I Goal: Formulate as continuous NLPI Do not want to assume
particular probability distributionI Do not want to assume
convexity (or convex approximations)I Want to avoid combinatorial
approachI Use powerful NLP algorithms and techniques
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Problem Statement
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
f (x) : Rn → Rc(x , ξ) : Rn × Ξ→ Rm
I Random variable ξ with support ΞI Assume f (x) and c(x , ξ)
are sufficiently smooth for all ξ ∈ ΞI X ⊆ Rn: Captures additional
constraintsI Goal: Formulate as continuous NLP
I Do not want to assume particular probability distributionI Do
not want to assume convexity (or convex approximations)I Want to
avoid combinatorial approachI Use powerful NLP algorithms and
techniques
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Constraint in Probability SpaceImpose p(x) = P[c(x , ξ) ≤ 0] ≥
1− α
-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5-0.5
0
0.5
1
1.5
( - 1) - p(x)
y = 0
c(x , ξ) = x2 − 2 + ξξ ∼ N (0, 1)
IssuesI Linearization is poor approximationI Always nonconvexI
Used in [Hu, Hong, Zhang 13], [Bremer, Henrion, Möller 15]
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Quantile Formulation
-3 -2 -1 0 1 2 3
x
0
0.2
0.4
0.6
0.8
1
Cu
mu
lative
Dis
trib
utio
n F
un
ctio
n
Nomal CDF
I Let Y be a real-valued random variableI (1− α)-quantile:
QY1−α = inf{y ∈ R : P[Y ≤ y ] ≥ 1− α}
p(x) ≥ 1− α ⇐⇒ q1−α(x) = Qc(x ,ξ)1−α ≤ 0
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Quantile Formulation
-3 -2 -1 0 1 2 3
x
0
0.2
0.4
0.6
0.8
1
Cu
mu
lative
Dis
trib
utio
n F
un
ctio
n
Nomal CDF
I Let Y be a real-valued random variable (Y = c(x , ξ))I (1−
α)-quantile:
Qc(x ,ξ)1−α = inf{y ∈ R : P[c(x , ξ) ≤ y ] ≥ 1− α}
p(x) ≥ 1− α ⇐⇒ q1−α(x) = Qc(x ,ξ)1−α ≤ 0
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Quantile Formulation
-3 -2 -1 0 1 2 3
x
0
0.2
0.4
0.6
0.8
1
Cu
mu
lative
Dis
trib
utio
n F
un
ctio
n
Nomal CDF
I Let Y be a real-valued random variable (Y = c(x , ξ))I (1−
α)-quantile:
Qc(x ,ξ)1−α = inf{y ∈ R : P[c(x , ξ) ≤ y ] ≥ 1− α}
p(x) ≥ 1− α ⇐⇒ q1−α(x) = Qc(x ,ξ)1−α ≤ 0Andreas Wächter
Nonlinear Programming Formulation of Chance-Constraints
-
Choice of Formulation
p(x) ≥ 1− α ⇐⇒ q1−α(x) ≤ 0
-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5-0.5
0
0.5
1
1.5
( - 1) - p(x)
q1 -
(x)
y = 0
I Quantile formulation more suitable for NLP solver
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Sample Average Approximation (SAA)
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
f (x) : Rn → Rc(x , ξ) : Rn × Ξ→ R
I Finite scenario set Ξ̂N = {ξ̂1, . . . , ξ̂N} with ξ̂i ∈ Ξ
chosen i.i.d.I For simplicity: ci (x) = c(x , ξ̂i )
SAA approximation of quantileI Ensure that M = d(1− α)Ne
constraints are satisfiedI Scenario vector: C(x) = (c1(x), . . . ,
cN(x))I Order constraint values: c[1](x) ≤ c[2](x) ≤ . . . ≤
c[N](x)I Empirical quantile: q̃1−α(C(x)) = c[M](x) ≤ 0
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Sample Average Approximation (SAA)
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
f (x) : Rn → Rc(x , ξ) : Rn × Ξ→ R
I Finite scenario set Ξ̂N = {ξ̂1, . . . , ξ̂N} with ξ̂i ∈ Ξ
chosen i.i.d.I For simplicity: ci (x) = c(x , ξ̂i )
SAA approximation of quantileI Ensure that M = d(1− α)Ne
constraints are satisfied
I Scenario vector: C(x) = (c1(x), . . . , cN(x))I Order
constraint values: c[1](x) ≤ c[2](x) ≤ . . . ≤ c[N](x)I Empirical
quantile: q̃1−α(C(x)) = c[M](x) ≤ 0
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Sample Average Approximation (SAA)
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
f (x) : Rn → Rc(x , ξ) : Rn × Ξ→ R
I Finite scenario set Ξ̂N = {ξ̂1, . . . , ξ̂N} with ξ̂i ∈ Ξ
chosen i.i.d.I For simplicity: ci (x) = c(x , ξ̂i )
SAA approximation of quantileI Ensure that M = d(1− α)Ne
constraints are satisfiedI Scenario vector: C(x) = (c1(x), . . . ,
cN(x))
I Order constraint values: c[1](x) ≤ c[2](x) ≤ . . . ≤ c[N](x)I
Empirical quantile: q̃1−α(C(x)) = c[M](x) ≤ 0
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Sample Average Approximation (SAA)
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
f (x) : Rn → Rc(x , ξ) : Rn × Ξ→ R
I Finite scenario set Ξ̂N = {ξ̂1, . . . , ξ̂N} with ξ̂i ∈ Ξ
chosen i.i.d.I For simplicity: ci (x) = c(x , ξ̂i )
SAA approximation of quantileI Ensure that M = d(1− α)Ne
constraints are satisfiedI Scenario vector: C(x) = (c1(x), . . . ,
cN(x))I Order constraint values: c[1](x) ≤ c[2](x) ≤ . . . ≤
c[N](x)
I Empirical quantile: q̃1−α(C(x)) = c[M](x) ≤ 0
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Sample Average Approximation (SAA)
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
f (x) : Rn → Rc(x , ξ) : Rn × Ξ→ R
I Finite scenario set Ξ̂N = {ξ̂1, . . . , ξ̂N} with ξ̂i ∈ Ξ
chosen i.i.d.I For simplicity: ci (x) = c(x , ξ̂i )
SAA approximation of quantileI Ensure that M = d(1− α)Ne
constraints are satisfiedI Scenario vector: C(x) = (c1(x), . . . ,
cN(x))I Order constraint values: c[1](x) ≤ c[2](x) ≤ . . . ≤
c[N](x)I Empirical quantile: q̃1−α(C(x)) = c[M](x) ≤ 0
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Empirical QuantileI Feasible region for c(x , ξ) = ξ1x1 + ξ2x2 −
1 ξ1, ξ1 ∼ N (0, 1)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 5000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Empirical Quantile
I Feasible region for c(x , ξ) = ξ1x1 + ξ2x2 − 1 ξ1, ξ1 ∼ N (0,
1)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 5000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000
Observations:I Approximation improves as N increasesI Rough
boundary of feasible region results in spurious local minimaI A lot
of variance for small N as ξi are resampledI q̃1−α(C(x)) is
non-differentiable
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Variance Reduction using Kernels
0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1
0
0.5
1
1.5
2
I Y[1], . . . ,Y[N] ordered realizations of a random variable
(c(x , ξi ))I Empirical quantiles: For p ∈ [0, 1] define
Q̃Yp = Y[j], where j ∈ {1, ..,N} withj−1N < p ≤
jN
I Kernel smoothing [Parson 79]
QY ,N,h1−α =∫ 1
0Q̃Yp 1hK
(α−p
h
)dp
=N∑
j=1Y[j]
=:wN,hj︷ ︸︸ ︷∫ jN
j−1N
1hK
(α−p
h
)dp
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Variance Reduction using Kernels
0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1
0
0.5
1
1.5
2
I Y[1], . . . ,Y[N] ordered realizations of a random variable
(c(x , ξi ))I Empirical quantiles: For p ∈ [0, 1] define
Q̃Yp = Y[j], where j ∈ {1, ..,N} withj−1N < p ≤
jN
I Kernel smoothing [Parson 79]
QY ,N,h1−α =∫ 1
0Q̃Yp 1hK
(α−p
h
)dp
=N∑
j=1Y[j]
=:wN,hj︷ ︸︸ ︷∫ jN
j−1N
1hK
(α−p
h
)dp
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Variance Reduction using Kernels
0.9 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 1
0
0.5
1
1.5
2
I Y[1], . . . ,Y[N] ordered realizations of a random variable
(c(x , ξi ))I Empirical quantiles: For p ∈ [0, 1] define
Q̃Yp = Y[j], where j ∈ {1, ..,N} withj−1N < p ≤
jN
I Kernel smoothing [Parson 79]
QY ,N,h1−α =∫ 1
0Q̃Yp 1hK
(α−p
h
)dp =
N∑j=1
Y[j]
=:wN,hj︷ ︸︸ ︷∫ jN
j−1N
1hK
(α−p
h
)dp
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Kernel Estimation of QuantilesI Approximate q1−α(C(x)) = QY1−α
with Yi = c(x ; ξi ) by kernel
estimate
qN,h1−α(C(x)) :=N∑
i=1wN,hi c[i](x) ≤ 0
I The weights depend only on N and h and can be precomputed
I Epanechnikov kernel: K (u) = 34(1− u2)1|u|
-
Kernel Estimation of QuantilesI Approximate q1−α(C(x)) = QY1−α
with Yi = c(x ; ξi ) by kernel
estimate
qN,h1−α(C(x)) :=N∑
i=1wN,hi c[i](x) ≤ 0
I The weights depend only on N and h and can be precomputedI
Epanechnikov kernel: K (u) = 34(1− u
2)1|u|
-
Feasible Region of Kernel-Quantile Formulationc(x , ξ) = ξ1x1 +
ξ2x2 − 1 ξ1, ξ1 ∼ N (0, 1) h ∈ {0.05, 0.01}
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 5000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Feasible Region of Kernel-Quantile Formulationc(x , ξ) = ξ1x1 +
ξ2x2 − 1 ξ1, ξ1 ∼ N (0, 1) h ∈ {0.05, 0.01}
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 5000 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
N = 2000Observations:
I “Looks” smoother and convexI Reduces variance as ξi are
resampledI Large h creates biasI qN,h1−α(C(x)) has more points of
non-differentiabilityI Usually “solved” well with Knitro, but no
proper termination
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Smoothed Empirical CDF
-3 -2 -1 0 1 2 3
x
0
0.2
0.4
0.6
0.8
1
Cu
mu
lative
Dis
trib
utio
n F
un
ctio
n
Nomal CDF
I Quantile QYp = inf{y ∈ R : Φ(y) ≥ p} where Φ(y) is cdf of
Y
I Y1, . . . ,YN (unordered) realizations of a random variableI
Empirical cdf:
Φ̃(y) = 1N
N∑i=1
1≥0(y − Yi )I Smoothed cdf [Azzalini 81]: (K�(t) ≈ 1≥0(t)
differentiable)
ΦN,�(y) = 1N
N∑i=1
K�(y − Yi )
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Smoothed Empirical CDF
-2 -1.5 -1 -0.5 0
c(x, )
0
0.2
0.4
0.6
0.8
1
Cum
ula
tive D
istr
ibution F
unction
Smoothing of empirical CDF
Empirical CDF
I Quantile QYp = inf{y ∈ R : Φ(y) ≥ p} where Φ(y) is cdf of YI
Y1, . . . ,YN (unordered) realizations of a random variableI
Empirical cdf:
Φ̃(y) = 1N
N∑i=1
1≥0(y − Yi )
I Smoothed cdf [Azzalini 81]: (K�(t) ≈ 1≥0(t)
differentiable)
ΦN,�(y) = 1N
N∑i=1
K�(y − Yi )
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Smoothed Empirical CDF
-2 -1.5 -1 -0.5 0
c(x, )
0
0.2
0.4
0.6
0.8
1
Cum
ula
tive D
istr
ibution F
unction
Smoothing of empirical CDF
Empirical CDF
epsilon = 0.1
epsilon = 0.5
I Quantile QYp = inf{y ∈ R : Φ(y) ≥ p} where Φ(y) is cdf of YI
Y1, . . . ,YN (unordered) realizations of a random variableI
Empirical cdf:
Φ̃(y) = 1N
N∑i=1
1≥0(y − Yi )I Smoothed cdf [Azzalini 81]: (K�(t) ≈ 1≥0(t)
differentiable)
ΦN,�(y) = 1N
N∑i=1
K�(y − Yi )
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Differentiable Quantile Estimates
-2 -1.5 -1 -0.5 0
c(x, )
0
0.2
0.4
0.6
0.8
1
Cum
ula
tive D
istr
ibution F
unction
Smoothing of empirical CDF
Empirical CDF
epsilon = 0.1
epsilon = 0.5
I Yi = c(x , ξi )I Smoothed cdf: ΦN,�(y) = 1N
∑Ni=1 K�(y − ci (x))
I Resulting quantile estimate q̂N,�p (C(x)) is the solution y∗
of
p = ΦN,�(y)
I Implicit function theorem guarantees that q̂N,�p (C(x)) exists
and isdifferentiable
(for p ∈ {0, 1N ,2N , . . . ,
N−1N })
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Differentiable Quantile Estimates
-2 -1.5 -1 -0.5 0
c(x, )
0
0.2
0.4
0.6
0.8
1
Cum
ula
tive D
istr
ibution F
unction
Smoothing of empirical CDF
Empirical CDF
epsilon = 0.1
epsilon = 0.5
I Yi = c(x , ξi )I Smoothed cdf: ΦN,�(y) = 1N
∑Ni=1 K�(y − ci (x))
I Resulting quantile estimate q̂N,�p (C(x)) is the solution y∗
of
p = 1N
N∑i=1
K�(y − ci (x))
+ 12N
I Implicit function theorem guarantees that q̂N,�p (C(x)) exists
and isdifferentiable
(for p ∈ {0, 1N ,2N , . . . ,
N−1N })
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Differentiable Quantile Estimates
-2 -1.5 -1 -0.5 0
c(x, )
0
0.2
0.4
0.6
0.8
1
Cum
ula
tive D
istr
ibution F
unction
Smoothing of empirical CDF
Empirical CDF
epsilon = 0.1
epsilon = 0.5
I Yi = c(x , ξi )I Smoothed cdf: ΦN,�(y) = 1N
∑Ni=1 K�(y − ci (x))
I Resulting quantile estimate q̂N,�p (C(x)) is the solution y∗
of
p = 1N
N∑i=1
K�(y − ci (x)) +12N
I Implicit function theorem guarantees that q̂N,�p (C(x)) exists
and isdifferentiable (for p ∈ {0, 1N ,
2N , . . . ,
N−1N })
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Final Estimate
1. Kernel estimate using empirical quantiles c[i](x)
qN,h1−α(C(x)) =N∑
i=1wN,hi c[i](x)
2. Smoothed empirical quantile q̂N,�p (C(x)), solution of
p = 1N
N∑i=1
K�(y − ci (x)) +12N
3. Final estimate
qN,h,�1−α (C(x)) :=N∑
i=1wN,hi q̂
N,�iN
(C(x))
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Final Estimate
1. Kernel estimate using empirical quantiles c[i](x)
qN,h1−α(C(x)) =N∑
i=1wN,hi c[i](x)
2. Smoothed empirical quantile q̂N,�p (C(x)), solution of
p = 1N
N∑i=1
K�(y − ci (x)) +12N
3. Final estimate
qN,h,�1−α (C(x)) :=N∑
i=1wN,hi q̂
N,�iN
(C(x))
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Final Estimate
1. Kernel estimate using empirical quantiles c[i](x)
qN,h1−α(C(x)) =N∑
i=1wN,hi c[i](x)
2. Smoothed empirical quantile q̂N,�p (C(x)), solution of
p = 1N
N∑i=1
K�(y − ci (x)) +12N
3. Final estimate
qN,h,�1−α (C(x)) :=N∑
i=1wN,hi q̂
N,�iN
(C(x))
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Feasible Region with Smoothing (N = 100)Smoothing with =1, 0.5,
0.1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Kernel h=0.01 with smoothing = 0.06
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
x1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
x2
Smoothing with =1, 0.5, 0.1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
x1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
x2
Kernel h=0.01 with smoothing = 0.06
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
x1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
x2
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Example: Portfolio Optimization
maxx∈Rn
E[rT x ] = µT x
s.t. Pr [rT x ≥ −0.05] ≥ 0.95n∑
i=1xi = 1, x ≥ 0
I r ∼ N (µ,Σ), fixed µ ∈ R100 and Σ ∈ R100×100 pos. def.I 100
replications
Iterations with Knitro
N h min iter mean iter max iter500 0.05 15 26.7 54
10,000 0.005 17 29.3 45
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Example: Portfolio Optimization
maxx∈Rn
E[rT x ] = µT x
s.t. Pr [rT x ≥ −0.05] ≥ 0.95n∑
i=1xi = 1, x ≥ 0
I r ∼ N (µ,Σ), fixed µ ∈ R100 and Σ ∈ R100×100 pos. def.I 100
replications
Iterations with Knitro
N h min iter mean iter max iter500 0.05 15 26.7 54
10,000 0.005 17 29.3 45
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Portfolio Optimization
100 500 1000 5000 10000
Number of samples
0.89
0.9
0.91
0.92
0.93
0.94
0.95
0.96
0.97
P(c
(x,
))
h = 5.00e-02
100 500 1000 5000 10000
Number of samples
-3
-2
-1
0
1
2
3
Re
lative
fu
nctio
n e
rro
r (%
)
10-3 h = 5.00e-02
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Portfolio Optimization
100 500 1000 5000 10000
Number of samples
0.87
0.88
0.89
0.9
0.91
0.92
0.93
0.94
0.95
0.96
0.97
P(c
(x,
))
h = 1.00e-02
100 500 1000 5000 10000
Number of samples
-5
-4
-3
-2
-1
0
1
2
3
Re
lative
fu
nctio
n e
rro
r (%
)
10-3 h = 1.00e-02
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Portfolio Optimization
100 500 1000 5000 10000
Number of samples
0.87
0.88
0.89
0.9
0.91
0.92
0.93
0.94
0.95
0.96
0.97
P(c
(x,
))
h = 1.00e-03
100 500 1000 5000 10000
Number of samples
-3
-2
-1
0
1
2
3
Re
lative
fu
nctio
n e
rro
r (%
)
10-3 h = 1.00e-03
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Portfolio Optimization
100 500 1000 5000 10000
Number of samples
0.87
0.88
0.89
0.9
0.91
0.92
0.93
0.94
0.95
0.96
0.97
P(c
(x,
))
h = 1.00e-04
100 500 1000 5000 10000
Number of samples
-3
-2
-1
0
1
2
3
Re
lative
fu
nctio
n e
rro
r (%
)
10-3 h = 1.00e-04
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Joint Chance Constraints
I Original Problem
minx∈X
f (x)
s.t. Pξ[c(x , ξ) ≤ 0] ≥ 1− α
I Scenario vector
C(x) = (c(x , ξ1), . . . , c(x , ξN))
I NLP formulation
minx∈X
f (x)
s.t. qN,h,�1−α (C(x)) ≤ 0
q(Ĉ(x)) := qN,h,�1−α (Ĉ(x))
Note: Constraint no longer differentiable
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Joint Chance Constraints
I Original Problem
minx∈X
f (x)
s.t. Pξ[cj(x , ξ) ≤ 0, j = 1, . . . ,m] ≥ 1− α
I Scenario vector
C(x) = (c(x , ξ1), . . . , c(x , ξN))
I NLP formulation
minx∈X
f (x)
s.t. qN,h,�1−α (C(x)) ≤ 0
q(Ĉ(x)) := qN,h,�1−α (Ĉ(x))
Note: Constraint no longer differentiable
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Joint Chance Constraints
I Original Problem
minx∈X
f (x)
s.t. Pξ[ĉ(x , ξ) ≤ 0] ≥ 1− αĉ(x , ξ) := maxj cj(x , ξi )
I Scenario vector
C(x) = (c(x , ξ1), . . . , c(x , ξN))
I NLP formulation
minx∈X
f (x)
s.t. qN,h,�1−α (C(x)) ≤ 0
q(Ĉ(x)) := qN,h,�1−α (Ĉ(x))
Note: Constraint no longer differentiable
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Joint Chance Constraints
I Original Problem
minx∈X
f (x)
s.t. Pξ[ĉ(x , ξ) ≤ 0] ≥ 1− αĉ(x , ξ) := maxj cj(x , ξi )
I Scenario vector
Ĉ(x) = (ĉ(x , ξ1), . . . , ĉ(x , ξN))
I NLP formulation
minx∈X
f (x)
s.t. qN,h,�1−α (Ĉ(x)) ≤ 0
q(Ĉ(x)) := qN,h,�1−α (Ĉ(x))
Note: Constraint no longer differentiableAndreas Wächter
Nonlinear Programming Formulation of Chance-Constraints
-
Joint Chance Constraints
I Original Problem
minx∈X
f (x)
s.t. Pξ[ĉ(x , ξ) ≤ 0] ≥ 1− αĉ(x , ξ) := maxj cj(x , ξi )
I Scenario vector
Ĉ(x) = (ĉ(x , ξ1), . . . , ĉ(x , ξN))
I NLP formulation
minx∈X
f (x)
s.t. q(Ĉ(x)) ≤ 0q(Ĉ(x)) := qN,h,�1−α (Ĉ(x))
Note: Constraint no longer differentiableAndreas Wächter
Nonlinear Programming Formulation of Chance-Constraints
-
Exact Penalty Function
minx∈X
f (x)
s.t. q(Ĉ(x)) ≤ 0
I Exact penalty function:
φρ(x) = f (x) + ρmax{q(Ĉ(x)), 0} (+penalty for X )
I Model of scenario vector Ĉ(x) = (. . . ,maxj cj(x , ξi ), . .
.)
mĈ (xk , d) =(. . . ,max
j{cj(xk , ξi ) +∇cj(xk , ξi )T d}, . . .
)I Model of penalty function
mφ(xk , d) =f (xk) +∇f (xk)T d + 12dT Bkd+
ρmax{q(Ĉ(xk)) +∇q(Ĉ(xk))T (mĈ (xk , d)− Ĉ(xk)), 0}
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Exact Penalty Function
minx∈X
f (x)
s.t. q(Ĉ(x)) ≤ 0
I Exact penalty function:
φρ(x) = f (x) + ρmax{q(Ĉ(x)), 0} (+penalty for X )I Model of
scenario vector Ĉ(x) = (. . . ,maxj cj(x , ξi ), . . .)
mĈ (xk , d) =(. . . ,max
j{cj(xk , ξi ) +∇cj(xk , ξi )T d}, . . .
)
I Model of penalty functionmφ(xk , d) =f (xk) +∇f (xk)T d +
12d
T Bkd+ρmax{q(Ĉ(xk)) +∇q(Ĉ(xk))T (mĈ (xk , d)− Ĉ(xk)), 0}
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Exact Penalty Function
minx∈X
f (x)
s.t. q(Ĉ(x)) ≤ 0
I Exact penalty function:
φρ(x) = f (x) + ρmax{q(Ĉ(x)), 0} (+penalty for X )I Model of
scenario vector Ĉ(x) = (. . . ,maxj cj(x , ξi ), . . .)
mĈ (xk , d) =(. . . ,max
j{cj(xk , ξi ) +∇cj(xk , ξi )T d}, . . .
)I Model of penalty function
mφ(xk , d) =f (xk) +∇f (xk)T d + 12dT Bkd+
ρmax{q(Ĉ(xk)) +∇q(Ĉ(xk))T (mĈ (xk , d)− Ĉ(xk)), 0}
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Trust Region S`1QP Algorithm
mφ(xk , d) =f (xk) +∇f (xk)T d + 12dT Bkd+
ρmax{q(Ĉ(xk)) +∇q(Ĉ(xk))T (mĈ (xk , d)− Ĉ(xk)), 0}
I Can show: |φρ(xk)−mφ(xk , d)| = O(‖d‖2)I Standard S`1QP-type
trust region algorithm can be applied
I QP Subproblem
mind ,z,t
f (xk) +∇f (xk)T d + 12dT Bkd + ρ t
s.t. q(Ĉ(xk)) +∇q(Ĉ(xk))T Z ≤ t [Z = (z1, . . . , zn)]cj(xk ,
ξi ) +∇cj(xk , ξi )T d ≤ zi for all i , j
t ≥ 0, xk + d ∈ X̃ , ‖d‖∞ ≤ ∆k
I Size of QP proportional to number of nonzeros in ∇q(Ĉ(xk)),
not N
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Trust Region S`1QP Algorithm
mφ(xk , d) =f (xk) +∇f (xk)T d + 12dT Bkd+
ρmax{q(Ĉ(xk)) +∇q(Ĉ(xk))T (mĈ (xk , d)− Ĉ(xk)), 0}
I Can show: |φρ(xk)−mφ(xk , d)| = O(‖d‖2)I Standard S`1QP-type
trust region algorithm can be appliedI QP Subproblem
mind ,z,t
f (xk) +∇f (xk)T d + 12dT Bkd + ρ t
s.t. q(Ĉ(xk)) +∇q(Ĉ(xk))T Z ≤ t [Z = (z1, . . . , zn)]cj(xk ,
ξi ) +∇cj(xk , ξi )T d ≤ zi for all i , j
t ≥ 0, xk + d ∈ X̃ , ‖d‖∞ ≤ ∆k
I Size of QP proportional to number of nonzeros in ∇q(Ĉ(xk)),
not N
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints
-
Summary
I Goal: Formulate chance-constrained problem as NLPI Constrain
the quantile q1−α(x) ≤ 0
I Better linearization than p(x) ≥ (1− α)I Sample-based
empirical quantile
I No need to know probability distributionI Variance reduction
using kernel
I Improves approximation of feasible region, “smoother
boundary”I Quantile estimates using smoothed empirical cdf
I Constraint becomes differentiableI Joint chance
constraints
I Exact merit functionI S`1QP-type trust region algorithm
Andreas Wächter Nonlinear Programming Formulation of
Chance-Constraints