Lipschitzian Piecewise Smooth Minimization via Algorithmic Differentiation Sabrina Fiege 1 Andreas Griewank 2 Andrea Walther 1 1 Institut für Mathematik, Universität Paderborn 2 Yachay Tech, Ecuador 18th Euro AD Workshop 2015 Paderborn, Germany
Lipschitzian Piecewise Smooth Minimizationvia Algorithmic Differentiation
Sabrina Fiege1 Andreas Griewank2 Andrea Walther1
1Institut für Mathematik, Universität Paderborn2Yachay Tech, Ecuador
18th Euro AD Workshop 2015Paderborn, Germany
Motivation
New Optimization Approach
Our goal: Locate local optima of a piecewise smooth function by
successive approximation by piecewise linear models and⇒ Piecewise Linearizationexplicit handling of kink structure in PL model.
Hierarchy of problems:
locally Lipschitz continuous
∪piecewise smooth (PS)
∪piecewise linear (PL)
∪piecewise linear and convex
S. Fiege, A. Griewank, and A. Walther 1 / 30 December 1, 2015
Motivation
New Optimization ApproachOur goal: Locate local optima of a piecewise smooth function by
successive approximation by piecewise linear models and⇒ Piecewise Linearizationexplicit handling of kink structure in PL model.
Hierarchy of problems:
locally Lipschitz continuous
∪piecewise smooth (PS)
∪piecewise linear (PL)
∪piecewise linear and convex
Lipschitz Optimization based on gray-box piecewise linearization,A. Griewank, A. Walther, SF, T. Bosse, Mathematical Programming, 2015
S. Fiege, A. Griewank, and A. Walther 1 / 30 December 1, 2015
Motivation
New Optimization ApproachOur goal: Locate local optima of a piecewise smooth function by
successive approximation by piecewise linear models and⇒ Piecewise Linearizationexplicit handling of kink structure in PL model.
Hierarchy of problems:
locally Lipschitz continuous
∪piecewise smooth (PS)
∪ →piecewise linear (PL)
∪piecewise linear and convex
Lipschitz Optimization based on gray-box piecewise linearization,A. Griewank, A. Walther, SF, T. Bosse, Mathematical Programming, 2015
Work in Progress!Today’s talk.
S. Fiege, A. Griewank, and A. Walther 1 / 30 December 1, 2015
Motivation
Observations
Solving min f (x) with f PL is not easy:
Global minimization is NP-hard.
Steepest descent with exact linesearch may fail.
Zeno behaviour possible,i.e., solution trajactory with infinitenumber of direction changes in afinite amount of time.
J.-B. Hiriart-Urruty and C. Lemaréchal,Convex Analysis and Minimization Algorithms I,
Springer, 1993
y
x-100-50
050
-20
-10
0
10
20
-400
-300
-200
-100
f(x,
y)
0
100
200
−100 −50 0 50−20
−15
−10
−5
0
5
10
15
20
x1
x 2
Nondifferentiable points of f
f0(x)
f2(x)
f−2(x)
f1(x)
f−1(x)x0=(9,−3)
S. Fiege, A. Griewank, and A. Walther 2 / 30 December 1, 2015
Motivation
Assumptions
We consider Lipschitzian piecewise smooth funtions
f : Rn → R.
All nondifferentiabilities are incorporated by abs().
min(u, v) = (v + u − abs(v − u))/2,max(u, v) = (v + u + abs(v − u))/2and complementarity conditions are covered.
Handling of abs() is included in algorithmic differentiation tool ADOL-C.
S. Fiege, A. Griewank, and A. Walther 3 / 30 December 1, 2015
AD Drivers
Outline
1 Motivation
2 AD DriversPiecewise LinearizationDirectional Active GradientAbs-normal Form
3 Lipschitzian Piecewise Smooth MinimizationMinimization of Piecewise Linear FunctionsMinimization of Piecewise Smooth FunctionNumerical Results
4 Conclusion and Outlook
S. Fiege, A. Griewank, and A. Walther 4 / 30 December 1, 2015
AD Drivers Piecewise Linearization
Adapted Evaluation Procedure for PS Objectives
vi−n = xi i = 1 ... nzi = ψi (vj )j≺iσi = sign(zi ) i = 1 ... svi = σizi = abs(zi )y = ψs(vj )j≺s
Table : Reduced evaluation procedure
s ∈ N number of evaluations of absolut value function.σ = {−1, 0, 1}s is called signature vector.z ∈ Rs is called switching vector.
S. Fiege, A. Griewank, and A. Walther 5 / 30 December 1, 2015
AD Drivers Piecewise Linearization
Piecewise Linearization
Construction of tangent approximation for each elemental function
∆vi = ∆vj ±∆vk for vi = vj ± vk∆vi = vj ∗∆vk + vk ∗∆vj for vi = vj ∗ vk∆vi = ϕ′(vj )j≺i ∗∆(vj )j≺i for vi = ϕi (vj )j≺i 6= abs(vj )
∆vi = abs(vj + ∆vj )− vi for vi = abs(vj )
One obtains the piecewise linearization
fPL,x (∆x) = f (x) + ∆f (x ; ∆x)
of the original PS function f at a point x with the argument ∆x .Andreas Griewank. On stable piecewise linearization and generalized algorithmic differentiation,Optimization Methods & Software, 28(6), 1139–1178 2013.
S. Fiege, A. Griewank, and A. Walther 6 / 30 December 1, 2015
AD Drivers Piecewise Linearization
Example: Minimum and MaximumRemark: One obtains as the linearization of the min and max functions, themaximum and minimum of the linearized arguments.
−4 −3 −2 −1 0 1 2 3 4−5
0
5
10
15
20
25
x
x2−1
−0.1*(x−2)3+1
max(x2−1,−0.1*(x−2)
3+1)
linearization of x2−1
linearization of −0.1*(x−2)3+1
maximum of the two linearizations
−4 −3 −2 −1 0 1 2 3 4−5
0
5
10
15
20
25
x
x2−1
−0.1*(x−2)3+1
min(x2−1,−0.1*(x−2)
3+1)
linearization of x2−1
linearization of −0.1*(x−2)3+1
minimum of the two linearizations
max{x2 − 1,−0.1(x − 2)3 + 1} min{x2 − 1,−0.1(x − 2)3 + 1}
S. Fiege, A. Griewank, and A. Walther 7 / 30 December 1, 2015
AD Drivers Piecewise Linearization
AD Drivers provided by ADOL-C
zos_pl_forward(tag,1,n,1,x,y,z)
Evaluates the PL at x , returns the function value y and the switchingvector z at that point.
s=get_num_switches(tag)
Returns the number of evaluations of the absolut value function.
fos_pl_forward(tag,1,n,x,deltax,y,deltay,z,deltaz)
Computes the increment ∆y = ∆f (x ; ∆x). Returns additionally theswitching vector z and its linearization ∆z.
ADOL-C: https://projects.coin-or.org/ADOL-C
S. Fiege, A. Griewank, and A. Walther 8 / 30 December 1, 2015
AD Drivers Directional Active Gradient
Selection Functions and Limiting Gradients
PS functions can be represented by selection functions fσ as
f (x) ∈ {fσ(x) : σ ∈ E ⊂ {−1, 0, 1}s}.
where the selection functions fσ are continuously differentiable on openneigborhoods of points.
The Clarke subdifferential is given by
∂f (x) ≡ conv(∂Lf (x)) with ∂Lf (x) ≡ {∇fσ(x) : fσ(x) = f (x)}
where the elements of ∂Lf (x) are called limiting gradients.
S. Fiege, A. Griewank, and A. Walther 9 / 30 December 1, 2015
AD Drivers Directional Active Gradient
AD Drivers provided by ADOL-C
A directionally active gradient g is given by
g ≡ g(x , d) ∈ ∂Lf (x) such that f ′(x , d) = gT d
and g(x ; d) equals ∇fσ(x) of a locally differentiable selection function fσ.
directional_active_gradient(tag,n,x,d,g)
Returns g(x ; d) at a given point x and a given direction d .
S. Fiege, A. Griewank, and A. Walther 10 / 30 December 1, 2015
AD Drivers Abs-normal Form
The abs-normal form for PL functions (1)
Example
F (x1, x2) = x1 + |z1|+ |z3|with z1 = x1 − x2 z2 = x2 z3 = x1 − |z2|
z1z2z3y
=
0000
+
1 −1 0 0 00 1 0 0 01 0 0 −1 01 0 1 0 1
x1x2|z1||z2||z3|
S. Fiege, A. Griewank, and A. Walther 11 / 30 December 1, 2015
AD Drivers Abs-normal Form
The Abs-normal Form for PL Functions (2)
Definition Abs-normal form for PL F : Rn → R[
zy
]=
[c1c2
]+
[Z LaT bT
] [x|z|
]Z ∈ Rs×n, L ∈ Rs×s, a ∈ Rn, b ∈ Rs c1 ∈ Rs, c2 ∈ R
L is stricly lower triangular
Σ ≡ diag(σ) and |z| = Σ · zPL function fPL approximation of PS function.
PL fPL,x ≡ y can be written as abs-normal form.
Andreas Griewank. On stable piecewise linearization and generalized algorithmic differentiation,Optimization Methods & Software, 28(6), 1139–1178 2013.
S. Fiege, A. Griewank, and A. Walther 12 / 30 December 1, 2015
AD Drivers Abs-normal Form
The Abs-normal Form for PL Functions (2)
Definition Abs-normal form for PL F : Rn → R[
zy
]=
[c1c2
]+
[Z LaT bT
] [x
Σ · z
]Z ∈ Rs×n, L ∈ Rs×s, a ∈ Rn, b ∈ Rs c1 ∈ Rs, c2 ∈ R
Take the first row, solve for z and plug into the 2nd
fσ(x) ≡ y = c2 + bT Σ(I − LΣ)−1c1︸ ︷︷ ︸≡γσ(x)
+ (aT + bT Σ(I − LΣ)−1Z )︸ ︷︷ ︸≡gσ(x)
x
The abs-normal form represents a PL function fσ : Rn → R with
fσ(x) = γσ(x) + gσ(x) · x
S. Fiege, A. Griewank, and A. Walther 12 / 30 December 1, 2015
AD Drivers Abs-normal Form
AD Driver provided by ADOL-C
Definition Abs-normal form for PL F : Rn → R[
zy
]=
[c1c2
]+
[Z LaT bT
] [x
Σ · z
]Z ∈ Rs×n, L ∈ Rs×s, a ∈ Rn, b ∈ Rs c1 ∈ Rs, c2 ∈ R
abs_normal(tag,n,x,sigma,y,z,c1,c2,a,b,Z,L)
Computes a PL for a given PS function f and a given point x .Remark: c1, c2, a, b, Z and L only depent on the PS function f .
S. Fiege, A. Griewank, and A. Walther 13 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PL Functions
Outline
1 Motivation
2 AD DriversPiecewise LinearizationDirectional Active GradientAbs-normal Form
3 Lipschitzian Piecewise Smooth MinimizationMinimization of Piecewise Linear FunctionsMinimization of Piecewise Smooth FunctionNumerical Results
4 Conclusion and Outlook
S. Fiege, A. Griewank, and A. Walther 14 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PL Functions
Description of Polyhedral Structure
The polyhedra Pσ ≡ {x ∈ Rn : σ(x) = σ}are relatively open and convex.
are mutually disjoint, their union is the whole Rn.
1
0.5x0
-0.5
-1
-3
y
-2
-1
f(x,
y)
-1-0.5
00.5
1
0
1
2
−2 −1 0 1 2−2
−1
0
1
2
x1
x2
σ=(−1,−1)
σ=(−1,1)
σ=(−1,0) ↓
σ=(1,−1)
σ=(1,1)
↑ σ=(1,0)
← σ=(0,−1)
← σ=(0,1)
← σ=(0,0)
S. Fiege, A. Griewank, and A. Walther 15 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PL Functions
Description of Polyhedral Structure
The polyhedra Pσ ≡ {x ∈ Rn : σ(x) = σ}are relatively open and convex.
are mutually disjoint, their union is the whole Rn.Further properties:
fσ is essentially active at all points in P̄σ providedPσ is open.
The corresponding σ are are called essential and
E = {σ ∈ {−1, 0, 1}s : ∅ 6= Pσ open}.
The signature vectors are partially ordered by
σ � σ̃ :⇐⇒ σ2i ≤ σ̃i σi for 1 ≤ i ≤ s.
1
0.5x0
-0.5
-1
-3
y
-2
-1
f(x,
y)
-1-0.5
00.5
1
0
1
2
−2 −1 0 1 2−2
−1
0
1
2
x1
x2
σ=(−1,−1)
σ=(−1,1)
σ=(−1,0) ↓
σ=(1,−1)
σ=(1,1)
↑ σ=(1,0)
← σ=(0,−1)
← σ=(0,1)
← σ=(0,0)
S. Fiege, A. Griewank, and A. Walther 15 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PL Functions
Solution of PL Function by PLMin()
PLMin(): Preconditions: x0 ∈ Rn, q ≥ 0, ∆x = 0, σ = σ(x0)
1 Determine solution ∆x of local QP on current polyhedron Pσ.
2 Compute bundle G.
3 Compute direction d that identifies the new polyhedra Pσ.
4 Update xk+1 = xk + ∆x , k = k + 1
5 If d = 0: STOP, else go to 1.
S. Fiege, A. Griewank, and A. Walther 16 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PL Functions
Step 1: Solve local quadratic problemSolve local QP on current, open polyhedron Pσ.
min∆x
fσ +q2‖∆x‖2,
s.t. eTi (z(xk ) +∇z(xk )T ∆x) =
{≥ 0 σ > 0≤ 0 σ < 0
This yields xk+1 = xk + ∆x , σ̂ = σ(xk+1), active set  = {i|σ̂ = 0 or λi 6= 0}.
−100 −50 0 50−20
−15
−10
−5
0
5
10
15
20
x
y
S. Fiege, A. Griewank, and A. Walther 17 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PL Functions
Step 2 & 3: Compute bundle G and direction d (1)
Given q ≥ 0 and ∅ 6= G ⊂ ∂Lf (x). Compute new direction d by
d(x) = shortest(qx ,G)
= argmin
||d ||∣∣∣∣∣∣d =
m∑j=1
βjgj − qx , gj ∈ G, βj ≥ 0,m∑
j=1
βj = 1
.Interpretation of d :
d = 0 Stationary point
(g + qx)T d < 0 Direction of descent
(g + qx)T d > 0 Use computeStep() to collect further gradients g
S. Fiege, A. Griewank, and A. Walther 18 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PL Functions
Step 2 & 3: Compute bundle G and direction d (2)
Interpretation of d :
d = 0 Stationary point
(g + qx)T d < 0 Direction of descent
(g + qx)T d > 0 Use computeStep() to collect further gradients g
computeStep(x,q,G)repeat
{ d = −shortest(qx ,G)g = g(x ; d)G = G ∪ {g} }
until (g + qx)>d ≤ −‖d‖2G = ∅return d , G
S. Fiege, A. Griewank, and A. Walther 19 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PL Functions
Step 2 & 3: Compute bundle G and direction d (2)
Interpretation of d :
d = 0 Stationary point
(g + qx)T d < 0 Direction of descent
(g + qx)T d > 0 Use computeStep() to collect further gradients g
computeStep(x,q,G)repeat
{ d = −shortest(qx ,G)g = g(x ; d)G = G ∪ {g} }
until (g + qx)>d ≤ −‖d‖2G = ∅return d , G
−100 −50 0 50−20
−15
−10
−5
0
5
10
15
20
x
y
S. Fiege, A. Griewank, and A. Walther 19 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PL Functions
Convergence of Algorithm
Argument space is divided only into finitely many polyhedra.
Function value is decreased each time we switch from one polyheron toanother.
Algorithm must reach stationary point x̂ after finitely many steps
S. Fiege, A. Griewank, and A. Walther 20 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PS Function
LiPsMinLiPsMin
Lipschitzian Piecewise Smooth Minimization
LiPSMin(): Let f be a PS function. Preconditions: x0 ∈ Rn, q ≥ 0for k = 0, 1, 2...
1 Generate local model f̂xk (∆x) = fPL,xk (∆x) +q2 ||∆x ||
2 with q ≥ 0.
2 Compute ∆x as stationary point of local model s.t. f (xk + ∆x) < f (xk ).
3 Update xk+1 = xk + ∆x .
4 If ||∆x || = 0: STOP5 Update q = max{q, q̂(xk )} and k = k + 1.
S. Fiege, A. Griewank, and A. Walther 21 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PS Function
Step 1: Generate Local Model
Piecewise Linearization can be written in abs-normal form.
PL is of second order in the distance to the base point.
Add quadratic term to ensure the boundedness.
Generate local model f̂xk (∆x) = fPL,xk (∆x) +q2 ||∆x ||
2 with q ≥ 0.
Example: f : R2 → R, f (x1, x2) = max{x22 −max{x1, 0}, 0}
1
0.5x0
-0.5
-1
-3
y
-2
-1
f(x,
y)
0
1
-1-0.5
00.5
1
2
1
0.5x0
-0.5
-1
-3
y
-2
-1
f(x,
y)
-1-0.5
00.5
1
0
1
2
PS function and its local model at x0 = (−1, 1) with q = 0.01
S. Fiege, A. Griewank, and A. Walther 22 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PS Function
Step 1: Generate Local Model
Piecewise Linearization can be written in abs-normal form.
PL is of second order in the distance to the base point.
Add quadratic term to ensure the boundedness.
Generate local model f̂xk (∆x) = fPL,xk (∆x) +q2 ||∆x ||
2 with q ≥ 0.
Example: f : R2 → R, f (x1, x2) = max{x22 −max{x1, 0}, 0}
1
0.5x0
-0.5
-1
-3
y
-2
-1
f(x,
y)
0
1
-1-0.5
00.5
1
2
1
0.5x0
-0.5
-1
-3
y
-2
-1
f(x,
y)
-1-0.5
00.5
1
0
1
2
PS function and its local model at x0 = (−1, 1) with q = 0.01
S. Fiege, A. Griewank, and A. Walther 22 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PS Function
Step 2 & 3: Optimization of Local Model (1)
Compute ∆x as stationary point of the local model f̂xk by PLMin().
Exploit structure of the domain of the function.
Update xk+1 = xk + ∆x .
Example: f : R2 → R, f (x1, x2) = max{x22 −max{x1, 0}, 0}
1
0.5x0
-0.5
-1
-3
y
-2
-1
f(x,
y)
-1-0.5
00.5
1
0
1
2
1
0.5x0
-0.5
-1
-3
y
-2
-1
f(x,
y)
0
1
-1-0.5
00.5
1
2
Minimization of local model and new iterate x̂ = x1 = (−1, 0.5) of PS function
S. Fiege, A. Griewank, and A. Walther 23 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PS Function
Step 2 & 3: Optimization of Local Model (1)
Compute ∆x as stationary point of the local model f̂xk by PLMin().
Exploit structure of the domain of the function.
Update xk+1 = xk + ∆x .
Example: f : R2 → R, f (x1, x2) = max{x22 −max{x1, 0}, 0}
1
0.5x0
-0.5
-1
-3
y
-2
-1
f(x,
y)
-1-0.5
00.5
1
0
1
2
1
0.5x0
-0.5
-1
-3
y
-2
-1
f(x,
y)
0
1
-1-0.5
00.5
1
2
Minimization of local model and new iterate x̂ = x1 = (−1, 0.5) of PS function
S. Fiege, A. Griewank, and A. Walther 23 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PS Function
Step 2 & 3: Optimization of Local Model (2)
PLMin() does not guarantee that f (xk + ∆x) < f (xk ). Therefore we put in athird routine:
GuaranteeDescent(): // Precondition: x ,∆x ∈ Rn, q ≥ 0
for k = 0, 1, 2...
1 Set ∆x = 0 .
2 Call PLMin(x,∆x ,q).
3 Check if f (x + ∆x) < f (x) then STOP else increase q and go to 1.
Ongoing work: Prove that the algorithm above terminates after finitely manyiterations.
S. Fiege, A. Griewank, and A. Walther 24 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PS Function
Step 5: Penalty coefficient
Update q = max{0.9q + 0.1q̂(xk ,∆x), q̂(xk ,∆x), q0} with∆x = xk+1 − xk and
q̂(xk ,∆x) =|f (xk+1)− f (xk )− fPL(xk ; ∆x)|
‖∆x‖2
Quadratic coefficient q ensures that local model is also bounded below.Example: For f (x) = x2 one obtains at x̄ = 1 the f̂x̄ (x̄ ; x − x̄) = 2x − 1.
Example: f : R2 → R, f (x1, x2) = max{x22 −max{x1, 0}, 0}
1
0.5x0
-0.5
-1
-3
y
-2
-1
f(x,
y)
0
1
-1-0.5
00.5
1
2
x
y
-3
-2
f(x,
y) -1
1
0.5
0
-0.5
-1
-1-0.5
00.5
1
0
1
1
0.5x
0
-0.5
-1
-3
y
-2
-1
0f(x,
y)
1
-1-0.5
00.5
1
2
3
4
PS function, PL function with and without quadratic term with q = 1
S. Fiege, A. Griewank, and A. Walther 25 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PS Function
Step 5: Penalty coefficient
Update q = max{0.9q + 0.1q̂(xk ,∆x), q̂(xk ,∆x), q0} with∆x = xk+1 − xk and
q̂(xk ,∆x) =|f (xk+1)− f (xk )− fPL(xk ; ∆x)|
‖∆x‖2
Quadratic coefficient q ensures that local model is also bounded below.Example: For f (x) = x2 one obtains at x̄ = 1 the f̂x̄ (x̄ ; x − x̄) = 2x − 1.
Example: f : R2 → R, f (x1, x2) = max{x22 −max{x1, 0}, 0}
1
0.5x0
-0.5
-1
-3
y
-2
-1
f(x,
y)
0
1
-1-0.5
00.5
1
2
x
y
-3
-2
f(x,
y) -1
1
0.5
0
-0.5
-1
-1-0.5
00.5
1
0
1
1
0.5x
0
-0.5
-1
-3
y
-2
-1
0f(x,
y)
1
-1-0.5
00.5
1
2
3
4
PS function, PL function with and without quadratic term with q = 1
S. Fiege, A. Griewank, and A. Walther 25 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Minimization of PS Function
Convergence of Algorithm
Convergence of LiPSMin
Under the assumptions
PS functionf has bounded level set with x0 the starting point,
{qk} is bounded, {∆xk} and {q̂k} are uniformly boundedand GuaranteeDescent() terminates after finitely many iterations,
all cluster points x∗ of the infinite sequence {xk}k∈N generated by LiPSMinsatisfy the first order minimality condition f ′(x∗, ·) ≥ 0 for Lipschitzianpiecewise smooth problems.
S. Fiege, A. Griewank, and A. Walther 26 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Numerical Results
Example
f : R2 7→ R, f (x1, x2) = max{−100, 3x1 − 2x2, 2x1 − 5x2, 3x1 + 2x2, 2x1 + 5x2}
−80 −60 −40 −20 0 20 40−15
−10
−5
0
5
10
15
x1
x2
f0(x)
f2(x)
f−2(x)
f1(x)
f−1(x) x0=(9,−3)
x*=(−50,0)
S. Fiege, A. Griewank, and A. Walther 27 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Numerical Results
Results: Chained LQ
f (x) =n−1∑i=1
max−xi − xi+1,−xi − xi+1 + x2i + x2i+1 − 1
with x0i = −0.5, ∀i = 1, ..., n and f (x∗) = −(n − 1)√
2
n f ∗ #f #g #QP #iter5 -5.657 29 63 63 14
LiPsMin 10 -12.728 21 57 57 1020 -26.87 21 660 659 105 -5.657 88 88 - 51
MPBNGC 10 -12.728 123 123 - 10620 -26.87 1011 1011 - 1000
MPBNGC is a multiobjective proximal bundle method for nonconvex,nonsmooth (nondifferentiable) and generally constrained minimization, seeM.M.Mäkelä. Multiobjective Proximal Bundle Method for Nonconvex, Nonsmooth Optimization:Fortran Subroutine MPBNGC 2.0, Reports of the Department of Mathematical InformationTechnology, Series B, Scientific computing, No. B 13/2003, University of Jyväskylä, 2003.
S. Fiege, A. Griewank, and A. Walther 28 / 30 December 1, 2015
Lipschitzian Piecewise Smooth Minimization Numerical Results
Results: Active faces
f (x) = max1≤i≤n
{g(−n∑
i=1
xi ), g(xi ), } with g(y) = ln(|y |+ 1)
with x0i = 1, ∀i = 1, ..., n and f (x∗) = 0
n f ∗ #f #g #QP #iter5 1e-15 5 6 6 2
LiPsMin 10 1e-15 7 7 7 320 1e-15 9 11 11 45 0 18 18 - 15
MPBNGC 10 1e-11 1000 1000 - 99420 1e-11 1000 1000 - 991
Test problems, seeM. Haarala, K.Miettinen, M.M.Mäkelä.New Limited Memory Bundle Method for Large Scale Nonsmooth Optimization,OMS, 2007.
S. Fiege, A. Griewank, and A. Walther 29 / 30 December 1, 2015
Conclusion and Outlook
Conclusion and Outlook
AD drivers provided by ADOL-C
Minimization method for Lipschitzian PS functions: LiPsMin
Numerical results
Future Work:
Convergence theory
Strategy for building the bundle
Thank you for your attention! Questions?
S. Fiege, A. Griewank, and A. Walther 30 / 30 December 1, 2015
Conclusion and Outlook
Conclusion and Outlook
AD drivers provided by ADOL-C
Minimization method for Lipschitzian PS functions: LiPsMin
Numerical results
Future Work:
Convergence theory
Strategy for building the bundle
Thank you for your attention! Questions?
S. Fiege, A. Griewank, and A. Walther 30 / 30 December 1, 2015
MotivationAD DriversPiecewise LinearizationDirectional Active GradientAbs-normal Form
Lipschitzian Piecewise Smooth MinimizationMinimization of Piecewise Linear FunctionsMinimization of Piecewise Smooth FunctionNumerical Results
Conclusion and Outlook