3.6 Lagrangian relaxation Consider a generic ILP problem min {c t x : Ax ≥ b , Dx ≥ d , x ∈ Z n } with integer coefficients. Suppose Dx ≥ d are the ”complicating” constraints in the sense that the ILP without them is ”easy”. Often the linear relaxation and the relaxation by elimination of Dx ≥ d yield weak bounds (e.g., TSP/UFL deleting cut-set/demand constraints) More general setting: min {c t x : Dx ≥ d , x ∈ X ⊆ R n } (1) Idea : Delete the complicating constraints Dx ≥ d and, for each one of them, add to the objective function a term with a multiplier u i , which penalizes its violation and that is ≤ 0 for all feasible solutions of the problem (1). Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 1 / 43
43
Embed
3.6 Lagrangian relaxation - Intranet DEIBhome.deib.polimi.it/.../IP-lagrangian-relaxation-14.pdf3.6 Lagrangian relaxation Consider a generic ILP problem minfctx : Ax b; Dx d;x 2Zng
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
3.6 Lagrangian relaxation
Consider a generic ILP problem
min {c tx : Ax ≥ b, Dx ≥ d , x ∈ Zn}
with integer coefficients.
Suppose Dx ≥ d are the ”complicating” constraints in the sense that the ILP withoutthem is ”easy”.
Often the linear relaxation and the relaxation by elimination of Dx ≥ d yield weakbounds (e.g., TSP/UFL deleting cut-set/demand constraints)
More general setting:min {c tx : Dx ≥ d , x ∈ X ⊆ Rn} (1)
Idea: Delete the complicating constraints Dx ≥ d and, for each one of them, add to theobjective function a term with a multiplier ui , which penalizes its violation and that is≤ 0 for all feasible solutions of the problem (1).
Definition: The Lagrangian dual of the primal problem (2) is
w∗ = maxu≥0
w(u) (4)
Only nonnegativity constraints!
N.B.: By relaxing (dualizing) linear constraints, the objective function remains linear.
The other constraints can be of any type, provided subproblem (3) is ”sufficiently easy”.
For Linear Programs the Lagrangian dual coincides with the LP dual (cf. exercise).
Corollary: (Weak Duality)For each pair of feasible solutions x ∈ {x ∈ X : Dx ≥ d} of the primal problem (2) andu ≥ 0 of the Lagrangian dual (4), we have
i) If x is feasible for the primal problem (2), u is feasible for the Lagrangian dual (4) andw(u) = c t x , then x is optimal for the primal (2) and u is optimal for the dual (4).
ii) In particular w∗ = maxu≥0 w(u) ≤ z∗ = min {c tx : Dx ≥ d , x ∈ X}. If either theprimal or the dual is unbounded (admits no finite optimal solution), then the other one isinfeasible.
Recall: For any pair of primal-dual Linear Programs (LPs) that are bounded we havestrong duality, namely w∗ = z∗.
Observation: Unlike for LPs, discrete optimization problems can have a duality gap,that is w∗ < z∗.
Lagrangian relaxation of equality constraints:
Only difference: the Lagrange multipliers associated to equality constraints areunrestriced in sign, namely ui ∈ R.
If all the m relaxed/dualized constraints are equality constraints, the Lagrangian dual is:
can be solved in linear time by setting to 1 (0) the variables with nonnegative(nonpositive) coefficient and choosing an arbitrary value for those with zero coefficient.
For each j ∈ N, the subproblem (10) can be solved by inspection:
If yj = 0, then xij = 0 for each i and the objective function value is 0.
If yj = 1, it is convenient to serve all clients with positive profit, namely xij = 1 for allindices i such that pij − ui > 0, with an objective function value of∑
i∈M
max{pij − ui , 0} − fj .
Thus wj(u) = max{0,∑
i∈M max{pij − ui , 0} − fj}.
Example: see Chapter 10 of L. Wolsey, Integer Programming, p. 169-170
In some cases, solving the Lagrangian dual (4) yields an optimal solution of the primalproblem (2).
Proposition: If u ≥ 0 and
i) x(u) is an optimal solution of the Lagrangian subproblem (3)
ii) Dx(u) ≥ d
iii) (Dx(u))i = di for each Lagrange multiplier ui > 0 (complementary slacknessconditions),
then x(u) is an optimal solution of the primal poblem (2).
Proof:
Due to (i) we have w∗ ≥ w(u) = c tx(u) + ut(d − Dx(u)) and to (iii) we havec tx(u) + ut(d − Dx(u)) = c tx(u).
According to (ii), x(u) is a feasible solution of the primal (2) and hence c tx(u) ≥ z∗.
Therefore w∗ ≥ c tx(u) + ut(d − Dx(u)) = c tx(u) ≥ z∗ and, since w∗ ≤ z∗, everythingholds with equality and x(u) is an optimal solution of the primal (2).
Observation: If Lagrangian relaxation is applied to equality constraints, conditions (iii)are automatically satisfied, and an optimal solution of the Lagrangian subproblem isoptimal for primal problem (2) if it is feasible.
Illustration: from D. Bertsimas, R. Weismantel, Optimization over integers, DynamicIdeas, 2005, p. 144-146
Consider the ILP problemmin 3x1 − x2
s.t. x1 − x2 ≥ −1
−x1 + 2x2 ≤ 5
3x1 + 2x2 ≥ 3
6x1 + x2 ≤ 15
x1, x2 ≥ 0 integer
- Represent graphically the feasible region and the optimal solutions of the ILP and of itslinear relaxation: x ILP = (1, 2) with zILP = 1 and xLP = (1/5, 6/5) with zLP = −3/5.
- Apply Lagrangian relaxation to the first constraint.
For every u ≥ 0, the Lagrangian subproblem is
w(u) = min(x1,x2)∈X
(3x1 − x2 + u(−1− x1 + x2))
where X is the set of all integer solutions that satisfy all the other constraints.
1) A convex function f : C → R has at least one subgradient in each interior point x ofC .
N.B.: The existence of (at least) one subgradient in any point of int(C), with C convex,is a necessary and sufficient condition for f to be convex on int(C).
2) If f is convex and x ∈ C , ∂f (x) is a nonempty, convex, closed and bounded set.
3) x∗ is a global minimum of f if and only if 0 ∈ ∂f (x∗).
Example: min−1≤x1,x2≤1 f (x1, x2) with f (x1, x2) = max{−x1, x1 + x2, x1 − 2x2} noteverywhere differentiable
Level curves in brown, points of nondifferentiability in green (of type: (t, 0), (−t, 2t) and(−t,−t) for t ≥ 0), global minimum x∗ = (0, 0).
If xk = (1, 0) and we consider the subgradient γk
= (1, 1) ∈ ∂f (xk), f (x) increases along
the half-line {x ∈ R2 : x = xk − αkγk, αk ≥ 0} but if the step αk is sufficiently small
xk+1 = xk − αkγk
is closer to x∗.
From Chapter 8, Bazaraa et al., Nonlinear Programming, Wiley, 2006, p. 436-437Edoardo Amaldi (PoliMI) Ottimizzazione A.A. 2013-14 21 / 43
Theorem:
If f is convex, lim‖x‖→∞ f (x) = +∞, limk→∞ αk = 0 and∑∞
k=0 αk =∞, then thesubgradient method terminates after a finite number of iterations with an optimalsolution x∗ or it generates an infinite sequence {xk} which admits a subsequenceconverging to x∗.
Choice of the stepsize:
In practice, sequences {αk} such that limk→∞ αk = 0,∑∞
k=0 αk =∞ (e.g., αk = 1/k)are too slow.
An alternative is to choose αk = α0ρk , for a given ρ < 1. A more sophisticated and
popular rule is
αk = εkf (xk)− f
‖γk‖2 ,
where 0 < εk < 2 and f is the optimal value (minimum) f (x∗) or an estimate.
Stopping criterion: prescribed maximum number of iterations because, even if0 ∈ ∂f (xk) that subgradient may non be considered in xk .
N.B.: Since it is not a monotone method, one needs to store the best solution xk foundso far.
The method can be easily extended to the case with bound constraints by including asimple projection step at each iteration.
Recall that a Hamiltonian cycle is a 1-tree (i.e., a spanning tree on nodes {2, . . . , n} plustwo edges incident to node 1) in which all nodes have exactly two incident edges.
Since∑
e∈E cexe +∑
i∈V ui (2−∑
e∈δ(i) xe) =∑
e={i,j}∈E (ce − ui − uj)xe + 2∑
i∈V ui ,
relaxing/dualizing the degree constraints (14) for all nodes except for node 1, we obtain
where u1 = 0 and E(S) = {{i , j} ∈ E : i ∈ S , j ∈ S}.
N.B.: The set of feasible solutions of this problem correspond exactly to the set of all1-trees.
To find a minimum cost 1-tree: determine with a greedy algorithm (e.g., Kruskal orPrim) a minimum cost spanning tree on the nodes {2, . . . , n} and select two edges ofsmallest cost among those incident to node 1.
trivial solution: assign each process i ∈ I to machine j ∈ J with min cij + ujwij .
N.B.: If the integrality conditions were relaxed, the solution would not change (thesubproblem has the ”integrality property”).
2) Relaxing the assignment constraints:
w2(v) = min∑
i∈I∑
j∈J cijxij −∑
i∈I vi (∑
j∈J xij − 1)
s.t.∑
i∈I wijxij ≤ bj ∀j ∈ J
xij ∈ {0, 1} ∀i ∈ I , ∀j ∈ J
Since the variables xij corresponding to different machines j are not linked, thesubproblem decomposes into |J| independent binary knapsack subproblems, withprofit −cij + vi and weight wij for item i ∈ I .
3) Relaxing all the constraints, we obtain the Lagrangian subproblem:
w3(u, v) = min∑
i∈I∑
j∈J cijxij −∑
i∈I vi (∑
j∈J xij − 1)−∑
j∈J uj(bj −∑
i∈I wijxij)
xij ∈ {0, 1} ∀i ∈ I , ∀j ∈ J
trivial solution: set xij = 1 if cij − vi + ujwij < 0, xij = 0 if cij − vi + ujwij > 0 andarbitrarily xij = 0 or xij = 1 if cij − vi + ujwij = 0.
Observations:
i) The first and third Lagrangian relaxations are as weak as (not stronger than) the linearrelaxation.
ii) Since the ideal formulation for the binary knapsack problem contains many otherinequalities (e.g., cover inequalities), the second Lagrangian relaxation provides apotentially stronger bound.