Michal Pióro 1 dual separation in LP and column generation In many problems most of potential variables are not used in the primal problem formulation. Constraints correspond to primal variables that are used. It can happen that we are able to produce one (or more) new dual constraints (corresponding to primal variables that are not considered in the problem) violated by current optimal dual solution π π π*. Then by adding these new constraints we are potentially able to decrease the optimal dual solution (since we are add constraints). If we decrease the dual maximum then we decrease the primal minimum as W* = F*. Besides, if we are not able to eliminate current π π π* then the current primal solution is optimal in the general sense (with all potential primal variables included).
48
Embed
dual separation in LP and column generationwits.gist.ac.kr/uploads/Main/netwrok optimization - parts... · · 2011-11-22MichałPióro 1 dual separation in LP and column generation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Michał Pióro 1
dual separation in LP and column generation
In many problems most of potential variables are not used in theprimal problem formulation.
Constraints correspond to primal variables that are used. It can happen that we are able to produce one (or more) new dual
constraints (corresponding to primal variables that are not considered in the problem) violated by current optimal dual solution ππππ*.
Then by adding these new constraints we are potentially able to decrease the optimal dual solution (since we are add constraints).
If we decrease the dual maximum then we decrease the primal minimum as W* = F*.
Besides, if we are not able to eliminate current ππππ* then the current primal solution is optimal in the general sense (with all potential primal variables included).
Michał Pióro 2
example - path generation:
flow allocation problem – LP formulation
variables xdp flow realizing demand d on path p
constraints Σp xdp = hd d=1,2,…,D Σd Σp δedpxdp ≤ ce e=1,2,…,E flow variables are continuous and non-negative
D+E non-zero flows at most depending on the number of saturated links if all links unsaturated: D flows only!
How to define the proper lists of paths? By path generation.
Michał Pióro 3
flow allocation problem
– adjusted formulation
variables xdp flow realizing demand d on path p z auxiliary variable
objective minimize z
constraints Σp xdp = hd d=1,2,…,D (λd - unconstrained) Σd Σp δedpxdp ≤ ce+ z e=1,2,…,E (πe ≥ 0) flow variables are continuous and non-negative, z is continuous
Michał Pióro 4
dual
L(x,z; ππππ,λλλλ) = z + Σd λd(hd - Σp xdp) + Σe πe(Σd Σp δedpxdp - ce- z) xdp ≥ 0 for all (d,p)
if we can find a path shorter than ΣΣΣΣe δδδδedpππππe* then we will get a more constrained dual problem and hence have a chance to improve (decrease) the optimal dual value
shortest path algorithm can be used for finding shortest paths withrespect to ππππ*
Michał Pióro 6
path generation - how it works
We can start with only one single path on the list for each demand (Pd = 1 for all d).
Then we solve the extended problem and add one shortest path foreach demand d (if such a path, i.e., a path shorter than all the paths on the current list for demand d, exists).
This process will terminate typically (although not always) after onlyseveral steps.
Cycling may occur, so it is better not to remove paths.
Section 5.4.2
Michał Pióro 7
path generation
note that in the link-path formulation the lists of candidate paths arepredefined
using of full lists is not realistic (exponential number of paths)
optimal dual multipliers πe assocoated with capacity constraints areused to generate new shortest paths
the paths can be generated using Dijkstra (or some other shortest pathalgorithm), e.g., with limited number of hops
Michał Pióro 8
Benders’ decomposition:
projecting out some variables
LP:
maximize z = cysubject to Ax + Dy ≤ b, x, y ≥ 0
The problem has the feasible polyhedron P in (x,y)-space. The idea is to get an equivalent problem only in variables y in
polyhedron Q in the y-space. Q is the projection of P onto the y-space.
LP:
maximize z = cysubject to Ey ≤ g, y ≥ 0
Lasdon, Minoux
Michał Pióro 9
Benders’ decomposition:
dimensioning problem (LP or MIP)
variables xdp flow realizing demand d on path p (continuous) ye link capacities, continuous (LP) or integer (MIP)
objective minimize Σe ξeye
constraints Σp xdp = hd d=1,2,…,D Σd Σp δedpxdp ≤ ye e=1,2,…,E
Michał Pióro 10
Benders’ decomposition
iterative process
eliminate flow variables x introduce new constraints on y
Section 5.4.3
Michał Pióro 11
BD scheme
Step 0 ΩΩΩΩ := ye ≥ 0 : e=1,2,...,E (set of inequality constraints on y)
Step 1: solve master problem (y*) minimize Σe ξeye
subject to ΩΩΩΩ Step 2: perform the feasibility test ( ππππ*,λλλλ*)
maximize W(ππππ,λλλλ) = Σd λdhd - Σe πeye* subject to
Σe πe = 1 λd ≤ Σe δedpπe d=1,2,…,D p=1,2,...,Pd
πe ≥ 0 e=1,2,...,Eif W(ππππ*,λλλλ*) > 0 then add inequality Σd λd*hd - Σe πe* ye ≤ 0 to ΩΩΩΩ
Step 3: if new inequality is added then go to Step 1, otherwise STOP
Michał Pióro 12
BD scheme - remarks
In MP there are no flow variables FT is a dual to the flow allocation problem:
objective minimize z constraints
Σp xdp = hd d=1,2,…,D (λd - unconstrained) Σd Σp δedpxdp ≤ ye*+ z e=1,2,…,E (πe ≥ 0) flow variables are continuous and non-negative, z is
continuous Problem: too many constraints Can be much effective than the direct approach for the MIP
version
Michał Pióro 13
IPs and MIPs
Integer Program (IP)
maximize z = cxsubject to Ax ≤ b, x ≥ 0 (linear constraints)
x integer (integrality constraint)
Mixed Integer Program (MIP)
maximize z = cx + dysubject to Ax + Dy ≤ b, x, y ≥ 0 (linear constraints)
Solve the linear relaxation of the problem. If the solution is integer, then we are done. Otherwise create two new subproblems by branching on a fractional variable.
A subproblem is not active when any of the following occurs:
you have already used the subproblem to branch on
all variables in the solution are integer the subproblem is infeasible
you can fathom the subproblem by a bounding argument.
Choose an active subproblem and branch on a fractional variable. Repeat until there are no active subproblems.
Remarks If x is restricted to integer (but not necessarily to 0 or 1), then if x = 4.27 you would
branch with the constraints x ≤ 4 and x ≥ 5. If some variables are not restricted to integer you do not branch on them.
Michał Pióro 24
single-path allocation problem – IP formulation
variables udp binary flow realizing demand d on path p
Also, integer MIP can always be converted into binary MIPtransformation: xj = 20uj0 + 21uj1 + ... + 2qujq (xj ≤ 2q+1 -1)
Lagrangean relaxation can also be used for finding lower bounds (instead of linear relaxation).
Branch-and-Cut (B&C ) combination of B&B with the cutting plane method the most effective exact approach to NP-complete MIPs idea: add „valid inequalities” which define the facets of the integer
polyhedron the valid inequalities generation is problem dependent, and not based on
general „formulas”as for the cutting plane method (e.g., Gomory fractional cuts)
Michał Pióro 32
cutting plane method for MIPs (Gomory)
Definitions
P(A,D,b) = (x,y) ∈Rn+p: Ax + Dy ≤ b, x ≥ 0, y ≥ 0 – MIPLP domain
DM = (x,y) ∈Zn×Rp: Ax + Dy ≤ b, x ≥ 0, y ≥ 0 – MIP domain
An inequality fx + gy ≤ f0 is called a valid inequality for MIP if
fx + gy ≤ f0 for all (x,y) ∈ DM A valid inequality for MIP is a cut (or cutting plane) if
P(A,D,b) ∩ (x,y) ∈Rn+p: fx + gy ≤ f0 ⊂ P(A,D,b) where the containment is proper
Note that MIP is equivalent to LP: max cx + dy: (x,y) ∈∈∈∈ Conv(DM)
In a tutorial by Manfred Padberg: „Cutting Plane Methods for Mixed-Integer Programming”
Let P0 = P(A,D,b) = (x,y) ∈Rn+p: Ax + Dy ≤ b, x ≥ 0, y ≥ 0 and z0 = cx 0 + dy0 = max cx + dy : (x,y) ∈P0
If x0 ∈ Zn or z0 ∈ -∞,+∞ then STOP.
Otherwise, let F be a family of cuts for MIP, such that fx 0 + gy0 > f0for all (f,g,f0) ∈ F.Let P1 = P0 ∩ (x,y) ∈Rn+p: fx + gy ≤ f0 for all (f,g,f0) ∈ F ⊂ P0
and z1 = cx 1 + dy1 = max cx + dy : (x,y) ∈P1 .
Michał Pióro 35
cutting plane method - continued
Note that DM ⊆ P1 ⊂ P0 and we can iterate, generating a sequence of polyhedra
P0 ⊃ P1 ⊃ ... ⊃ Pk ⊃ Pk+1 ⊃ ... ⊇ conv(DM) ⊇ DM
such that
zk+1 = cx k+1 + dy k+1 ≤ zk = cx k + dyk
where zk = cx k + dy k = max cx + dy : (x,y) ∈Pk .
We stop when xk ∈ Zn or zk = -∞ (i.e., when Pk = ∅).
max Σi=13 xi : x1+ x2+ 2x3 + 2 ≤ 3 = 1; α4= 2 – 1 = 1
LCI: x1+ x2 + x3 + x4 ≤ 2
Step 2: assume x5 = 1
max Σi=14 xi : x1+ x2+ 2x3 + 2x4 + 3 ≤ 3 = 0;
α5 = 2 - 0 = 2LCI: x1+ x2 + x3 + x4 + 2x5 ≤ 2
Michał Pióro 43
problem-dependent generation of cuts (separators)
Let P = Conv DM = polyhedron for MIP
Let SP = set of all valid inequalities for P
= (f,g,f0) ∈ Rn+p+1 : P ⊆ (x,y) ∈Rn+p : fx + gy ≤ f0
Ideally:minimize G(f,g,f0) over all (f,g,f0) ∈ SPwhere
G(f,g,f0) = max z = cx + dy : (x,y) ∈ Pk ∩ (x,y)∈Rn+p : fx + gy ≤ f0 (we want to a separator that drives the objective function down as fast as possible)
Too difficult: In practice we try to find a most violated separator :maximize fxk + gyk - f0
over all (f,g,f0) ∈ SP such that f+ g = 1
10100 facets, two suffice (outer cone)
Michał Pióro 44
separation theorem
The polyhedral separation problem for P:
Given (x,y) ∈Rn+p either
(i) conclude that (x,y) ∈ P(ii) find a most violated separator (cut) (f,g,f0) ∈ SP
(such that f+ g = 1) for (x,y) and P.
TheoremMIP is solvable in polynomial time if and only if
the polyhedral separation problem is solvable in polynomial time.
Still, even though generating cuts is in general difficult for difficult MIPs, there is a chance to effectively generate good cuts.
Michał Pióro 45
B&B algorithims - comments
A priori addition of cuts at the root of the B&B tree – only a limited number
Branch-and-Cut (B&C ) combination of B&B with the cutting plane method the most effective exact approach to MIPs idea: add „valid inequalities” which define the facets of the integer
polyhedron generation of valid inequalities can be problem dependent, and not
necessarily based on general „formulas”as for the cutting plane method(e.g., Gomory fractional cuts)
Branch-and-Price (B&P) combination of B&B and column generation
B&C&P as well
Michał Pióro 46
IP (and MIP) is NP-hard
SATISFIABILTY PROBLEM (SAT) can be expressed as IP even as a binary program (all integer variables are binary)
Every real-life problem can be modeled(with a sufficient accuracy) as a MIP!
Michał Pióro 47
X - set of vectors x = (x1,x2,...,xn)
x ∈ X iff Ax ≤ b and x are integers
Decision problem:Instance: given n, A, b, C, and linear function f(x).
Question: is there x ∈ X such that f(x) ≤ C?
The SAT problem is directly reducible to a binary IP problem. assign binary variables xi and xi with each Boolean variables ui and ui
an inequality for each clause of the instance of SAT (x1 + x2 + x4 ≥ 1) add inequalities: xi + xi = 1, i=1,2,...,n
Integer Programming is NP-hard
Michał Pióro 48
Why LPs, MIPs, and IPs are so important?
in practice only LP guarantees efficient solutions
decomposition methods are available for LPs
all real-life problems can be expressed as MIPs!
MIPs and IPs can be solved by general solvers by the branch-
and-cut method, based on LP
CPLEX, XPRESS
sometimes very efficiently
otherwise, we have to use (frequently) unreliable stochastic