7/29/2019 Tutorial Cpa i Or
1/329
CPAIOR tutorialMay 2009 Slide 1
Tutorial: Operations Research inConstraint Programming
John HookerCarnegie Mellon University
May 2009Revised June 2009
7/29/2019 Tutorial Cpa i Or
2/329
CPAIOR tutorialMay 2009 Slide 2
Why Integrate OR and CP?
Complementary strengths
Computational advantages
Outline of the Tutorial
7/29/2019 Tutorial Cpa i Or
3/329
CPAIOR tutorialMay 2009 Slide 3
Complementary Strengths
CP: Inference methods
Modeling
Exploits local structure
OR: Relaxation methods
Duality theory
Exploits global structure
Lets bring themtogether!
7/29/2019 Tutorial Cpa i Or
4/329
CPAIOR tutorialMay 2009 Slide 4
Computational Advantage of
Integrating CP and ORUsing CP + relaxation from MILP
30 to 40 timesfaster than CP,
MILP
Productconfiguration
Thorsteinsson &Ottosson (2001)
4 to 150 timesfaster than MILP.
Flow shopscheduling, etc.
Hooker & Osorio(1999)
2 to 200 timesfaster than MILP
Piecewise linearcosts
Refalo (1999)
2 to 50 times fasterthan CP
Lessontimetabling
Focacci, Lodi,Milano (1999)
SpeedupProblem
7/29/2019 Tutorial Cpa i Or
5/329
CPAIOR tutorialMay 2009 Slide 5
Computational Advantage of
Integrating CP and MILPUsing CP + relaxation from MILP
Solved 67 of 90, CPsolved only 12
Scheduling withearliness &
tardiness costs
Beck & Refalo(2003)
Up to 600 timesfaster than MILP.
2 problems: 20 hrs for MILP
Structural design(nonlinear)
Bollapragada,Ghattas &Hooker (2001)
Better than CP in
less time
Stable set
problem
Van Hoeve
(2001)
1 to 10 times fasterthan CP, MILP
Automaticrecording
Sellmann &Fahle (2001)
SpeedupProblem
7/29/2019 Tutorial Cpa i Or
6/329
CPAIOR tutorialMay 2009 Slide 6
Computational Advantage of
Integrating CP and MILPUsing CP-based Branch and Price
First to solve8-team instance
Travelingtournamentscheduling
Easton,Nemhauser &Trick (2002)
Optimal schedulefor 210 trips, vs.
120 for traditionalbranch and price
Urban transitcrew scheduling
Yunes, Moura &de Souza (1999)
SpeedupProblem
7/29/2019 Tutorial Cpa i Or
7/329
CPAIOR tutorialMay 2009 Slide 7
Computational Advantage of
Integrating CP and MILPUsing CP/MILP Benders methods
Solved previouslyinsoluble problemin 10 min
Polypropylenebatch schedulingat BASF
Timpe (2002)
10 times fasterthan Jain &Grossmann
Min-cost planning& scheduling
Thorsteinsson(2001)
20 to 1000 timesfaster than CP,
MILP
Min-cost planning& scheduing
Jain &Grossmann(2001)
SpeedupProblem
7/29/2019 Tutorial Cpa i Or
8/329
CPAIOR tutorialMay 2009 Slide 8
Computational Advantage of
Integrating CP and MILPUsing CP/MILP Benders methods
10-1000 timesfaster than CP,
MILP
Min tardinessplanning & cumulative
scheduling
Hooker (2005)
100-1000 timesfaster than CP,
MILP
Min-cost,min-makespan
planning & cumulativescheduling
Hooker (2004)
Solved twice asmany instances
as traditional
Benders
Call center schedulingBenoist, Gaudin,Rottembourg(2002)
SpeedupProblem
7/29/2019 Tutorial Cpa i Or
9/329
CPAIOR tutorialMay 2009 Slide 9
Software for Integrated Methods
ECLiPSe
Exchanges information between ECLiPSEe solver, Xpress-MP OPL Studio
Combines CPLEX and ILOG CP Optimizer with script language
Mosel
Combines Xpress-MP, Xpress-Kalis with low-level modeling
BARON
Global optimization with relaxation + domain reduction
SIMPL
Full integration with high-level modeling (prototype)
SCIP
Combines MILP and CP-based propagation
7/29/2019 Tutorial Cpa i Or
10/329
CPAIOR tutorialMay 2009 Slide 10
Outline of the Tutorial
Why Integrate OR and CP?
A Glimpse at CP
Initial Example: Integrated Methods
CP Concepts
CP Filtering Algorithms
Linear Relaxation and CP
Mixed Integer/Linear Modeling
Cutting Planes
Lagrangean Relaxation and CP
Dynamic Programming in CP CP-based Branch and Price
CP-based Benders Decomposition
7/29/2019 Tutorial Cpa i Or
11/329
CPAIOR tutorialMay 2009 Slide 11
Detailed Outline
Why Integrate OR and CP? Complementary strengths
Computational advantages Outline of the tutorial A Glimpse at CP
Early successes Advantages and disadvantages
Initial Example: Integrated Methods Freight Transfer Bounds Propagation Cutting Planes
Branch-infer-and-relax Tree
7/29/2019 Tutorial Cpa i Or
12/329
CPAIOR tutorialMay 2009 Slide 12
Detailed Outline
CP Concepts Consistency
Hyperarc Consistency Modeling Examples CP Filtering Algorithms
Element Alldiff
Disjunctive Scheduling Cumulative Scheduling
Linear Relaxation and CP Why relax?
Algebraic Analysis of LP Linear Programming Duality LP-Based Domain Filtering Example: Single-Vehicle Routing Disjunctions of Linear Systems
7/29/2019 Tutorial Cpa i Or
13/329
CPAIOR tutorialMay 2009 Slide 13
Detailed Outline Mixed Integer/Linear Modeling
MILP Representability 4.2 Disjunctive Modeling
4.3 Knapsack Modeling Cutting Planes
0-1 Knapsack Cuts Gomory Cuts
Mixed Integer Rounding Cuts Example: Product Configuration
Lagrangean Relaxation and CP Lagrangean Duality Properties of the Lagrangean Dual
Example: Fast Linear Programming Domain Filtering Example: Continuous Global Optimization
7/29/2019 Tutorial Cpa i Or
14/329
CPAIOR tutorialMay 2009 Slide 14
Detailed Outline Dynamic Programming in CP
Example: Capital Budgeting Domain Filtering
Recursive Optimization CP-based Branch and Price
Basic Idea Example: Airline Crew Scheduling
CP-based Benders Decomposition Benders Decomposition in the Abstract Classical Benders Decomposition Example: Machine Scheduling
7/29/2019 Tutorial Cpa i Or
15/329
CPAIOR tutorialMay 2009 Slide 15
Background Reading
This tutorial is based on:
J. N. Hooker, Integrated Methods for Optimization, Springer(2007). Contains 295 exercises.
J. N. Hooker, Operations research methods in constraint
programming, in F. Rossi, P. van Beek and T. Walsh, eds.,Handbook of Constraint Programming, Elsevier (2006), pp.
527-570.
7/29/2019 Tutorial Cpa i Or
16/329
CPAIOR tutorialMay 2009 Slide 16
A Glimpse at Constraint Programming
Early Successes
Advantages and Disadvantages
7/29/2019 Tutorial Cpa i Or
17/329
CPAIOR tutorialMay 2009 Slide 17
What is constraint programming?
It is a relatively new technology developed in the computer
science and artificial intelligence communities.
It has found an important role in scheduling, logistics and supplychain management.
7/29/2019 Tutorial Cpa i Or
18/329
CPAIOR tutorialMay 2009 Slide 18
Container port scheduling(Hong Kong and Singapore)
Circuit design (Siemens)
Real-time control(Siemens, Xerox)
Early commercial successes
7/29/2019 Tutorial Cpa i Or
19/329
CPAIOR tutorialMay 2009 Slide 19
Applications
Job shop scheduling
Assembly line smoothing
and balancing
Cellular frequencyassignment
Nurse scheduling
Shift planning
Maintenance planning
Airline crew rostering and scheduling
Airport gate allocation and stand planning
7/29/2019 Tutorial Cpa i Or
20/329
CPAIOR tutorialMay 2009 Slide 20
Production scheduling
chemicals
aviationoil refiningsteellumber
photographic platestires
Transport scheduling (food,nuclear fuel)
Warehouse management
Course timetabling
Applications
7/29/2019 Tutorial Cpa i Or
21/329
CPAIOR tutorialMay 2009 Slide 21
Advantages and Disadvantages
CP vs. Mathematical Programming
Constraint-basedprocessing
Independence of modeland algorithm
BranchingBranching
High-level modeling(global constraints)
Atomistic modeling(linear inequalities)
Inference (filtering,constraint propagation)
Relaxation
Logic processingNumerical calculation
CPMP
7/29/2019 Tutorial Cpa i Or
22/329
CPAIOR tutorialMay 2009 Slide 22
Programming programming
In constraint programming:
programming= a form of computer programming(constraint-based processing)
In mathematical programming:
programming= logistics planning (historically)
7/29/2019 Tutorial Cpa i Or
23/329
CPAIOR tutorialMay 2009 Slide 23
CP vs. MP
In mathematical programming, equations
(constraints) describe the problem but dont tell how tosolve it.
In constraint programming, each constraint invokes a
procedure that screens out unacceptable solutions.
Much as each line of a computer program invokesan operation.
7/29/2019 Tutorial Cpa i Or
24/329
CPAIOR tutorialMay 2009 Slide 24
Advantages of CP
Better at sequencing and scheduling
where MP methods have weak relaxations.
Adding messy constraints makes the problem easier.
The more constraints, the better.
More powerful modeling language.
Global constraints lead to succinct models.
Constraints convey problem structure to the solver.
Better at highly-constrained problems
Misleading better when constraints propagate well, orwhen constraints have few variables.
7/29/2019 Tutorial Cpa i Or
25/329
CPAIOR tutorialMay 2009 Slide 25
Disdvantages of CP
Weaker for continuous variables.
Due to lack of numerical techniques May fail when constraints contain many variables.
These constraints dont propagate well.
Often not good for finding optimal solutions. Due to lack of relaxation technology.
May not scale up
Discrete combinatorial methods Software is not robust
Younger field
7/29/2019 Tutorial Cpa i Or
26/329
CPAIOR tutorialMay 2009 Slide 26
Obvious solution
Integrate CP and MP.
7/29/2019 Tutorial Cpa i Or
27/329
CPAIOR tutorialMay 2009 Slide 27
Trends
CP is better known in continental Europe, Asia.
Less known in North America, seen as threat to OR. CP/MP integration is growing
Eclipse, Mozart, OPL Studio, SIMPL, SCIP, BARON
Heuristic methods increasingly important in CP Discrete combinatorial methods
MP/CP/heuristics may become a single technology.
7/29/2019 Tutorial Cpa i Or
28/329
CPAIOR tutorialMay 2009 Slide 28
Initial Example: Integrated Methods
Freight Transfer
Bounds Propagation
Cutting Planes
Branch-infer-and-relax Tree
7/29/2019 Tutorial Cpa i Or
29/329
CPAIOR tutorialMay 2009 Slide 29
Example: Freight Transfer
Transport 42 tons of freight using 8 trucks, which come in
4 sizes
40334
50433
6053290731
Costper
truck
Capacity
(tons)
Numberavailable
Trucksize
7/29/2019 Tutorial Cpa i Or
30/329
CPAIOR tutorialMay 2009 Slide 30
40334
50433
6053290731
Costper
truck
Capacity
(tons)
Numberavailable
Trucktype
+ + +
+ + +
+ + +
1 2 3 4
1 2 3 4
1 2 3 4
min 90 60 50 40
7 5 4 3 42
8
{0,1,2,3}i
x x x x
x x x x
x x x x
x
Number of trucks of type 1
Knapsackcoveringconstraint
Knapsackpackingconstraint
7/29/2019 Tutorial Cpa i Or
31/329
CPAIOR tutorialMay 2009 Slide 31
+ + +
+ + +
+ + +
1 2 3 4
1 2 3 4
1 2 3 4
min 90 60 50 40
7 5 4 3 42
8
{0,1,2,3}i
x x x x
x x x x
x x x x
x
Bounds propagation
=
1
42 5 3 4 3 3 31
7x
7/29/2019 Tutorial Cpa i Or
32/329
CPAIOR tutorialMay 2009 Slide 32
+ + +
+ + +
+ + +
1 2 3 4
1 2 3 4
1 2 3 4
1 2 3 4
min 90 60 50 40
7 5 4 3 42
8
{1,2,3}, , , {0,1,2,3}
x x x x
x x x x
x x x x
x x x x
Bounds propagation
=
1
42 5 3 4 3 3 31
7x
Reduced
domain
7/29/2019 Tutorial Cpa i Or
33/329
CPAIOR tutorialMay 2009 Slide 33
Let {Lj, , Uj} be the domain of xj
A constraint set is bounds consistent if for eachj: xj= Lj in some feasible solution and
xj= Uj in some feasible solution.
Bounds consistency we will not set xj to any infeasiblevalues during branching.
Bounds propagation achieves bounds consistency for asingle inequality.
7x1 + 5x2 + 4x3 + 3x4 42 is bounds consistent when thedomains are x1 {1,2,3} and x2, x3, x4 {0,1,2,3}.
But not necessarily for a set of inequalities.
Bounds consistency
7/29/2019 Tutorial Cpa i Or
34/329
CPAIOR tutorialMay 2009 Slide 34
Bounds propagation may not achieve bounds consistency
for a set of constraints.
Consider set of inequalities
with domains x1, x2 {0,1}, solutions (x1,x2) = (1,0), (1,1).
Bounds propagation has no effect on the domains.
But constraint set is not bounds consistent because x1 = 0
in no feasible solution.
Bounds consistency
1 2
1 2
1
0
x x
x x
+
7/29/2019 Tutorial Cpa i Or
35/329
CPAIOR tutorialMay 2009 Slide 35
+ + +
+ + +
+ + +
1 2 3 4
1 2 3 4
1 2 3 4
1
min 90 60 50 40
7 5 4 3 42
8
0 3, 1i
x x x x
x x x x
x x x x
x x
Cutting PlanesBegin with continuous relaxation
Replace domainswith bounds
This is a linear programming problem, which is easy tosolve.
Its optimal value provides a lower bound on optimal
value of original problem.
7/29/2019 Tutorial Cpa i Or
36/329
CPAIOR tutorialMay 2009 Slide 36
+ + +
+ + +
+ + +
1 2 3 4
1 2 3 4
1 2 3 4
1
min 90 60 50 40
7 5 4 3 42
8
0 3, 1i
x x x x
x x x x
x x x x
x x
Cutting planes (valid inequalities)
We can create a tighter relaxation (larger minimumvalue) with the addition of cutting planes.
7/29/2019 Tutorial Cpa i Or
37/329
CPAIOR tutorialMay 2009 Slide 37
+ + +
+ + +
+ + +
1 2 3 4
1 2 3 4
1 2 3 4
1
min 90 60 50 40
7 5 4 3 42
8
0 3, 1i
x x x x
x x x x
x x x x
x x
Cutting planes (valid inequalities)
All feasible solutions of theoriginal problem satisfy acutting plane (i.e., it is valid).
But a cutting plane mayexclude (cut off) solutions ofthe continuous relaxation.
Cuttingplane
Feasible solutions
Continuousrelaxation
7/29/2019 Tutorial Cpa i Or
38/329
CPAIOR tutorialMay 2009 Slide 38
+ + +
+ + +
+ + +
1 2 3 4
1 2 3 4
1 2 3 4
1
min 90 60 50 40
7 5 4 3 42
8
0 3, 1i
x x x x
x x x x
x x x x
x x
Cutting planes (valid inequalities)
{1,2} is a packingbecause 7x1 + 5x2 alone cannot satisfy the inequality,even with x1 = x2 = 3.
7/29/2019 Tutorial Cpa i Or
39/329
CPAIOR tutorialMay 2009 Slide 39
+ + +
+ + +
+ + +
1 2 3 4
1 2 3 4
1 2 3 4
1
min 90 60 50 40
7 5 4 3 42
8
0 3, 1i
x x x x
x x x x
x x x x
x x
Cutting planes (valid inequalities)
{1,2} is a packing
{ }
+ + =
3 4
42 (7 3 5 3)2
max 4,3x x
So, + + 3 44 3 42 (7 3 5 3)x x
which implies
Knapsack cut
7/29/2019 Tutorial Cpa i Or
40/329
CPAIOR tutorialMay 2009 Slide 40
Cutting planes (valid inequalities)
Let xihave domain [Li,Ui] and let a 0.
In general, a packing Pfor ax a0 satisfies
0i i i i i P i P
a x a a U
and generates a knapsack cut
{ }
0
max
i ii P
ii P i
i P
a a U
xa
7/29/2019 Tutorial Cpa i Or
41/329
CPAIOR tutorial
May 2009 Slide 41
+ + +
+ + +
+ + +
1 2 3 4
1 2 3 4
1 2 3 4
1
min 90 60 50 40
7 5 4 3 42
8
0 3, 1i
x x x x
x x x x
x x x x
x x
Cutting planes (valid inequalities)
x2 + x3 3{1,4}
x2 + x4 2{1,3}
x3 + x4 2{1,2}
Knapsack cutsMaximal Packings
Knapsack cuts corresponding to nonmaximal
packings can be nonredundant.
7/29/2019 Tutorial Cpa i Or
42/329
CPAIOR tutorial
May 2009 Slide 42
+ + +
+ + +
+
+
+
+
+ +
1 2 3 4
1 2 3 4
1
3 4
2 3
2
1
2 3
4
4
min 90 60 50 40
7 5 4 3 42
8
0 3, 1
2
2
3
i
x x x x
x x x x
x x x
x x
x x
x x
x
x x
Continuous relaxation with cuts
Optimal value of 523.3 is a lower bound on optimal valueof original problem.
Knapsack cuts
7/29/2019 Tutorial Cpa i Or
43/329
CPAIOR tutorial
May 2009 Slide 43
Branch-
infer-and-relax tree
Propagate bounds
and solverelaxation oforiginal problem.
x1 { 123}x2 {0123}
x3 {0123}x4 {0123}
x =(2,3,2,0)
value = 523
7/29/2019 Tutorial Cpa i Or
44/329
CPAIOR tutorial
May 2009 Slide 44
Branch on a
variable withnonintegral valuein the relaxation.
x1 { 123}x2 {0123}
x3 {0123}x4 {0123}
x =(2,3,2,0)
value = 523
x1 {1,2}x1 = 3
Branch-infer-and-relax tree
7/29/2019 Tutorial Cpa i Or
45/329
CPAIOR tutorial
May 2009 Slide 45
Propagate bounds
and solverelaxation.
Since relaxationis infeasible,
backtrack.
x1 { 123}x2 {0123}
x3 {0123}x4 {0123}
x =(2,3,2,0)
value = 523
x1 { 12 }x2 { 23}
x3 { 123}x4 { 123}infeasible
relaxation
x1 {1,2}x1 = 3
Branch-infer-and-relax tree
7/29/2019 Tutorial Cpa i Or
46/329
CPAIOR tutorial
May 2009 Slide 46
Propagate bounds
and solverelaxation.
Branch onnonintegral
variable.
x1 { 123}x2 {0123}
x3 {0123}x4 {0123}
x =(2,3,2,0)
value = 523
x1 { 12 }x2 { 23}
x3 { 123}x4 { 123}infeasible
relaxation
x1 { 3}x2 {0123}
x3 {0123}x4 {0123}x =(3,2.6,2,0)
value = 526
x1 {1,2}x1 = 3
x2 {0,1,2}
x2 = 3
Branch-infer-and-relax tree
7/29/2019 Tutorial Cpa i Or
47/329
CPAIOR tutorial
May 2009 Slide 47
Branch again.
x1 { 123}x2 {0123}
x3 {0123}x4 {0123}
x =(2,3,2,0)
value = 523
x1 { 12 }x2 { 23}
x3 { 123}x4 { 123}infeasible
relaxation
x1 { 3}x2 {0123}
x3 {0123}x4 {0123}x =(3,2.6,2,0)
value = 526
x1 { 3}x
2 {012 }
x3 { 123}x4 {0123}
x =(3,2,2,0)value = 527
x1 {1,2}x1 = 3
x2 {0,1,2}
x2 = 3
x3 {1,2}
x3 = 3
Branch-infer-and-relax tree
7/29/2019 Tutorial Cpa i Or
48/329
CPAIOR tutorial
May 2009 Slide 48
Solution of
relaxationis integral andtherefore feasiblein the originalproblem.
This becomes theincumbentsolution.
x1 { 123}x2 {0123}
x3 {0123}x4 {0123}
x =(2,3,2,0)
value = 523
x1 { 12 }x2 { 23}
x3 { 123}x4 { 123}infeasible
relaxation
x1 { 3}x2 {0123}
x3 {0123}x4 {0123}x =(3,2.6,2,0)
value = 526
x1 { 3}x
2 {012 }
x3 { 123}x4 {0123}
x =(3,2,2,0)value = 527
x1 { 3}x2 { 12 }x3 { 12 }x4 { 123}x =(3,2,2,1)
value = 530
feasible solution
x1 {1,2}x1 = 3
x2 {0,1,2}
x2 = 3
x3 {1,2}
x3 = 3
Branch-infer-and-relax tree
7/29/2019 Tutorial Cpa i Or
49/329
CPAIOR tutorial
May 2009 Slide 49
Solution is
nonintegral, butwe can backtrackbecause value ofrelaxation isno better thanincumbent solution.
x1 { 123}x2 {0123}
x3 {0123}x4 {0123}
x =(2,3,2,0)
value = 523
x1 { 12 }x2 { 23}
x3 { 123}x4 { 123}infeasible
relaxation
x1 { 3}x2 {0123}
x3 {0123}x4 {0123}x =(3,2.6,2,0)
value = 526
x1 { 3}x
2 {012 }
x3 { 123}x4 {0123}
x =(3,2,2,0)value = 527
x1
{ 3}x2 { 12 }x3 { 12 }x4 { 123}x =(3,2,2,1)
value = 530
feasible solution
x1 { 3}
x2 {012 }x3 { 3}x4 {012 }
x =(3,1,3,)value = 530
backtrack
due to bound
x1 {1,2}x1 = 3
x2 {0,1,2}
x2 = 3
x3
{1,2}x3 = 3
Branch-infer-and-relax tree
7/29/2019 Tutorial Cpa i Or
50/329
CPAIOR tutorial
May 2009 Slide 50
Another feasible
solution found.
No better thanincumbent solution,which is optimal
because searchhas finished.
x1 { 123}x2 {0123}
x3 {0123}x4 {0123}
x =(2,3,2,0)
value = 523
x1 { 12 }x2 { 23}
x3 { 123}x4 { 123}infeasible
relaxation
x1 { 3}x2 {0123}
x3 {0123}x4 {0123}x =(3,2.6,2,0)
value = 526
x1 { 3}x
2 {012 }
x3 { 123}x4 {0123}
x =(3,2,2,0)value = 527
x1 { 3}x2 { 3}
x3 {012 }x4 {012 }x =(3,3,0,2)
value = 530feasible solution
x1
{ 3}x2 { 12 }x3 { 12 }x4 { 123}x =(3,2,2,1)
value = 530
feasible solution
x1 { 3}
x2 {012 }x3 { 3}x4 {012 }
x =(3,1,3,)value = 530
backtrack
due to bound
x1 {1,2}x1 = 3
x2 {0,1,2}
x2 = 3
x3
{1,2}x3 = 3
Branch-infer-and-relax tree
7/29/2019 Tutorial Cpa i Or
51/329
CPAIOR tutorial
May 2009 Slide 51
Two optimal solutions
= (3,2,2,1)x
= (3,3,0,2)x
7/29/2019 Tutorial Cpa i Or
52/329
CPAIOR tutorial
May 2009 Slide 52
Constraint Programming Concepts
Consistency
Generalized Arc Consistency
Modeling Examples
7/29/2019 Tutorial Cpa i Or
53/329
CPAIOR tutorial
May 2009 Slide 53
Consistency A constraint set is consistent if every partial assignment to thevariables that violates no constraint is feasible.
i.e., can be extended to a feasible solution.
Consistency feasibility
Consistency means that any infeasible partial assignment isexplicitly ruled out by a constraint.
Fully consistent constraint sets can be solved withoutbacktracking.
7/29/2019 Tutorial Cpa i Or
54/329
CPAIOR tutorial
May 2009 Slide 54
Consistency
Consider the constraint set
It is not consistent, because x1 = 0 violates no constraintand yet is infeasible (no solution has x1 = 0).
Adding the constraint x1 = 1 makes the set consistent.
{ }
1 100
1 100
10
0,1j
x xx x
x
+
1x x+
7/29/2019 Tutorial Cpa i Or
55/329
CPAIOR tutorial
May 2009 Slide 55
subtree with 299 nodesbut no feasible solution
By adding the constraint
x1 = 1, the left subtree iseliminated
{ }
1 100
1 100
1
1
other constraints
0,1j
x x
x x
x
+
1
0x =1
1x =
7/29/2019 Tutorial Cpa i Or
56/329
CPAIOR tutorial
May 2009 Slide 56
Generalized Arc Consistency (GAC)
Also known as hyperarc consistency.
A constraint set is GAC if every value in every variabledomain is part of some feasible solution.
That is, the domains are reduced as much as
possible. If all constraints are binary (contain 2 variables),GAC = arc consistency.
Domain reduction is CPs biggest engine.
G h l i bl th t b l d b
7/29/2019 Tutorial Cpa i Or
57/329
CPAIOR tutorial
May 2009 Slide 57
Graph coloring problem that can be solved by arc
consistency maintenance alone. Color nodes with red,green, blue with no two adjacent nodes having the samecolor.
Graph coloring problem that can be solved by arc
7/29/2019 Tutorial Cpa i Or
58/329
CPAIOR tutorial
May 2009 Slide 58
Graph coloring problem that can be solved by arc
consistency maintenance alone. Color nodes with red,green, blue with no two adjacent nodes having the samecolor.
Graph coloring problem that can be solved by arc
7/29/2019 Tutorial Cpa i Or
59/329
CPAIOR tutorial
May 2009 Slide 59
Graph coloring problem that can be solved by arc
consistency maintenance alone. Color nodes with red,green, blue with no two adjacent nodes having the samecolor.
Graph coloring problem that can be solved by arc
7/29/2019 Tutorial Cpa i Or
60/329
CPAIOR tutorial
May 2009 Slide 60
Graph coloring problem that can be solved by arc
consistency maintenance alone. Color nodes with red,green, blue with no two adjacent nodes having the samecolor.
Graph coloring problem that can be solved by arc
7/29/2019 Tutorial Cpa i Or
61/329
CPAIOR tutorial
May 2009 Slide 61
Graph coloring problem that can be solved by arc
consistency maintenance alone. Color nodes with red,green, blue with no two adjacent nodes having the samecolor.
Graph coloring problem that can be solved by arc
7/29/2019 Tutorial Cpa i Or
62/329
CPAIOR tutorial
May 2009 Slide 62
Graph coloring problem that can be solved by arc
consistency maintenance alone. Color nodes with red,green, blue with no two adjacent nodes having the samecolor.
Graph coloring problem that can be solved by arc
7/29/2019 Tutorial Cpa i Or
63/329
CPAIOR tutorial
May 2009 Slide 63
Graph coloring problem that can be solved by arc
consistency maintenance alone. Color nodes with red,green, blue with no two adjacent nodes having the samecolor.
7/29/2019 Tutorial Cpa i Or
64/329
CPAIOR tutorial
May 2009 Slide 64
Modeling Examples with Global ConstraintsTraveling Salesman
Traveling salesman problem:
Let cij= distance from city ito cityj.
Find the shortest route that visits each of ncities exactlyonce.
P l 0 1 d l
7/29/2019 Tutorial Cpa i Or
65/329
CPAIOR tutorial
May 2009 Slide 65
Popular 0-1 model
Let xij= 1 if city iimmediately precedes cityj, 0 otherwise
{ }
{ }
min
s.t. 1, all
1, all
1, all disjoint , 1, ,
0,1
ij ij ij
iji
ijj
iji V j W
ij
c x
x j
x i
x V W n
x
=
=
Subtour elimination constraints
A CP d l
7/29/2019 Tutorial Cpa i Or
66/329
CPAIOR tutorial
May 2009 Slide 66
A CP model
Let yk= the kth city visited.
The model would be written in a specific constraint programminglanguage but would essentially say:
Variable indices
Global constraint{ }
1
1
min
s.t. alldiff( , , )
1, ,
k ky yk
n
k
c
y y
y n
+
A lt t CP d l
7/29/2019 Tutorial Cpa i Or
67/329
CPAIOR tutorial
May 2009 Slide 67
{ }1
min
s.t. circuit( , , )
1, ,
kkyk
n
k
c
y y
y n
An alternate CP model
Let yk= the city visited after city k.
Hamiltonian circuitconstraint
Element constraint
7/29/2019 Tutorial Cpa i Or
68/329
CPAIOR tutorial
May 2009 Slide 68
The constraint xy 5 can be implemented
Element constraint
The constraint cy 5 can be implemented:
Assign zthe yth
value in the list
(this is a slightly different constraint)
Add theconstraintz= x
y
( )1
5element ,( , , ),n
zy c c z
( )1
5
element ,( , , ),n
z
y x x z
7/29/2019 Tutorial Cpa i Or
69/329
CPAIOR tutorial
May 2009 Slide 69
Day: 1 2 3 4 5 6 7 8
A B A
Product
At most one product manufactured on each day.
Demands for each product on each day.
Minimize setup + holding cost.
Modeling example: Lot sizing and scheduling
min h s q
+ Many variables
7/29/2019 Tutorial Cpa i Or
70/329
CPAIOR tutorial
May 2009 Slide 70
,
, 1
, 1
, 1
, 1
, 1
min
s.t. , all ,
, all ,
, all ,1 , all ,
1, all , ,
, all , ,
, all , ,
it it ij ijt t i j t
i t it it it
it it i t
it it
it i t
ijt i t jt
ijt i t
ijt jt
i
h s q
s x d s i t
z y y i t
z y i t z y i t
y y i j t
y i j t
y i j t
x
+
+ = +
+
, all ,
1, all
, , {0,1}
, 0
t it
it
i
it it ijt
it it
Cy i t
y t
y z
x s
=
Integerprogrammingmodel
(Wolsey)
Many variables
7/29/2019 Tutorial Cpa i Or
71/329
CPAIOR tutorial
May 2009 Slide 71
( ) ( )
1
, 1
min
s.t. , all ,
0 , 0, all ,
0 , all ,
t ty y i it t i
i t it it it
it it
t it
q h s
s x d s i t
x C s i t
y i x i t
+
+ = +
=
CP model
Minimize holding and setup costs
Inventory balance
Production capacity
7/29/2019 Tutorial Cpa i Or
72/329
CPAIOR tutorial
May 2009 Slide 72
( ) ( )
1
, 1
min
s.t. , all ,
0 , 0, all ,
0 , all ,
t ty y i it t i
i t it it it
it it
t it
q h s
s x d s i t
x C s i t
y i x i t
+
+ = +
=
CP model
Minimize holding and setup costs
Variable indices
Product manufactured in period t
Production level of product i in period t
Inventory balance
Production capacity
7/29/2019 Tutorial Cpa i Or
73/329
CPAIOR tutorial
May 2009 Slide 73
Used for resource-constrained scheduling.
Total resources consumed by jobs at any one time mustnot exceed L.
Job start times(variables)
Job processing times
Job resourcerequirements
Cumulative scheduling constraint
( )1 1 1cumulative ( , , ),( , , ),( , , ),n n nt t p p c c L
Cumulative scheduling constraint
7/29/2019 Tutorial Cpa i Or
74/329
CPAIOR tutorial
May 2009 Slide 74
( )1 5
1
5
min
s.t. cumulative ( , , ),(3,3,3,5,5),(3,3,3,2,2),7
3
2
z
t t
z t
z t
+
+
Minimize makespan (no deadlines, all release times = 0):
Min makespan = 8
L
1
2 3
4
5
time
resources
Job start times
Processing times
Resources used
L
Cumulative scheduling constraint
7/29/2019 Tutorial Cpa i Or
75/329
CPAIOR tutorial
May 2009 Slide 75
Will use ILOGs OPL Studio modeling language.
Example is from OPL manual.
The problem
Load 34 items on the ship in minimum time (min makespan)
Each item requires a certain time and certain number ofworkers.
Total of 8 workers available.
Modeling example: Ship loading
LaborDuraItem LaborDuraItem
7/29/2019 Tutorial Cpa i Or
76/329
CPAIOR tutorial
May 2009 Slide 76
6217
3316
3215
3514
4113
5212
4311
8210
439
348
437
526
555
464
343
442
431
LaborDura-
tion
Item
3234
3233
3132
3231
3330
8129
6228
3127
3126
8225
8524
7423
4222
4121
4120
4119
7218
LaborDura-
tion
Item
Problem data
Precedence constraints
7/29/2019 Tutorial Cpa i Or
77/329
CPAIOR tutorial
May 2009 Slide 77
1 2,42 33 5,74 55 66 87 8
8 99 109 1410 1110 12
11 1312 13
13 15,1614 1515 1816 1717 18
18 1918 20,2119 2320 23
21 22
22 2323 24
24 2525 26,30,31,3226 2727 2828 29
30 2831 2832 3333 34
Use the cumulative scheduling constraint.
7/29/2019 Tutorial Cpa i Or
78/329
CPAIOR tutorial
May 2009 Slide 78
g
( )
1 2
1 34
2 1 4 1
min
s.t. 3, 4, etc.
cumulative ( , , ),(3,4, ,2),(4,4, ,3),83, 3, etc.
z
z t z t
t tt t t t
+ +
+ +
OPL model
7/29/2019 Tutorial Cpa i Or
79/329
CPAIOR tutorial
May 2009 Slide 79
int capacity = 8;
int nbTasks = 34;
range Tasks 1..nbTasks;
int duration[Tasks] = [3,4,4,6,,2];int totalDuration =
sum(t in Tasks) duration[t];
int demand[Tasks] = [4,4,3,4,,3];
struct Precedences {int before;
int after;
}
{Precedences} setOfPrecedences = {, , , };
7/29/2019 Tutorial Cpa i Or
80/329
CPAIOR tutorial
May 2009 Slide 80
scheduleHorizon = totalDuration;
Activity a[t in Tasks](duration[t]);
DiscreteResource res(8);
Activity makespan(0);minimize
makespan.end
subject to
forall(t in Tasks)
a[t] precedes makespan;
forall(p in setOfPrecedences)
a[p.before] precedes a[p.after];
forall(t in Tasks)
a[t] requires(demand[t]) res;};
Modeling example: Production scheduling with
7/29/2019 Tutorial Cpa i Or
81/329
CPAIOR tutorial
May 2009 Slide 81
CapacityC1
CapacityC2
CapacityC3
ManufacturingUnit
StorageTanks
PackingUnits
intermediate storage
Filling of storage tank
7/29/2019 Tutorial Cpa i Or
82/329
CPAIOR tutorial
May 2009 Slide 82
Level
t u t + (b/r) u + (b/s)
Filling starts
Packing starts Filling endsPacking ends
Batch size
Manufac-turing rate Packing rate
Need to enforcecapacity constrainthere only
min TMakespan
7/29/2019 Tutorial Cpa i Or
83/329
CPAIOR tutorial
May 2009 Slide 83
( )
1
1
s.t. , all
, all
cumulative , , ,
, all
1 , all
cumulative , , , , ,
0
j
j
j
j j
ii i i
i
ii i i i
i
n
n
j j
bT u j
s
t R j
t v e mb
v u t i s
sb s u C i r
b bu e p
s su t
+
= +
+
Job release time
mstorage tanks
Job duration
Tank capacity
ppacking units
e= (1,,1)
7/29/2019 Tutorial Cpa i Or
84/329
CPAIOR tutorial
May 2009 Slide 84
Modeling example: Employee scheduling
Schedule four nurses in 8-hour shifts.
A nurse works at most one shift a day, at least 5 days a week. Same schedule every week.
No shift staffed by more than two different nurses in a week.
A nurse cannot work different shifts on two consecutive days.
A nurse who works shift 2 or 3 must do so at least two days ina row.
Two ways to view the problem
7/29/2019 Tutorial Cpa i Or
85/329
CPAIOR tutorial
May 2009 Slide 85
DCCDDDDShift 3
BBBBCCCShift 2
AAAAABAShift 1
SatFriThuWedTueMonSun
Assign nurses to shifts
3003333Nurse D
0330222Nurse C2222010Nurse B
1111101Nurse A
SatFriThuWedTueMonSun
Assign shifts to nurses
0 = day off
7/29/2019 Tutorial Cpa i Or
86/329
CPAIOR tutorial
May 2009 Slide 86
Use both formulations in the same model!First, assign nurses to shifts.
Let wsd= nurse assigned to shift son day d
1 2 3alldiff( , , ), alld d dw w w dThe variables w1d, w2d,w3d take different values
That is, schedule 3
different nurses on eachday
U b h f l i i h d l!
7/29/2019 Tutorial Cpa i Or
87/329
CPAIOR tutorial
May 2009 Slide 87
( )1 2 3alldiff( , , ), all
cardinality | ( , , , ),(5,5,5,5),(6,6,6,6)
d d dw w w
w A B C
d
D
A occurs at least 5 and at most 6times in the array w, and similarlyfor B, C, D.
That is, each nurse works at least5 and at most 6 days a week
Use both formulations in the same model!First, assign nurses to shifts.
Let wsd= nurse assigned to shift son day d
U b th f l ti i th d l!
7/29/2019 Tutorial Cpa i Or
88/329
CPAIOR tutorial
May 2009 Slide 88
( )
( )
( )
1 2 3
,Sun ,Sat
alldiff , , , all
cardinality | ( , , , ),(5,5,5,5),(6,6,6,6)
nvalues ,..., |1,2 , all
d d d
s s
w w w
w
d
A B C D
w w s
The variables ws,Sun, , ws,Sat takeat least 1 and at most 2 different
values.
That is, at least 1 and at most 2nurses work any given shift.
Use both formulations in the same model!First, assign nurses to shifts.
Let wsd= nurse assigned to shift son day d
R i i t i t t il d i thi
7/29/2019 Tutorial Cpa i Or
89/329
CPAIOR tutorial
May 2009 Slide 89
Remaining constraints are not easily expressed in thisnotation.
So, assign shifts to nurses.
Let yid= shift assigned to nurse ion day d
( )1 2 3,alldiff , all,d d dy y y d
Assign a different nurse to eachshift on each day.
This constraint is redundant ofprevious constraints, butredundant constraints speedsolution.
Remaining constraints are not easily expressed in this
7/29/2019 Tutorial Cpa i Or
90/329
CPAIOR tutorial
May 2009 Slide 90
( )
( )
1 2 3
,Sun ,Sat
alldiff , all
stretch , , | (2,3),(2,2),(6,6), , all
, ,d d d
i i
y
P i
y
y y
dy
Every stretch of 2s has length between 2 and 6.Every stretch of 3s has length between 2 and 6.
So a nurse who works shift 2 or 3 must do so at leasttwo days in a row.
Remaining constraints are not easily expressed in thisnotation.
So, assign shifts to nurses.
Let yid= shift assigned to nurse ion day d
Remaining constraints are not easily expressed in thisi
7/29/2019 Tutorial Cpa i Or
91/329
CPAIOR tutorial
May 2009 Slide 91
( )
( )
1 2 3
,Sun ,Sat
alldiff , all
stretch , , | (2,3),(2,2),(6,6), , all
, ,d d d
i i
y
P i
y
y y
dy
Here P= {(s,0),(0,s) | s= 1,2,3}
Whenever a stretch of as immediately precedes a stretch of bs,(a,b) must be one of the pairs in P.
So a nurse cannot switch shifts without taking at least one day off.
Remaining constraints are not easily expressed in thisnotation.
So, assign shifts to nurses.
Let yid= shift assigned to nurse ion day d
Now we must connect the wsd variables to the yid variables
7/29/2019 Tutorial Cpa i Or
92/329
CPAIOR tutorial
May 2009 Slide 92
Now we must connect the wsdvariables to the yidvariables.Use channeling constraints:
, all ,
, all ,
i
d
d
s
d
w
y
dy
i iw d
s s d
=
=
Channeling constraints increase propagation and make theproblem easier to solve.
The complete model is:
7/29/2019 Tutorial Cpa i Or
93/329
CPAIOR tutorial
May 2009 Slide 93
The complete model is:
, all ,
, all ,
i
d
d
s
d
w
y
dy
i iw d
s s d
=
=
( )
( )
( )
1 2 3
,Sun ,Sat
alldiff , , , all
cardinality | ( , , , ),(5,5,5,5),(6,6,6,6)
nvalues ,..., |1,2 , all
d d d
s s
w w w
w
d
A B C D
w w s
( )
( )1 2 3
,Sun ,Sat
alldiff , all
stretch , , | (2,3),(2,2),(6,6), , all
, ,d d d
i i
y
P i
y
y y
dy
7/29/2019 Tutorial Cpa i Or
94/329
CPAIOR tutorial
May 2009 Slide 94
CP Filtering Algorithms
Element
Alldiff
Disjunctive Scheduling
Cumulative Scheduling
Filtering for element
7/29/2019 Tutorial Cpa i Or
95/329
CPAIOR tutorial
May 2009 Slide 95
Variable domains can be easily filtered to maintain GAC.
Domain of z
( )1element ,( , , ),ny x x z
{ }{ }
|
if
otherwise
j
y
j
j
j
z z x
j D
y y z x
z y
xx
D D D
D D j D D
D D jD
D
=
Filtering for element
7/29/2019 Tutorial Cpa i Or
96/329
CPAIOR tutorial
May 2009 Slide 96
Example...
The initial domains are: The reduced domains are:
( )1 2 3 4element ,( , , , ),y x x x x z
{ }
{ }
{ }{ }
{ }
{ }
1
2
3
4
20,30,60,80,90
1,3,4
10,5010,20
40,50,80,90
40,50,70
z
y
x
x
x
x
D
D
DD
D
D
=
=
==
=
=
{ }
{ }
{ }{ }
{ }
{ }
1
2
3
4
80,90
3
10,5010,20
80,90
40,50,70
z
y
x
x
x
x
D
D
DD
D
D
=
=
==
=
=
Filtering for alldiff
7/29/2019 Tutorial Cpa i Or
97/329
CPAIOR tutorial
May 2009 Slide 97
Domains can be filtered with an algorithm based on maximumcardinality bipartite matching and a theorem of Berge.
It is a special case of optimality conditions for max flow.
( )1alldiff , , ny y
Filtering for alldiff
7/29/2019 Tutorial Cpa i Or
98/329
CPAIOR tutorial
May 2009 Slide 98
Consider the domains
{ }{ }
{ }
{ }{ }
1
2
3
4
5
1
2,3,5
1,2,3,5
1,5
1,2,3,4,5,6
y
y
y
y
y
Indicate domains with edges
7/29/2019 Tutorial Cpa i Or
99/329
CPAIOR tutorial
May 2009 Slide 99
y1
y2
y3
y4
y5
1
2
3
4
5
6
g
Indicate domains with edges
7/29/2019 Tutorial Cpa i Or
100/329
CPAIOR tutorial
May 2009 Slide 100
y1
y2
y3
y4
y5
1
2
3
4
5
6
g
Find maximum cardinality bipartitematching.
Indicate domains with edges
7/29/2019 Tutorial Cpa i Or
101/329
CPAIOR tutorial
May 2009 Slide 101
y1
y2
y3
y4
y5
1
2
3
4
5
6
g
Find maximum cardinality bipartitematching.
Indicate domains with edges
7/29/2019 Tutorial Cpa i Or
102/329
CPAIOR tutorial
May 2009 Slide 102
y1
y2
y3
y4
y5
1
2
3
4
5
6
Find maximum cardinality bipartitematching.
Mark edges in alternating pathsthat start at an uncovered vertex.
Indicate domains with edges
7/29/2019 Tutorial Cpa i Or
103/329
CPAIOR tutorial
May 2009 Slide 103
y1
y2
y3
y4
y5
1
2
3
4
5
6
Find maximum cardinality bipartitematching.
Mark edges in alternating pathsthat start at an uncovered vertex.
Indicate domains with edges
7/29/2019 Tutorial Cpa i Or
104/329
CPAIOR tutorial
May 2009 Slide 104
y1
y2
y3
y4
y5
1
2
3
4
5
6
Find maximum cardinality bipartitematching.
Mark edges in alternating pathsthat start at an uncovered vertex.
Mark edges in alternating cycles.
Indicate domains with edges
7/29/2019 Tutorial Cpa i Or
105/329
CPAIOR tutorial
May 2009 Slide 105
y1
y2
y3
y4
y5
1
2
3
4
5
6
Find maximum cardinality bipartitematching.
Mark edges in alternating pathsthat start at an uncovered vertex.
Mark edges in alternating cycles.
Remove unmarked edges not inmatching.
Indicate domains with edges
7/29/2019 Tutorial Cpa i Or
106/329
CPAIOR tutorial
May 2009 Slide 106
y1
y2
y3
y4
y5
1
2
3
4
5
6
Find maximum cardinality bipartitematching.
Mark edges in alternating pathsthat start at an uncovered vertex.
Mark edges in alternating cycles.
Remove unmarked edges not inmatching.
Filtering for alldiff
7/29/2019 Tutorial Cpa i Or
107/329
CPAIOR tutorial
May 2009 Slide 107
Domains have been filtered:
{{ }
{ }
{ }{ }
1
2
3
4
5
12,3,5
1,2,3,5
1,5
1,2,3,4,5,6
yy
y
y
y
{ }{ }
{ }
{ }{ }
1
2
3
4
5
12,3
2,3
5
4,6
yy
y
y
y
GAC achieved.
Disjunctive scheduling
7/29/2019 Tutorial Cpa i Or
108/329
CPAIOR tutorial
May 2009 Slide 108
Consider a disjunctive scheduling constraint:
( )1 2 3 5 1 2 3 5disjunctive ( , , , ),( , , , )s s s s p p p p
Start time variables
Edge finding for disjunctive scheduling
7/29/2019 Tutorial Cpa i Or
109/329
CPAIOR tutorial
May 2009 Slide 109
Consider a disjunctive scheduling constraint:
( )1 2 3 5 1 2 3 5disjunctive ( , , , ),( , , , )s s s s p p p p
Processing times
Edge finding for disjunctive scheduling
7/29/2019 Tutorial Cpa i Or
110/329
CPAIOR tutorial
May 2009 Slide 110
Consider a disjunctive scheduling constraint:
( )1 2 3 5 1 2 3 5disjunctive ( , , , ),( , , , )s s s s p p p p
Variable domains defined by timewindows and processing times
1
2
3
5
[0,10 1]
[0,10 3][2,7 3]
[4,7 2]
s
ss
s
Edge finding for disjunctive scheduling
7/29/2019 Tutorial Cpa i Or
111/329
CPAIOR tutorial
May 2009 Slide 111
Consider a disjunctive scheduling constraint:
( )1 2 3 5 1 2 3 5disjunctive ( , , , ),( , , , )s s s s p p p p
A feasible (min makespan) solution:
Time window
Edge finding for disjunctive scheduling
But lets reduce 2 of the deadlines to 9:
7/29/2019 Tutorial Cpa i Or
112/329
CPAIOR tutorial
May 2009 Slide 112
Edge finding for disjunctive scheduling
But lets reduce 2 of the deadlines to 9:
7/29/2019 Tutorial Cpa i Or
113/329
CPAIOR tutorial
May 2009 Slide 113
We will use edge findingto prove that there is no
feasible schedule.
Edge finding for disjunctive scheduling
We can deduce that job 2 must precede jobs 3 and 5: 2 3,5
7/29/2019 Tutorial Cpa i Or
114/329
CPAIOR tutorial
May 2009 Slide 114
Because if job 2 is not first, there is not enough time for all 3jobs within the time windows:
{2,3,5} {3,5} {2,3,5}L E p J, and update Ek to
( )( ) 0
( )( )max
J k J J
J k J J JJ J
ke C c L E
e C c L E Ec
>
+
In general, if ( ){ } { }J k J k J e C L E > then k< J, and update Lk to
( )( ) 0
( )( )min
J k J J
J k J J J
J J ke C c L E
e C c L E L
c
>
Edge finding for cumulative scheduling
There is an O(n2) algorithm that finds all applications of theedge finding rules.
7/29/2019 Tutorial Cpa i Or
140/329
CPAIOR tutorial
May 2009 Slide 140
g g
Other propagation rules for cumulative
scheduling
Extended edge finding.
7/29/2019 Tutorial Cpa i Or
141/329
CPAIOR tutorial
May 2009 Slide 141
Timetabling.
Not-first/not-last rules.
Energetic reasoning.
7/29/2019 Tutorial Cpa i Or
142/329
CPAIOR tutorial
May 2009 Slide 142
Linear Relaxation
Why Relax?
Algebraic Analysis of LP
Linear Programming Duality
LP-Based Domain Filtering
Example: Single-Vehicle RoutingDisjunctions of Linear Systems
Why Relax?
Solving a relaxation of a problem can:
Tighten variable bounds
7/29/2019 Tutorial Cpa i Or
143/329
CPAIOR tutorial
May 2009 Slide 143
Tighten variable bounds.
Possibly solve original problem. Guide the search in a promising direction.
Filter domains using reduced costs or Lagrange multipliers.
Prune the search tree using a bound on the optimal value.
Provide a more global view, because a single OR relaxationcan pool relaxations of several constraints.
Some OR models that can provide relaxations:
Linear programming (LP).
Mixed integer linear programming (MILP)
7/29/2019 Tutorial Cpa i Or
144/329
CPAIOR tutorial
May 2009 Slide 144
g p g g ( )
Can itself be relaxed as an LP.
LP relaxation can be strengthened with cutting planes.
Lagrangean relaxation.
Specialized relaxations.
For particular problem classes. For global constraints.
Motivation
Linear programming is remarkably versatile for representingreal-world problems.
LP i b f h id l d l f l i
7/29/2019 Tutorial Cpa i Or
145/329
CPAIOR tutorial
May 2009 Slide 145
LP is by far the most widely used tool for relaxation.
LP relaxations can be strengthened by cutting planes.
- Based on polyhedral analysis.
LP has an elegant and powerful duality theory.
- Useful for domain filtering, and much else.
The LP problem is extremely well solved.
An example
Algebraic Analysis of LP
7/29/2019 Tutorial Cpa i Or
146/329
CPAIOR tutorial
May 2009 Slide 146
1 2
1 2
1 2
1 2
min 4 72 3 6
2 4
, 0
x xx x
x x
x x
++
+
2x1 + 3x2 6
2x1 + x2 4
4x1 + 7x2 = 12
Optimal solution
x= (3,0)
1 2min 4 7x x+
Rewrite
1 2min 4 7x x+
as
Algebraic Analysis of LP
7/29/2019 Tutorial Cpa i Or
147/329
CPAIOR tutorial
May 2009 Slide 147
1 2
1 2
1 2
2 3 6
2 4
, 0
x x
x x
x x
+
+
1 2 3
1 2 4
1 2 3 4
2 3 6
2 4
, , , 0
x x x
x x x
x x x x
+ =
+ =
In general an LP has the form min
0
cx
Ax b
x
=
Algebraic analysis of LP
Write as
=
min cx
Ax b
+
+ =
min B B N N
B N
c x c x
Bx Nx b
where
[ ]=A B N
7/29/2019 Tutorial Cpa i Or
148/329
CPAIOR tutorial
May 2009 Slide 148
0
Ax b
x , 0
B N
B Nx x
m nmatrixBasicvariables
[ ]
Any set ofmlinearlyindependent
columns of A.
These form abasis for thespace spanned
by the columns.
Nonbasic
variables
Algebraic analysis of LP
Write as
=
min cx
Ax b
+
+ =
min B B N N
B N
c x c x
Bx Nx b
where
[ ]=A B N
7/29/2019 Tutorial Cpa i Or
149/329
CPAIOR tutorial
May 2009 Slide 149
0
Ax b
x , 0
B N
B Nx x
[ ]
Solve constraint equation for xB: = 1 1B Nx B b B Nx
All solutions can be obtained by setting xN to some value.
The solution is basic if xN= 0.
It is a basic feasible solution if xN= 0 and xB 0.
Example = basic feasiblesolution
x2
7/29/2019 Tutorial Cpa i Or
150/329
CPAIOR tutorial
May 2009 Slide 150
1 2
1 2 3
1 2 4
1 2 3 4
min 4 72 3 6
2 4
, , , 0
x xx x x
x x x
x x x x
++ =
+ =
2x1 + 3x2 6
2x1 + x2 4
x2, x3 basic
x1, x2 basic
x1, x4 basic
x2, x4 basic
x3, x4 basicx1, x3 basic
x1
Algebraic analysis of LP
Write as
=
min cx
Ax b
+
+ =
min B B N N
B N
c x c x
Bx Nx b
where
[ ]=A B N
7/29/2019 Tutorial Cpa i Or
151/329
CPAIOR tutorial
May 2009 Slide 151
0x , 0B Nx x
Solve constraint equation for xB: = 1 1B Nx B b B Nx
Express cost in terms of nonbasic variables:
+ 1 1( )B N B N c B b c c B N x
Vector of reduced costs
Since xN 0,basic solution (xB,0)
is optimal ifreduced costs arenonnegative.
Examplex2
7/29/2019 Tutorial Cpa i Or
152/329
CPAIOR tutorial
May 2009 Slide 152
1 2
1 2 3
1 2 4
1 2 3 4
min 4 72 3 6
2 4
, , , 0
x xx x x
x x x
x x x x
++ =
+ =
x1, x4 basic
Consider this
basic feasiblesolution
x1
Example
Write as cBxB cNxN
7/29/2019 Tutorial Cpa i Or
153/329
CPAIOR tutorial
May 2009 Slide 153
[ ] [ ]
+
+ =
21
34
1 1
4 4
1 1
4 4
min 4 0 7 0
2 0 3 1 6
2 1 1 0 4
0,
0
xx
xx
x x
x x
x x
x x
BxB
NxN b
1 2
1 2 3
1 2 4
1 2 3 4
min 4 72 3 6
2 4
, , , 0
x xx x x
x x x
x x x x
++ =
+ =
Example
cBxB cNxN
7/29/2019 Tutorial Cpa i Or
154/329
CPAIOR tutorial
May 2009 Slide 154
[ ] [ ] +
+ =
21
34
1 1
4 4
1 1
4 4
min 4 0 7 0
2 0 3 1 6
2 1 1 0 4
0,
0
xx
xx
x x
x x
x x
x x
b
cBxB cNxN
BxB
NxN
Example
= =
= = =
1 1 1
1/ 21 0 6 3
B Nx B b B Nx B b
x
Basic solution is
cBxB cNxN
7/29/2019 Tutorial Cpa i Or
155/329
CPAIOR tutorial
May 2009 Slide 155
= = = 4 1 1 4 2x
x1, x4 basic
x2
x1
[ ] [ ] +
+ =
21
34
1 1
4 4
1 1
4 4
min 4 0 7 0
2 0 3 1 6
2 1 1 0 4
0,
0
xx
xx
x x
x x
x x
x x
B B N N
BxB
NxN b
ExampleBasic solution is
cBxB cNxN
= =
= = =
1 1 1
1/ 21 0 6 3
B Nx B b B Nx B b
x
7/29/2019 Tutorial Cpa i Or
156/329
CPAIOR tutorial
May 2009 Slide 156
[ ] [ ]
[ ] [ ]
1
1/ 2 0 3 17 0 4 0
1 1 1 0
1 2 0 0
N Bc c B N
=
=
Reduced costs are
Solution isoptimal
[ ] [ ] +
+ =
21
34
1 1
4 4
1 1
4 4
min 4 0 7 0
2 0 3 1 6
2 1 1 0 4
0,
0
xx
xx
x x
x x
x x
x x
B B N N
BxB
NxN
= = = 4 1 1 4 2x
Linear Programming Duality
An LP can be viewed as an inference problem
max v=min cx
7/29/2019 Tutorial Cpa i Or
157/329
CPAIOR tutorial
May 2009 Slide 157
0x
Ax b cx v
0
Ax b
ximplies
Dual problem: Find the tightest lower bound on theobjective function that is implied by the constraints.
An LP can be viewed as an inference problem
max v=min cx
7/29/2019 Tutorial Cpa i Or
158/329
CPAIOR tutorial
May 2009 Slide 158
0x
Ax b cx v
0
Ax b
x
From Farkas Lemma: If Ax b, x 0 is feasible,
0 dominatesiff
for some 0
x Ax b cx v Ax b cx v
A c and bv
That is, some surrogate(nonnegative linearcombination) of
Ax b dominates cx v
An LP can be viewed as an inference problem
max v=min cx max b= This is the
7/29/2019 Tutorial Cpa i Or
159/329
CPAIOR tutorial
May 2009 Slide 159
0x
Ax b cx v
0
Ax b
x
From Farkas Lemma: If Ax b, x 0 is feasible,
0 dominatesiff
for some 0
x Ax b cx v Ax b cx v
A c and bv
0
A c classicalLP dual
This equality is called strong duality.
=min cx max b This is the
7/29/2019 Tutorial Cpa i Or
160/329
CPAIOR tutorial
May 2009 Slide 160
0
Ax b
x
If Ax b, x 0 is feasible
0
A c classicalLP dual
Note that the dual of the dual is the primal(i.e., the original LP).
+ =
+ 1
1 2
1 2
min 4 7
2 3 ( )6
x x
x x
Example
1 2
1 12 (
max 6 4
)
12
2 2 4 x
+ =
+
Primal Dual
7/29/2019 Tutorial Cpa i Or
161/329
CPAIOR tutorial
May 2009 Slide 161
+
1 2
1 2
1( )2 4
, 0
x x
x x
21 2
21
(3 7
,
)
0
x
+
A dual solution is (1,2) = (2,0)
2
2
1
1 2
1( 2)
(
6
0)
2 3
2 4
x x
x x
+
+
=
=
1 24 6 12x x+
1 24 7 12x x+
dominates
Dual multipliers
Surrogate
Tightest bound on cost
Weak Duality
If x* is feasible in theprimal problem
min cx
and * is feasible in thedual problem
then cx* *b.
7/29/2019 Tutorial Cpa i Or
162/329
CPAIOR tutorial
May 2009 Slide 162
min
0
cx
Ax b
x
max
0
bA c
This is becausecx* *Ax* *b
*is dualfeasible
and x* 0
x* is primal
feasible
and * 0
Dual multipliers as marginal costs
min cx
Ax b b +
Suppose we perturb the RHS of an LP(i.e., change the requirement levels):
7/29/2019 Tutorial Cpa i Or
163/329
CPAIOR tutorial
May 2009 Slide 163
0xThe dual of the perturbed LP has thesame constraints at the original LP:
max (
0
)bb
A c
+
So an optimal solution * of the original dual is feasible in theperturbed dual.
Dual multipliers as marginal costs
min cx
Ax b b +
Suppose we perturb the RHS of an LP(i.e., change the requirement levels):
7/29/2019 Tutorial Cpa i Or
164/329
CPAIOR tutorial
May 2009 Slide 164
0xBy weak duality, the optimal value of the perturbed LP is at least
*(b+ b) = *b+ *b.
So i* is a lower bound on the marginal cost of increasing thei-th requirement by one unit (bi= 1).
Optimal value of original LP, by strong duality.
If i* > 0, the i-th constraint must be tight (complementary slackness).
Dual of an LP in equality form
Primal Dual
+
+ =
m n
)
i
(
B B N N
B N
c x c x
Bx Nx b (
max
)
)(
B
N
B
B
x
b
B c
c xN
7/29/2019 Tutorial Cpa i Or
165/329
CPAIOR tutorial
May 2009 Slide 165
, 0B Nx x unrestric d
)
te
(N B
Dual of an LP in equality form
Primal Dual
+
+ =
m n
)
i
(
B B N N
B N
c x c x
Bx Nx b (
max
)
)(
B
N
B
B
x
b
B c
c xN
7/29/2019 Tutorial Cpa i Or
166/329
CPAIOR tutorial
May 2009 Slide 166
Recall that reduced cost vector is 1N Bc c B N
Nc N=
this solves the dualif (xB,0) solves the primal
, 0B Nx x unrestric dteN B
Dual of an LP in equality form
Primal Dual
+
+ =
m n
)
i
(
B B N N
B N
c x c x
Bx Nx b (
max
)
)(
B
N
B
B
x
b
B c
c xN
7/29/2019 Tutorial Cpa i Or
167/329
CPAIOR tutorial
May 2009 Slide 167
Check: 1
1
B B
B N
B c B B c
N c B N c
= =
=
Because reduced cost is nonnegativeat optimal solution (xB,0).
Recall that reduced cost vector is 1N Bc c B N
Nc N=
this solves the dualif (xB,0) solves the primal
, 0B Nx x unrestric dte
Dual of an LP in equality form
+
+ =
m n
)
i
(
B B N N
B N
c x c x
Bx Nx b
Primal Dual
(
max
)
)(
B
N
B
B
x
b
B c
c xN
7/29/2019 Tutorial Cpa i Or
168/329
CPAIOR tutorial
May 2009 Slide 168
, 0B Nx x unrestric dte
[ ] [ ]1/ 21 04 0 2 01 1
Bc B = = =
In the example,
Recall that reduced cost vector is 1N Bc c B N
Nc N=
this solves the dualif (xB,0) solves the primal
Dual of an LP in equality form
+
+ =
m n
)
i
(
B B N N
B N
c x c x
Bx Nx b
Primal Dual
(
max
)
)(
B
N
B
B
x
b
B c
c xN
7/29/2019 Tutorial Cpa i Or
169/329
CPAIOR tutorial
May 2009 Slide 169
, 0B Nx x unrestric dte
Note that the reduced cost of an individual variable xj is
Recall that reduced cost vector is 1N Bc c B N
Nc N=
j j jr c A=
Columnjof A
LP-based Domain Filtering
min
0
cx
Ax b
x
Let be an LP relaxation of a CP problem.
7/29/2019 Tutorial Cpa i Or
170/329
CPAIOR tutorial
May 2009 Slide 170
One way to filter the domain of xj is to minimize and maximize xjsubject to Ax b, x 0.
- This is time consuming.
A faster method is to use dual multipliers to derive validinequalities.
- A special case of this method uses reduced costs to bound orfix variables.
- Reduced-cost variable fixing is a widely used technique in OR.
min
0
cx
Ax b
x
Suppose:
has optimal solution x*, optimal value v*, and
optimal dual solution *.
7/29/2019 Tutorial Cpa i Or
171/329
CPAIOR tutorial
May 2009 Slide 171
and i*> 0, which means the i-th constraint is tight(complementary slackness);
and the LP is a relaxation of a CP problem;
and we have a feasible solution of the CP problem with valueU, so that Uis an upper bound on the optimal value.
min
0
cx
Ax bx
Supposinghas optimal solution x*, optimal value v*, and
optimal dual solution *:
If t h t l th th * th LHS f i th t i t
7/29/2019 Tutorial Cpa i Or
172/329
CPAIOR tutorial
May 2009 Slide 172
If xwere to change to a value other than x*, the LHS of i-th constraintAix bi would change by some amount bi.
Since the constraint is tight, this would increase the optimal value
as much as changing the constraint to Aix bi+ bi.
So it would increase the optimal value at least i*bi.
W h f d h i th t h Ai
b bi i
min
0
cx
Ax bx
Supposinghas optimal solution x*, optimal value v*, and
optimal dual solution *:
7/29/2019 Tutorial Cpa i Or
173/329
CPAIOR tutorial
May 2009 Slide 173
We have found: a change in xthat changes Axby bi increasesthe optimal value of LP at least i*bi.
Since optimal value of the LP optimal value of the CP U,
we have i*bi U v*, or **i
i
U vb
min
0
cx
Ax bx
Supposinghas optimal solution x*, optimal value v*, and
optimal dual solution *:
We have found: a change in x that changes Ai
x by bi increases
7/29/2019 Tutorial Cpa i Or
174/329
CPAIOR tutorial
May 2009 Slide 174
Since bi= Aix Aix* = Aix bi, this implies the inequality
*
*i
i
i
U vA x b +
which can be propagated.
We have found: a change in xthat changes Axby bi increasesthe optimal value of LP at least i*bi.
Since optimal value of the LP optimal value of the CP U,
we have i*bi U v*, or **i
i
U vb
1 2
1 2
1 2
1
1
( 2
min
)
4 7
2 3 6
2 4 ( 0)
x x
x x
x x
=
+
+ =
+
Example
Suppose we have a feasible solutionof the original CP with value U= 13.
7/29/2019 Tutorial Cpa i Or
175/329
CPAIOR tutorial
May 2009 Slide 175
1 2
21
1
, 0x x
*
11 *
1
U vA x b +
1 2
13 12
2 3 6 6.52x x
+ + =
Since the first constraint is tight, we can propagatethe inequality
or
Reduced-cost domain filtering
Suppose xj*= 0, which means the constraint xj 0 is tight.
*
i U vA x b + becomes*
U vx The inequality
7/29/2019 Tutorial Cpa i Or
176/329
CPAIOR tutorial
May 2009 Slide 176
*i
i
i
U vA x b + becomes j
j
U vxrThe inequality
The dual multiplier for xj 0 is the reduced cost
rjof xj, because increasing xj(currently 0) by 1increases optimal cost by rj.
Similar reasoning can bound a variable below when it is at its
upper bound.
1 2
1 2
1 2
1
1
( 2
min
)
4 7
2 3 6
2 4 ( 0)
x x
x x
x x
=
+
+ =
+
Example
Suppose we have a feasible solutionof the original CP with value U= 13.
7/29/2019 Tutorial Cpa i Or
177/329
CPAIOR tutorial
May 2009 Slide 177
1 2
21
1
, 0x x
Since x2* = 0, we have
or
*
2
2
U vx
r
2
13 12 0.52
x =
If x2 is required to be integer, we can fix it to zero.This is reduced-cost variable fixing.
Example: Single-Vehicle Routing
A vehicle must make several stops and return home, perhaps subjectto time windows.
The objective is to find the order of stops that minimizes travel time.
This is also known as the traveling salesman problem (with time
7/29/2019 Tutorial Cpa i Or
178/329
CPAIOR tutorial
May 2009 Slide 178
This is also known as the traveling salesman problem (with timewindows).
Stop i
Stopj
Travel time cij
Assignment Relaxation
min
1, all
ij ij ij
ij ji
j j
c x
x x i= =
= 1 if stop iimmediately precedes stopj
Stop iis preceded and
followed by exactly one stop
7/29/2019 Tutorial Cpa i Or
179/329
CPAIOR tutorial
May 2009 Slide 179
{ }0,1 , all ,j j
ijx i jfollowed by exactly one stop.
Assignment Relaxation
min
1, all
ij ij ij
ij ji
j j
c
xx i
x
= =
= 1 if stop iimmediately precedes stopj
Stop iis preceded and
followed by exactly one stop
7/29/2019 Tutorial Cpa i Or
180/329
CPAIOR tutorial
May 2009 Slide 180
0 1, all ,
j j
ij i jx followed by exactly one stop.
Because this problem is totally unimodular, it can be solved as an LP.
The relaxation provides a very weak lower bound on the optimal value.
But reduced-cost variable fixing can be very useful in a CP context.
Disjunctions of linear systems often occur naturally in problemsand can be given a convex hull relaxation.
A disjunction of linear systems min cx
Disjunctions of linear systems
7/29/2019 Tutorial Cpa i Or
181/329
CPAIOR tutorial
May 2009 Slide 181
A disjunction of linear systemsrepresents a union of polyhedra.
( )
min
k k
k
cx
A x b
Disjunctions of linear systems often occur naturally in problemsand can be given a convex hull relaxation.
A disjunction of linear systems min cx
Relaxing a disjunction of linear systems
7/29/2019 Tutorial Cpa i Or
182/329
CPAIOR tutorial
May 2009 Slide 182
d sju ct o o ea syste srepresents a union of polyhedra.
We want a convex hull relaxation(tightest linear relaxation).
( )
min
k k
k
cx
A x b
Relaxing a disjunction of linear systems
Disjunctions of linear systems often occur naturally in problemsand can be given a convex hull relaxation.
The closure of the convex hull of min cx
7/29/2019 Tutorial Cpa i Or
183/329
CPAIOR tutorial
May 2009 Slide 183
( )k kk
A x b
min
, all
1
0 1
k k k
k
kk
k
k
k
cx
A x b y k
y
x x
y
=
=
is described by
Why?
To derive convex hullrelaxation of a disjunction
min
allk k kcx
A x b k
7/29/2019 Tutorial Cpa i Or
184/329
CPAIOR tutorial
May 2009 Slide 184
Convex hull relaxation(tightest linear relaxation)
, all
1
0 1
kk
k
kk
k
A x b k
y
x y x
y
=
=
Write eachsolution as aconvex
combinationof points inthe
polyhedron
x1x
2x
Why? min
, all1
0 1
k k k
k
kk
k
k
k
cx
A x b y k y
x x
y
=
=
To derive convex hullrelaxation of a disjunction
min
allk k kcx
A x b k
Change of
variablek
kx y x=
7/29/2019 Tutorial Cpa i Or
185/329
CPAIOR tutorial
May 2009 Slide 185
Convex hull relaxation(tightest linear relaxation)
0 1ky , all
1
0 1
kk
k
kk
k
A x b k
y
x y x
y
=
=
Write eachsolution as aconvex
combinationof points inthe
polyhedron
x1x
2x
Mi d I t /Li M d li
7/29/2019 Tutorial Cpa i Or
186/329
CPAIOR tutorial
May 2009 Slide 186
Mixed Integer/Linear Modeling
MILP RepresentabilityDisjunctive Modeling
Knapsack Modeling
Motivation
A mixed integer/linear programming(MILP) problem has the form
min
, 0
integer
cx dy
Ax by b
x y
y
+
+
7/29/2019 Tutorial Cpa i Or
187/329
CPAIOR tutorial
May 2009 Slide 187
We can relax a CP problem by modeling some constraints with an MILP.
If desired, we can then relax the MILP by dropping the integrality constraint,to obtain an LP.
The LP relaxation can be strengthened with cutting planes.
The first step is to learn how to write MILP models.
MILP Representability
A subset Sof is MILP representable if it is the projection onto xof some MILP constraint set of the form
{ }
+ +
R R
, 00 1n m
Ax Bu Dy b
x y
Rn
7/29/2019 Tutorial Cpa i Or
188/329
CPAIOR tutorial
May 2009 Slide 188
{ } R R, , 0,1n m ky
x u y
MILP Representability
A subset Sof is MILP representable if it is the projection onto xof some MILP constraint set of the form
{ }
+ +
R R
, 00 1n m
Ax Bu Dy b
x y
Rn
Recession cone
7/29/2019 Tutorial Cpa i Or
189/329
CPAIOR tutorial
May 2009 Slide 189
{ } R R, , 0,1n m kx u y
Theorem. S is MILPrepresentable if and only ifSis the union of finitelymany polyhedra having the
same recession cone.
nR
Polyhedron
Recession coneof polyhedron
Example: Fixed charge function
Minimize a fixed charge function: 2
1
2
1 1
1
min0 if 0
if 0
0
xx
xf cx x
x
=
+ >
7/29/2019 Tutorial Cpa i Or
190/329
CPAIOR tutorial
May 2009 Slide 190
x1
x2
Example
Minimize a fixed charge function: 2
1
2
1 1
1
min0 if 0
if 0
0
xx
xf cx x
x
=
+ >
7/29/2019 Tutorial Cpa i Or
191/329
CPAIOR tutorial
May 2009 Slide 191
x1
x2
Feasible set
Example
Minimize a fixed charge function: 2
1
2
1 1
1
min0 if 0
if 0
0
xx
xf cx x
x
=
+ >
x
7/29/2019 Tutorial Cpa i Or
192/329
CPAIOR tutorial
May 2009 Slide 192
x1
x2
Union of two
polyhedraP1, P2
P1
Example
Minimize a fixed charge function: 2
1
2
1 1
1
min0 if 0
if 0
0
xx
xf cx x
x
=
+ >
x
7/29/2019 Tutorial Cpa i Or
193/329
CPAIOR tutorial
May 2009 Slide 193
x1
x2
Union of two
polyhedraP1, P2
P1
P2
Example
Minimize a fixed charge function: 2
1
2
1 1
1
min0 if 0
if 0
0
xx
xf cx x
x
=
+ >
x
7/29/2019 Tutorial Cpa i Or
194/329
CPAIOR tutorial
May 2009 Slide 194
x1
x2
The
polyhedrahavedifferentrecession
cones.
P1
P1recession
cone
P2
P2recession
cone
Example
Minimize a fixed charge function:
Add an upper bound on x1
2
1
2
1 1
1
min0 if 0
if
0
0
xx
xf cx x
x M
=
+ >
x
7/29/2019 Tutorial Cpa i Or
195/329
CPAIOR tutorial
May 2009 Slide 195
x1
x2
The
polyhedrahave thesamerecession
cone.
P1
P1recession
cone
P2
P2recession
coneM
Modeling a union of polyhedra
Start with a disjunction of linearsystems to represent the unionof polyhedra.
The kth polyhedron is {x | Akx b}
( )
min
k k
k
cx
A x b
7/29/2019 Tutorial Cpa i Or
196/329
CPAIOR tutorial
May 2009 Slide 196
Introduce a 0-1 variable yk that is1 when xis in polyhedron k.
Disaggregate xto create an xk foreach k.
{ }
min
, all
1
0,1
k k k
k
kk
k
k
k
cx
A x b y k
y
x x
y
=
=
Example
Start with a disjunction oflinear systems to representthe union of polyhedra
2
1 1
2 2 1
min
0 0
0
x
x x M
x x f cx
= +
x2
7/29/2019 Tutorial Cpa i Or
197/329
CPAIOR tutorial
May 2009 Slide 197
x1
x2
P1
P2
M
Example
Start with a disjunction oflinear systems to representthe union of polyhedra
2
1 1
2 2 1
min
0 0
0
x
x x M
x x f cx
= +
min cx
7/29/2019 Tutorial Cpa i Or
198/329
CPAIOR tutorial
May 2009 Slide 198
{ }
1 1
1 2
2 2 2
1 2 1 2 2
1 2
1 2
min
0, 0
0 ,1, 0,1k
cx
x x
x My cx x fy y y y
x x x
=
+ + =
= +
Introduce a 0-1 variable ykthat is 1 when xis inpolyhedron k.
Disaggregate xto create anxk for each k.
Example
To simplify:
Replace x12 with x1.
Replace x22 with x2.
Replace y2 with y. { }
2
1 1
1 2
2 2 2
1 2 1 2 2
1 2
1 2
min
0, 0
0 ,
1, 0,1k
x
x x
x My cx x fy
y y y
x x x
=
+
+ = +
7/29/2019 Tutorial Cpa i Or
199/329
CPAIOR tutorial
May 2009 Slide 199
x x x= +
This yields
{ }
2
1
2 1
min
0
0,1
x
x My
x fy cx
y
+
{ }
min
0
0,1
fy cx
x My
y
+
or
Big M
Disjunctive Modeling
Disjunctions often occur naturally in problems and can be givenan MILP model.
Recall that a disjunction of linearsystems (representing polyhedra
( )
mink k
cx
A x b
7/29/2019 Tutorial Cpa i Or
200/329
CPAIOR tutorial
May 2009 Slide 200
with the same recession cone) ( )k k
kA x b
{ }
min
, all
1
0,1
k k k
k
kk
k
k
k
cx
A x b y k
y
x x
y
=
=
has the MILP model
Example: Uncapacitated facility location
mpossiblefactory
locations nmarkets Locate factories to servemarkets so as to minimizetotal fixed cost and
transport cost.No limit on production
7/29/2019 Tutorial Cpa i Or
201/329
CPAIOR tutorial
May 2009 Slide 201
i j
fi cij
Fixed
cost Transportcost
pcapacity of each factory.
Uncapacitated facility location
nmarkets Disjunctive model:
min
0, all 0 1, all, all
0
i ij ij i ij
ij ij
i i i
z c x
x j x j i
z z f
+
= =
Fraction of
marketjs demandsatisfied fromlocation impossible
factorylocations
7/29/2019 Tutorial Cpa i Or
202/329
CPAIOR tutorial
May 2009 Slide 202
i j
fi cij
Fixed
cost Transportcost
0
1, all
i i i
iji
z z f
x j
=
No factoryat location i
Factoryat location i
7/29/2019 Tutorial Cpa i Or
203/329
Uncapacitated facility location
MILP formulation:
{ }
min
0 , all ,
0 1
i i ij ij i ij
ij i
f y c x
x y i j
y
+
Beginners model:
min
, all ,
i i ij ij i ij
ij ij
f y c x
x ny i j
+
Maximum output
from location i
7/29/2019 Tutorial Cpa i Or
204/329
CPAIOR tutorial
May 2009 Slide 204
{ }0,1iy { }0,1iy
Based on capacitated location model.
It has a weaker continuous relaxation
(obtained by replacing yi {0,1} with 0 yi 1).
This beginners mistake can be avoided bystarting with disjunctive formulation.
Knapsack Modeling
Knapsack models consist of knapsack covering andknapsack packing constraints.
The freight transfer model presented earlier is an example.
We will consider a similar example that combines disjunctiveand knapsack modeling.
7/29/2019 Tutorial Cpa i Or
205/329
CPAIOR tutorial
May 2009 Slide 205
Most OR professionals are unlikely to write a model as goodas the one presented here.
Note on tightness of knapsack models
The continuous relaxation of a knapsack model is not in generala convex hull relaxation.
- A disjunctive formulation would provide a convex hullrelaxation, but there are exponentially many disjuncts.
Knapsack cuts can significantly tighten the relaxation.
7/29/2019 Tutorial Cpa i Or
206/329
CPAIOR tutorial
May 2009 Slide 206
min
; 1, all
ii
i i j ij i j i
z
Q y a x j =
Example: Package transport
Each packagejhas size aj
Each truck i has
Disjunctive model Knapsackconstraints
7/29/2019 Tutorial Cpa i Or
207/329
CPAIOR tutorial
May 2009 Slide 207
{ }
1
0
0 , all
00 1, all
, 0,1
i
ii i
ij ij ij
ij
ij
ij i
y
yz c
z ia x Q
xx j
x y
= ==
= =
Each truck ihascapacity Qiand
costs ci tooperate
Truck iusedTruck inot used
1 if truck icarriespackagej 1 if truck iis used
Example: Package transport
min
; 1, all
ii
i i j ij i j i
z
Q y a x j =
Disjunctive modelMILP model
min
; 1, all
i ii
i i j ij i j i
c y
Q y a x j =
7/29/2019 Tutorial Cpa i Or
208/329
CPAIOR tutorial
May 2009 Slide 208
{ }
1
0
0 , all
00 1, all
, 0,1
i
ii i
ij ij ij
ij
ij
ij i
y
yz c
z ia x Q
xx j
x y
= ==
= =
{ }
, all
, all ,, 0,1
j ij i ij
ij i
ij i
a x Q y i
x y i j x y
7/29/2019 Tutorial Cpa i Or
209/329
Cutting Planes
7/29/2019 Tutorial Cpa i Or
210/329
CPAIOR tutorial
May 2009 Slide 210
0-1 Knapsack Cuts
Gomory Cuts
Mixed Integer Rounding Cuts
Example: Product Configuration
Cuttingplane Continuous
relaxation
To review
A cutting plane (cut, valid inequality) foran MILP model:
is valid
- It is satisfied by all feasible solutionsof the model.
cuts off solutions of the continuous
7/29/2019 Tutorial Cpa i Or
211/329
CPAIOR tutorial
May 2009 Slide 211
Feasible solutions
cuts o so ut o s o t e co t uousrelaxation.
- This makes the relaxation tighter.
Motivation
Cutting planes (cuts) tighten the continuous relaxation of anMILP model.
Knapsack cuts
- Generated for individual knapsack constraints.- We saw general integer knapsack cuts earlier.
7/29/2019 Tutorial Cpa i Or
212/329
CPAIOR tutorial
May 2009 Slide 212
- 0-1 knapsack cuts and lifting techniques are well studiedand widely used.
Rounding cuts
- Generated for the entire MILP, they are widely used.
- Gomory cuts for integer variables only.
- Mixed integer rounding cuts for any MILP.
0-1 Knapsack Cuts
0-1 knapsack cuts are designed for knapsack constraints with 0-1variables.
The analysis is different from that of general knapsack constraints,
to exploit the special structure of 0-1 inequalities.
7/29/2019 Tutorial Cpa i Or
213/329
CPAIOR tutorial
May 2009 Slide 213
0-1 Knapsack Cuts
0-1 knapsack cuts are designed for knapsack constraints with 0-1variables.
The analysis is different from that of general knapsack constraints,
to exploit the special structure of 0-1 inequalities.
Consider a 0-1 knapsack packing constraint ax a0. (Knapsackcovering constraints are similarly analyzed )
7/29/2019 Tutorial Cpa i Or
214/329
CPAIOR tutorial
May 2009 Slide 214
covering constraints are similarly analyzed.)
Index set Jis a cover if 0jj J
a a
>The cover inequality is a 0-1 knapsack cut for
ax a0
1jj J
x J
Only minimal covers need be considered.
Example
J= {1,2,3,4} is a cover for
1 2 3 4 5 66 5 5 5 8 3 17x x x x x x + + + + +
This gives rise to the cover inequality
1 2 3 4 3x x x x + + +
7/29/2019 Tutorial Cpa i Or
215/329
CPAIOR tutorial
May 2009 Slide 215
Index set Jis a cover if 0jj J
a a
>The cover inequality is a 0-1 knapsack cut for
ax a0
1jj J
x J
Only minimal covers need be considered.
Sequential lifting
A cover inequality can often be strengthened by lifting it into ahigher dimensional space.
That is, by adding variables.
Sequential lifting adds one variable at a time.
Sequence-independent lifting adds several variables at once.
7/29/2019 Tutorial Cpa i Or
216/329
CPAIOR tutorial
May 2009 Slide 216
Sequential lifting
To lift a cover inequality 1jj J
x J