Geometry and Theory of LP Standard (Inequality) Primal Problem: Dual Problem:

Geometry and Theory of LPGeometry and Theory of LPStandard (Inequality) Primal Problem:Standard (Inequality) Primal Problem:

Dual Problem:Dual Problem:

0

s.t.

Min

0

s.t.

Max

y

cy A

yb

x

bA x

xc

T

T

T

Geometry of the Prototype ExampleGeometry of the Prototype Example

P1

P2

0

Every point in this nonnegative quadrant isassociated with a specific production alternative.

( point = decision = solution )

Max 3 P1 + 5 P2Max 3 P1 + 5 P2

s.t. P1 + s.t. P1 + << 4 (Plant 1) 4 (Plant 1)

2 P2 2 P2 << 12 (Plant 2) 12 (Plant 2)

3 P1 + 2 P2 3 P1 + 2 P2 << 18 (Plant 3) 18 (Plant 3)

P1, P2 P1, P2 >> 0 0 (nonnegativity) (nonnegativity)

Geometry of the Prototype ExampleGeometry of the Prototype Example

P1

P2

(0,0)

Max 3 P1 + 5 P2Max 3 P1 + 5 P2

s.t. s.t. P1 + P1 + << 4 4 (Plant 1) (Plant 1)

2 P2 2 P2 << 12 (Plant 2) 12 (Plant 2)

3 P1 + 2 P2 3 P1 + 2 P2 << 18 (Plant 3) 18 (Plant 3)

P1, P2 P1, P2 >> 0 0 (nonnegativity) (nonnegativity)

(4,0)

Geometry of the Prototype ExampleGeometry of the Prototype Example

P1

P2

(0,0)

Max 3 P1 + 5 P2Max 3 P1 + 5 P2

s.t. s.t. P1 + P1 + << 4 4 (Plant 1) (Plant 1)

2 P2 2 P2 << 12 12 (Plant 2) (Plant 2)

3 P1 + 2 P2 3 P1 + 2 P2 << 18 (Plant 3) 18 (Plant 3)

P1, P2 P1, P2 >> 0 0 (nonnegativity) (nonnegativity)

(4,0)

(0,6)

Geometry of the Prototype ExampleGeometry of the Prototype Example

P1

P2

(0,0)

Max 3 P1 + 5 P2Max 3 P1 + 5 P2

s.t. s.t. P1 + P1 + << 4 4 (Plant 1) (Plant 1)

2 P2 2 P2 << 12 12 (Plant 2) (Plant 2)

3 P1 + 2 P2 3 P1 + 2 P2 << 18 18 (Plant 3) (Plant 3)

P1, P2 P1, P2 >> 0 0 (nonnegativity) (nonnegativity)

(4,0)

(0,6) (2,6)

(4,3)

(9,0)

Geometry of the Prototype ExampleGeometry of the Prototype Example

P1

P2

(0,0)

Max 3 P1 + 5 P2Max 3 P1 + 5 P2

s.t. s.t. P1 + P1 + << 4 4 (Plant 1) (Plant 1)

2 P2 2 P2 << 12 12 (Plant 2) (Plant 2)

3 P1 + 2 P2 3 P1 + 2 P2 << 18 18 (Plant 3) (Plant 3)

P1, P2 P1, P2 >> 0 0 (nonnegativity) (nonnegativity)

(4,0)

(0,6) (2,6)

(4,3)

(9,0)

P1

P2

(0,0)

Max 3 P1 + 5 P2Max 3 P1 + 5 P2

In Feasible RegionIn Feasible Region

(4,0)

(0,6) (2,6)

(4,3)

(9,0)

Feasible region is the set of points (solutions) that simultaneously satisfy all the constraints. There are infinitely many feasible points (solutions).

Geometry of the Prototype ExampleGeometry of the Prototype Example

P1

P2

(0,0)

Max Max 3 P1 + 5 P23 P1 + 5 P2

(4,0)

(0,6) (2,6)

(4,3)

(9,0)

Objective function contour

(iso-profit line)

3 P1 + 5 P2 = 12

Geometry of the Prototype ExampleGeometry of the Prototype Example

P1

P2

(0,0)

Max Max 3 P1 + 5 P23 P1 + 5 P2

s.t. s.t. P1 + P1 + << 4 4 (Plant 1) (Plant 1)

2 P2 2 P2 << 12 12 (Plant 2) (Plant 2)

3 P1 + 2 P2 3 P1 + 2 P2 << 18 18 (Plant 3) (Plant 3)

P1, P2 P1, P2 >> 0 0 (nonnegativity) (nonnegativity)

(4,0)

(0,6) (2,6)

(4,3)

(9,0)

Optimal Solution: the solution for the simultaneous boundary equations of two active constraints

3 P1 + 5 P2 = 36

DegeneracyDegeneracy

P1

P2

(0,0)

Max Max 3 P1 + 5 P23 P1 + 5 P2

s.t. s.t. P1 + P1 + << 4 4 (Plant 1) (Plant 1)

2 P2 2 P2 << 12 12 (Plant 2) (Plant 2)

3 P1 + 2 P2 3 P1 + 2 P2 << 24 24 (Plant 3) (Plant 3)

P1, P2 P1, P2 >> 0 0 (nonnegativity) (nonnegativity)

(4,0)

(0,6) (2,6)

(9,0)

The number of active constraints is more than the number of variables.

LP TerminologyLP Terminology solution (decision, point): any specification of values for all

decision variables, regardless of whether it is a desirable or even allowable choice

feasible solution: a solution for which all the constraints are satisfied.

feasible region (constraint set, feasible set): the collection of all feasible solution

objective function contour (iso-profit, iso-cost line) optimal solution (optimum): a feasible solution that has the

most favorable value of the objective function optimal (objective) value: the value of the objective function

evaluated at an optimal solution active constraint (binding constraint) inactive constraint redundant constraint interior, boundary extreme point (corner)

Unbounded or Infeasible CaseUnbounded or Infeasible Case

On the left, the objective function is unboundedOn the left, the objective function is unbounded On the right, the feasible set is emptyOn the right, the feasible set is empty

Graphical Solution SeekingGraphical Solution Seeking Plot the feasible region. Plot the feasible region. If the region is empty, stop: the problem is infeasible; there If the region is empty, stop: the problem is infeasible; there

must be conflicting constraints in the model. must be conflicting constraints in the model. Plot the objective function contour and choose the optimizing Plot the objective function contour and choose the optimizing

direction.direction. Determine whether the objective value is bounded or not. If Determine whether the objective value is bounded or not. If

not, stop: the problem is unbounded; there must be mistakes in not, stop: the problem is unbounded; there must be mistakes in model formulation.model formulation.

Determine an optimal corner point.Determine an optimal corner point. Identify active constraints at this corner.Identify active constraints at this corner. Solve simultaneous linear equations for the optimal solution.Solve simultaneous linear equations for the optimal solution. Evaluate the objective function at the optimal solution to Evaluate the objective function at the optimal solution to

obtain the optimal value of the problem.obtain the optimal value of the problem.

Theory of Linear ProgrammingTheory of Linear ProgrammingAn LP problem falls in one of three cases:An LP problem falls in one of three cases: Problem is Problem is infeasibleinfeasible: Feasible region is empty.: Feasible region is empty. Problem is Problem is unboundedunbounded: Feasible region is unbounded : Feasible region is unbounded

towards the optimizing direction.towards the optimizing direction. Problem is Problem is feasiblefeasible and and boundedbounded: then there exists an : then there exists an

optimal point; an optimal point is on the boundary of the optimal point; an optimal point is on the boundary of the feasible region; and there is always at least one optimal feasible region; and there is always at least one optimal corner point (if the feasible region has a corner point).corner point (if the feasible region has a corner point).

When the problem is feasible and bounded,When the problem is feasible and bounded, There may be a unique optimal point or multiple optima There may be a unique optimal point or multiple optima

(alternative optima).(alternative optima). If a corner point is not “worse” than all its neighbor If a corner point is not “worse” than all its neighbor

corners, then it is optimal.corners, then it is optimal.

Convexity of Feasible RegionConvexity of Feasible Region

Convex Set Non-Convex SetConvex Set Non-Convex Set

FF is a convex set if and only if for any two points, x and y, of is a convex set if and only if for any two points, x and y, of FF, their convex , their convex combination, combination, x + (1- x + (1- )y, for all real values 0 <= )y, for all real values 0 <= <= 1, is also in F. <= 1, is also in F.

Local Optimal => Global OptimalLocal Optimal => Global Optimal

Convex Set Convex Set

Proof by contradiction: If the point is not globally optimal, then it is not Proof by contradiction: If the point is not globally optimal, then it is not locally optimallocally optimal

LP DualityLP Duality

Standard (Inequality) Primal Problem:Standard (Inequality) Primal Problem:

Dual Problem:Dual Problem:

0

s.t.

Min

0

s.t.

Max

y

cy A

yb

x

bA x

xc

T

T

T

LP Duality (continued)LP Duality (continued)

Standard (Equality) Primal Form:Standard (Equality) Primal Form:

Dual Form:Dual Form:

cy A

yb

x

bA x

xc

T

T

T

s.t.

Max

0

s.t.

Min

General Rules for Constructing DualGeneral Rules for Constructing Dual

1. 1. The number of variables in the dual problem is equal to the number of The number of variables in the dual problem is equal to the number of constraints in the original (primal) problem. The number of constraints in constraints in the original (primal) problem. The number of constraints in the dual problem is equal to the number of variables in the original problem.the dual problem is equal to the number of variables in the original problem.

2. 2. Coefficient of the objective function in the dual problem come from the right-Coefficient of the objective function in the dual problem come from the right-hand side of the original problem.hand side of the original problem.

3.3. If the original problem is a If the original problem is a maxmax model, the dual is a model, the dual is a minmin model; if the original model; if the original problem is aproblem is a minmin model, the dual problem is the model, the dual problem is the maxmax problem.problem.

4.4. The coefficient of the first constraint function for the dual problem are the The coefficient of the first constraint function for the dual problem are the coefficients of the first variable in the constraints for the original problem, coefficients of the first variable in the constraints for the original problem, and the similarly for other constraints.and the similarly for other constraints.

5. 5. The right-hand sides of the dual constraints come from the objective function The right-hand sides of the dual constraints come from the objective function coefficients in the original problem.coefficients in the original problem.

General Rules for Constructing Dual ( Continued)General Rules for Constructing Dual ( Continued)

6. 6. The sense of the The sense of the iith constraint in the dual is = if and only if the th constraint in the dual is = if and only if the iith variable th variable in the original problem is unrestricted in sign.in the original problem is unrestricted in sign.

7. 7. If the original problem is man (min ) model, then after applying Rule 6, If the original problem is man (min ) model, then after applying Rule 6, assign to the remaining constraints in the dual a sense the same as (opposite assign to the remaining constraints in the dual a sense the same as (opposite to ) the corresponding variables in the original problem.to ) the corresponding variables in the original problem.

8. 8. The The iith variable in the dual is unrestricted in sigh if and only if the ith th variable in the dual is unrestricted in sigh if and only if the ith constraint in the original problem is an equality.constraint in the original problem is an equality.

9. 9. If the original problem is max (min) model, then after applying Rule 8, If the original problem is max (min) model, then after applying Rule 8, assign to the remaining variables in the dual a sense opposite to (the same assign to the remaining variables in the dual a sense opposite to (the same as) the corresponding constraints in the original problem.as) the corresponding constraints in the original problem.

Max model Min model xi >= 0 <=> ith constraint>= xi <= 0 <=> ith constraint <= xi free <=> ith constraint = ith const <= <=> yi >= 0 ith const >= <=> yi <= 0 ith const = <=> yi free

Economic InterpretationEconomic Interpretation

1.1. Max Model: the shadow price for the ith constraint = the optimal value Max Model: the shadow price for the ith constraint = the optimal value of the ith variable in the dual. of the ith variable in the dual.

2.2. Min Model: the shadow price for the ith constraint = - the optimal Min Model: the shadow price for the ith constraint = - the optimal value of the ith variable in the dual.value of the ith variable in the dual.

3.3. Suppose Factory 1’s production problem is Suppose Factory 1’s production problem is

MaxMax 3x1 + 5x23x1 + 5x2

s.t. x1 <= 4s.t. x1 <= 4

2x2 <= 122x2 <= 12

3x1 + 2x2 <= 183x1 + 2x2 <= 18

x1, x2 >= 0x1, x2 >= 0

Suppose that the management of Factory 2 decides to BUY the raw Suppose that the management of Factory 2 decides to BUY the raw material material in factory 1. What are ‘fair price’ for the three raw material? in factory 1. What are ‘fair price’ for the three raw material?

Amount Factory 2 pays = 4y1 + 12y2 + 18y3Amount Factory 2 pays = 4y1 + 12y2 + 18y3

Of course, Factory 2 wants to pay as less as possible. Its goal is toOf course, Factory 2 wants to pay as less as possible. Its goal is to

Min Min 4y1 + 12y2 + 18y34y1 + 12y2 + 18y3

Economic Interpretation ( continued ):Economic Interpretation ( continued ):

HoweverHowever, the prices must satisfy Factory 1 such that Factory 1 is willing to sell., the prices must satisfy Factory 1 such that Factory 1 is willing to sell.

Factory 1 sees that if it has 1 unit raw material 1 and 3 units of raw material 3, it Factory 1 sees that if it has 1 unit raw material 1 and 3 units of raw material 3, it

could produce on unit product 1 for a profit $3. Thus, to satisfy Factory 1,could produce on unit product 1 for a profit $3. Thus, to satisfy Factory 1,

Factory 2 will cover the loss of product 1 in Factory 1 due to the sell:Factory 2 will cover the loss of product 1 in Factory 1 due to the sell:

y1 + 3y3 >= 3y1 + 3y3 >= 3

The same is true for Product 2 in Factory 1:The same is true for Product 2 in Factory 1:

2y2 + 2y3 >=52y2 + 2y3 >=5

Finally,Finally, Factory 2 faces the optimal pricing problem Factory 2 faces the optimal pricing problem

min min 4y1 + 124y1 + 12 y2 + 18y3y2 + 18y3

s.t. s.t. y1 + 3y3 >= 3 y1 + 3y3 >= 3

2y2 + 2y3 >= 52y2 + 2y3 >= 5

y1, y2, y3 >= 0y1, y2, y3 >= 0

It provides ‘fair prices’ in the sense of prices that yield the minimum acceptableIt provides ‘fair prices’ in the sense of prices that yield the minimum acceptable

liquidation payment.liquidation payment.

Relations between Primal and Dual

1. The dual of the dual problem is again the primal problem.

2. Either of the two problems has an optimal solution if and only if the other does; if one problem is feasible but unbounded, then the other is infeasible; if one is infeasible, then the other is either infeasible or feasible/unbounded.

3.3. Weak Duality Theorem: The objective function value of the primal (dual) to be maximized evaluated at any primal (dual) feasible solution cannot exceed the dual (primal) objective function value evaluated at a dual (primal) feasible solution.

cTx >= bTy (in the standard equality form)

Relations between Primal and Dual (continued)

4.4. Strong Duality Theorem: When there is an optimal solution, the optimal objective value of the primal is the same as the optimal objective value of the dual.

cTx* = bTy*

5. 5. Complementary Slackness Theorem:Complementary Slackness Theorem: Consider an inequality constraint in any LP problem. If that constraint is inactive for an optimal solution to the problem, the corresponding dual variable will be zero in any optimal solution to the dual of that problem.

x*j (c-ATy*)j = 0, j=1,…,n.

Optimality ConditionsOptimality Conditions

Primal Feasibility:Primal Feasibility:

Dual Feasibility:Dual Feasibility:

Strong Duality:Strong Duality:

or Complementary Slackness:or Complementary Slackness:

,...,n, j y)(c-Ax

y bx c

cy A

x

bA x

jT

j

TT

T

10

0

States of the LP ProblemsStates of the LP Problems

DualDual

PrimalPrimal Have an Opt.Have an Opt.

SolutionSolution

UnboundedUnbounded InfeasibleInfeasible

Have an Opt.Have an Opt.

SolutionSolution

UnboundedUnbounded

InfeasibleInfeasible