LINEAR PROGRAMMING. “ A certain wide class of practical problems appears to be just beyond the range of modern computing machinery. These problems occur.

LINEAR PROGRAMMING

“A certain wide class of practical problems appears to be just beyond the range of modern computing machinery. These problems occur in everyday life; they run the gamut from some very simple situations that confront an individual to those connected with the national economy as a whole. Typically, these problems involve a complex of different activities in which one wishes to know which activities to emphasize in order to carry out desired objectives under known limitations.”

George B. Dantzig, 1948

LP History• George Dantzig, 1947

– Introduced LP and recognized it as more than a conceptual tool: Computing answers important.

– Invented “primal” simplex algorithm.– First LP solved: Laderman, 9 cons., 77 vars., 120 MAN-DAYS.

• First computer code – 1951• LP used commercially – Early 60s• Powerful mainframe codes introduced – Early 70s• Computational progress stagnated – Mid 80s• Remarkable progress last 15 years (PCs, new computer

science and mathematics)

Diet Problem*

Bob wants to plan a nutritious diet, but he is on a limited budget, so he wants to spend as little money as possible. His nutritional requirements are as follows:

1. 2000 kcal

2. 55 g protein

3. 800 mg calcium

* From Linear Programming, by Vaŝek Chvátal

Nutritional values

Diet Problem

Bob is considering the following foods:

Food Serving Size Energy (kcal) Protein (g) Calcium (mg) Price per serving

Oatmeal 28 g 110 4 2 $0.30Chicken 100 g 205 32 12 $2.40Eggs 2 large 160 13 54 $1.30Whole milk 237 cc 160 8 285 $0.90Cherry pie 170 g 420 4 22 $0.20Pork and beans 260 g 260 14 80 $1.90

Variables

Diet Problem

We can represent the number of servings of each type of food in the diet by the variables:

x1 servings of oatmeal

x2 servings of chicken

x3 servings of eggs

x4 servings of milk

x5 servings of cherry pie

x6 servings of pork and beans

Nutritional values

Diet Problem

Bob is considering the following foods:

Food Serving Size Energy (kcal) Protein (g) Calcium (mg) Price per serving

Oatmeal 28 g 110 4 2 $0.30Chicken 100 g 205 32 12 $2.40Eggs 2 large 160 13 54 $1.30Whole milk 237 cc 160 8 285 $0.90Cherry pie 170 g 420 4 22 $2.00Pork and beans 260 g 260 14 80 $1.90

x1

x2

x3

x4

x5

x6

KCAL constraint: 110x1 + 205x2 + 160x3 + 160x4 + 420x5 + 260x6 2000 (110x1 = kcals in oatmeal)

LP formulation

Diet Problem

110x1 + 205x2 + 160x3 + 160x4 + 420x5 + 260x6 2000

4x1 + 32x2 + 13x3 + 8x4 + 4x5 + 14x6 55

2x1 + 12x2 + 54x3 + 285x4 + 22x5 + 80x6 800

Minimize

subject to:

0,,,,, 654321 xxxxxx

Cost

Nutritional requirements

Bounds

0.3x1 + 2.40x2 + 1.30x3 + 0.90x4 + 2.0x5 + 1.9x6

Solution

Diet Problem

When we solve the preceding LP (using CPLEX, of course) we get a solution value of $6.71, which is achieved with the following menu:

14.24 servings of oatmeal

0 servings of chicken

0 servings of eggs

2.71 servings of milk

0 servings of cherry pie

0 servings of pork and beans

A linear program (LP) is an optimization problem of the form

uxl

bAxtoSubject

xcMinimize T

Solving Linear Programs

x

y

(0.882,0.706)

Objective =

0.9 x + 0.73 y =

40 million

Objective =

30 million

Feasible Solutions

Objective = 0.9 * 0.882 + 0.73 * 0.706 = 13.1 million

(0,6)

(1,0)

Constraint 1

(0,1)

(3,0)

Constraint 2

(0,1.5)

(2,0)

Constraint 3

A Graphical Solution

(0,0)

Maximize 0.90 x + 0.73 y (OBJECTIVE)Subject To Constraint 1: 0.42 x + 0.07 y <= 4200000 Constraint 2: 0.13 x + 0.39 y <= 3900000 Constraint 3: 0.35 x + 0.44 y <= 7000000 x >= 0 y >= 0

A graphical representation

The Simplex Algorithm

We now look at a graphical representation of the simplex method as it solves the following problem:

Maximize 3x1 + 2x2 + 2x3

Subject to x1 + x3 8

x1 + x2 7

x1 + 2x2 12

x1, x2, x3 0

The Simplex Algorithm

x1

x2

x3

(0,0,8) (0,6,8)

(2,5,6)

(0,6,0)

(2,5,0)(7,0,1)

(7,0,0)

Maximize z = 3x1 + 2x2 + 2x3

z = 0

z = 21

z = 23

Optimal!

z = 28

Log started (V8.2.0a1) Tue Apr 8 21:59:46 2003

Problem name: supply_chain.lp.gzConstraints : 10810259 Variables : 19916187

Reduced LP has 3140439 rows, and 7314332 variables.Presolve time = 339.43 sec.

Iteration log . . .Iteration: 0 Infeasibility = 1534390803.049594Iteration: 1247301 Infeasibility = 437397607.938387

...

Iteration: 2348954 Infeasibility = 10965.599999Elapsed time = 3776.48 sec. (2349000 iterations)Iteration: 2349222 Objective = 1775629743606097400.000000

...

Elapsed time = 5396.65 sec. (2687000 iterations)Iteration: 2687616 Objective = 1403833763253068800.000000Removing shift (10).

Primal simplex - Optimal: Objective = 1.4038337633e+18Solution time = 5410.00 sec. Iterations = 2687616 (2348954)

A Real LP: Sample Run

MIXED INTEGERPROGRAMMING

Mixed-Integer Programming (MIP)

(= linear programming + restriction that some [MIP] or all variables must take on integer values.)

Manufacturing Cereal

Cereal Problem:With setup cost

Minimize Total_Cost: 283 x1 + 266 x2 + 187 x3 + 188 x4 + 182 x5 + 181 x6 + 200 y1 + 250 y2 + 400 y3 + 350 y4 + 800 y5 + 1000 y6Subject To Boxes: x1 + x2 + x3 + x4 + x5 + x6 = 200 S1: - 50 y1 + x1 <= 0 S2: - 70 y2 + x2 <= 0 S3: - 40 y3 + x3 <= 0 S4: - 70 y4 + x4 <= 0 S5: - 100 y5 + x5 <= 0 S6: - 90 y6 + x6 <= 0Bounds x1, x2, x3, x4, x5, x6 >= 0 y1, y2, y3, y4, y5, y5 = 0 or 1 (binary)End

Cereal Problem:With setup cost – Solution log

Problem 'cereal.lp' read. Nodes Cuts/ Node Left Objective IInf Best Integer Best Node ItCnt 0 0 38220.0000 1 38220.0000 6* 0+ 0 0 38520.0000 38220.0000 6 38410.0000 2 38520.0000 Cuts: 4 8 38495.7143 2 38520.0000 Cuts: 4 12* 0+ 0 0 38510.0000 38495.7143 12 cutoff 38510.0000 Flowcuts: 2 13

Cover cuts applied: 1Flow cuts applied: 5Gomory fractional cuts applied: 2

Integer optimal solution: Objective = 3.8510000000e+004Solution time = 0.01 sec. Iterations = 13 Nodes = 0

Variable Name Solution Valuex3 40.000000x4 60.000000x5 100.000000y3 1.000000y4 1.000000y5 1.000000All other variables in the range 1-12 are zero.

objective

Linear Program

LP optimum

feasible solutions

y

x

objective

IP optimum

Integer Program

rounding down optimum of LP relaxation

feasible solutions =

y

x

optimum ofLP relaxation

Solving MIPs: Branch and Bound

Consider the following integer program:

Maximize x + y + 2z

Subject to 7x + 2y + 3z 36

5x + 4y + 7z 42

2x + 3y + 5z 28

x, y, z 0, integer

(IP0)

IP011

511obj

11

15

011

31

z

y

x

IP2 IP1

1x 2x

10obj

402

Best IP value

Best IP solution

4.11obj

2.501

402

10/

Integral

IP4 IP3

5z 6z

Infeasible

3

111obj

53

11

11obj

501

0y 1y

IP6 IP5

/ 11

501

Integral

2.11obj

6.411

IP8 IP7

4z 55 zz

9obj

421

Obj. incumbent

11obj

510

Obj. incumbent

Branch & Bound: Example

The Pill Salesman

Diet Problem

A pill salesman offers Bob energy, protein, and calcium pills to fulfill his nutritional needs. We will represent the costs of each of the pills as follows:

y1 cost (in dollars) of 1 kcal energy pill

y2 cost (in dollars) of 1 g protein pill

y3 cost (in dollars) of 1mg calcium pill

LP constraints

Diet Problem

110x1 + 205x2 + 160x3 + 160x4 + 420x5 + 260x6 2000 y1

4x1 + 32x2 + 13x3 + 8x4 + 4x5 + 14x6 55 y2

2x1 + 12x2 + 54x3 + 285x4 + 22x5 + 80x6 800 y3

Nutritional requirements

LP constraints

Diet Problem

110x1 + 205x2 + 160x3 + 160x4 + 420x5 + 260x6 2000 y1 kcal

4x1 + 32x2 + 13x3 + 8x4 + 4x5 + 14x6 55 y2 protein

2x1 + 12x2 + 54x3 + 285x4 + 22x5 + 80x6 800 y3 calcium

Nutritional requirements

x1 = servings of oatmeal: The cost of the nutrients in one serving of oatmeal shouldn’t exceed the cost of just buying one serving of oatmeal:110y1 + 4y2 + 2y3 0.3 (4 y2 = cost of protein in oatmeal)

Minimize Cost

0.3x1 + 2.40x2 + 1.30x3 + 0.90x4 + 2.0x5 + 1.9x6

The salesman’s requirements

Diet Problem

The pill salesman wants to make as much money as possible, given Bob’s constraints. He knows Bob wants 2000 kcal, 55g protein, and 800 mg calcium, so his problem is as follows:

Maximize 2000y1 + 55y2 + 800y3

Subject to 110y1 + 4y2 + 2y3 0.3

205y1 + 32y2 + 12y3 2.4

160y1 + 13y2 + 54y3 1.3

160y1 + 8y2 + 285y3 0.9

420y1 + 4y2 + 22y3 2.0

260y1 + 14y2 + 80y3 1.9

y1, y2, y3 0

Solution

Diet Problem

Solving this LP gives us the following pill prices:

$0.27 for 1 kcal energy pill

$0.00 for 1 g protein pill

$0.16 for 1mg calcium pill

Total cost = 0.27 (2000) + 0.16 (800) = $6.71

THE SAME AS THE LOWEST COST DIET!

Diet Problem

Weak Duality:

Cost of any feas. diet Price of any feas. pills

Strong Duality: (Von Neumann, 1947)

Optimal cost of diet = Optimal cost of pills

Diet Problem

MORAL:

The values of the dual variables, which are

always available as part of LP solution, equal

the marginal value of corresponding

resources.

The Simplex Algorithm

Consider the following linear program:

Maximize 5x1 + 4x2 + 3x3

Subject to 2x1 + 3x2 + x3 5

4x1 + x2 + 2x3 11

3x1 + 4x2 + 2x3 8

x1, x2, x3 0.

Slack Variables

The Simplex Algorithm

We add slack variables to each constraint to get equations:

Maximize 5x1 + 4x2 + 3x3

Subject to 2x1 + 3x2 + x3 + x4 = 5

4x1 + x2 + 2x3 + x5 = 11

3x1 + 4x2 + 2x3 + x6 = 8

x1, x2, x3, x4, x5, x6 0.

Getting started

The Simplex Algorithm

Rewriting we can express the slack variables and the objective, denoted z, in terms of the original variables. Setting x1=x2=x3=0, we easily read off the solution x4=5, x5=11, x6=8, z=0.

z = 5x1 + 4x2 + 3x3

x4 = 5 - 2x1 - 3x2 - x3

x5 = 11 - 4x1 - x2 - 2x3

x6 = 8 - 3x1 - 4x2 - 2x3

1st Iteration

The Simplex Algorithm

We want to increase the value of z from its current value of 0. Clearly, increasing the value of x1 will increase the value of z. By how much can we increase it?

z = 5x1 + 4x2 + 3x3

x4 = 5 - 2x1 - 3x2 - x3

x5 = 11 - 4x1 - x2 - 2x3

x6 = 8 - 3x1 - 4x2 - 2x3

x4 0 5 – 2x1 0 5 2x1 x1 5/2

x5 0 11 – 4x1 0 11 4x1 x1 11/4

x6 0 8 – 3x1 0 8 3x1 x1 8/3

2nd Iteration

The Simplex Algorithm

Now we see that increasing the value of x3 will improve the value of z.

z = 25/2 - 7/2x2 + 1/2x3 - 1/2x4

x1 = 5/2 - 3/2x2 - 1/2x3 - 1/2x4

x5 = 1 + 5x2 + 2x4

x6 = 1/2 + 1/2x2 - 1/2x3 + 3/2x4

x1 0 5/2 – 1/2x3 0 5/2 1/2x3 x3 5

x5 0 x3 unbounded

x6 0 1/2 – 1/2x3 0 1/2 1/2x3 x1 1

The Simplex Algorithm

z = 13 - 3x2 - x4 - x6

x1 = 2 - 2x2 - 2x4 + x6

x5 = 1 + 5x2 + 2x4

x3 = 1 + x2 + 3x4 - 2x6

Optimal!!!

Barrier Algorithms

Simplex solution path

Barrier central path

o Predictor

o Corrector

Optimum