1 Simplex Method (created by George Dantzig in late 1940s) A systematic way of searching for an optimal LP solution BMGT 434, Spring 2002 Instructor: Chien-Yu.

1

Simplex Method(created by George Dantzig in late 1940s)

A systematic way of searching for an optimal LP solution

BMGT 434, Spring 2002

Instructor: Chien-Yu Chen

2

Introduction

0 6 12 18 24 30 36 42 48 54 60 66 72 78 84 90 96 102 108 114 1200

4

8

12

16

20

24

28

32

36

40

44

48

52

56

60

64Y

X

1): 1 X + 2 Y = 120

2): 1 X + 1 Y = 90

3): 1 X + 0 Y = 70

4): 0 X + 1 Y = 50

Payoff: 20 X + 10 Y = 1600

Optimal Decisions(X,Y): ( 70, 20)

3

Two-Phase Procedure

• Phase I -- Initial Solution:– Find an initial basic feasible solution by setting

non-basic variables to zero.

• Phase II -- Optimal Solution:– Search for the optimal solution by pivoting

from an extreme point to its adjacent extreme point.

4

Standard Equality Constraint Form

• Astro/Cosmo LP Model:

Max 20A+10C

st A+ 2C<=120

A+ C<= 90

A <= 70

C<= 50

A, C >= 0

• Standard Equality Constraint Form:

Max 20A+10C

st A+ 2C+S1 =120

A+ C +S2 = 90

A +S3 = 70

C +S4= 50

A, C, S1, S2, S3, S4 >= 0

Note: S1, S2, S3 and S4 are slack variables.

5

Definitions of Solutions

• Solution -- A set of values for the variables that satisfy all the “major” constraints (ignoring the nonnegativity constraints).– e.g. (70, 50, -50, -30, 0, 0).

• Feasible Solution -- Any solution that are nonnegative as well (i.e. satisfying all the major and nonnegativity constraints).– e.g. (70, 0, 50, 20, 0, 50).

• Basic Solution -- A solution that consists of (n - m) zero-value non-basic variables and m basic variables from solving the remaining m-by-m subsystem, for an m-equality-constraint n-variable linear system.– e.g. (0, 0, 120, 90, 70, 50) and (70, 50, -50, -30, 0, 0).

6

Definitions of Solutions (cont.)

• Basic Feasible Solution -- A basic solution that is also feasible (i.e. satisfying all the major and nonnegativity constraints).

– e.g. (0, 0, 120, 90, 70, 50) and (70, 0, 50, 20, 0, 50).

– A basic feasible solution (an algebraic concept) actually represents an extreme/corner point of the feasible region (a geometric concept).

– On the other hand, an extreme/corner point may associate with more than one basic feasible solution, because of degeneracy.

7

Example• 13 Basic Solutions

– C(6,2) = 6!/(4!*2!) = 15.– A=0 S3=0 (impossible).– C=0 S4=0 (impossible).

• 6 Basic Feasible SolutionsI : (0, 0, 120, 90, 70, 50),II : (0, 50, 20, 40, 20, 0),III : (20, 50, 0, 20, 50, 0),IV: (60, 30, 0, 0, 10, 20),V : (70, 20, 10, 0, 0, 30), andVI: (70, 0, 50, 20, 0, 50).

0 70 90 120

0

50

60Y

X

1): 1 X + 2 Y = 120

2): 1 X + 1 Y = 90

3): 1 X + 0 Y = 70

4): 0 X + 1 Y = 50

I

II III

IV

V

VI

8

Degenerate v.s. Nondegenerate(with respect to a basic feasible solution)

(Assuming an m×n linear equation system)

• Degenerate -- Some basic variables have zero values; i.e., the number of positive variables is less than m.

• Nondegenerate -- All basic variables are positive; i.e., the number of positive variables is exactly m.

9

ExampleST X+Y<=10

X <= 5 Y<= 5X, Y >= 0

Nonbasic Variables Basic Variables Extreme PointX=0 & Y=0 S1=10, S2=5 & S3=5 CY=0 & S2=0 X=5, S1=5 & S3=5 BS1=0 & S2=0 X=5, Y=5 & S3=0 AS1=0 & S3=0 X=5, Y=5 & S2=0 AS2=0 & S3=0 X=5, Y=5 & S1=0 AX=0 & S3=0 Y=5, S2=5 & S1=5 D

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

Y

X

1): X + Y = 10 (i.e. S1 = 0)

2): X = 5 (i.e. S2=0)

3): Y = 5 (i.e. S3 = 0)

C

D

A

B

10

Adjacent Extreme Points

• Two extreme points are adjacent if their bases (i.e. sets of basic variables) are different by one variable only.

• Example:

Corner N.B. Basic VariablesI A, C S1, S2, S3, S4II A, S4 S1, S2, S3, CIII S1, S4 A, S2, S3, CIV S1, S2 A, S4, S3, CV S3, S2 A, S4, S1, CVI S3, C A, S4, S1, S2

0 70 90 1200

50

60Y

X

1): 1 X + 2 Y = 120

2): 1 X + 1 Y = 90

3): 1 X + 0 Y = 70

4): 0 X + 1 Y = 50

I

II III

IV

V

VI

11

Transformed Equations(Gauss-Jordan Elimination)

• Each basic variable appears in one and only one equation, and, in that equation, its coefficient is +1.

• Only constant terms are on the right hand side of the equations, and these constant terms are nonnegative.

• Each set of transformed equations represents a basic feasible solution (extreme point).

• Example:(Corner I) (Corner II)A+2C+S1 = 120 A +S1 -2S4 = 20A+ C +S2 = 90 A +S2 - S4 = 40A +S3 = 70 A +S3 = 70

C +S4 = 50 C + S4 = 50

12

Phase I: Obtaining an Initial Solution

• Any set of transformed equations is directly related to an extreme corner point (i.e. a basic feasible solution); therefore, it can be used as an initial solution.

• Thus, Phase I is designed to find a set of transformed equations corresponding to any initial corner of the constraint set.

13

Example[1] X1+ X2+ X3 = 5[2] X1+3X2+5X3 = 9

[3] = -1*[1]+[2] 2X2+4X3 = 4[4] = [3]/2 1X2+2X3 = 2[5] = -1*[4]+[1] 1X1 - X3 = 3

Transformed Equations:[5] X1 - X3 = 3[4] X2+2X3 = 2

Initial Solution:Basic: X1=3, X2=2 Nonbasic: X3=0

14

Phase II: Improving the Solution

• We could improve the solution by moving (pivoting) from current extreme point (basic feasible solution) to an adjacent extreme point (basic feasible solution) with better O.F. value.

• The basic feasible solution of an adjacent extreme point can be obtained by introducing a nonbasic variable (enter variable) into current basis to replace a basic variables (exit variable).

15

Phase II: Improving the Solution (cont.)

• Example -- Rewrite the Transformed Equations as

A+2C+S1 = 120 S1 = 120 - A - 2CA+ C +S2 = 90 S2 = 90 - A - CA +S3 = 70 S3 = 70 - A C +S4 = 50 S4 = 50 - C

• Iterate the pivoting process until there is no adjacent extreme point whose O.F. value is better current extreme point’s; so, the current basic feasible solution is optimal.

16

Effects on O.F. Value(regarding to 1 unit increase of a non-basic variable)

• Indirect Effect (Potential Loss)– Caused by the induced changes in the current basic variables.

• Direct Effect (Potential Gain)– Caused by the change in the current nonbasic variables.

• Opportunity Cost (Net Potential Gain)– The opportunity cost of NOT having the 1-unit increase of

the nonbasic variable.

17

Effects on O.F. Value (cont.)

• Current Extreme Point– (A, C, S1, S2, S3, S4) = (0, 0, 120, 90, 70, 50)– Basic variables: S1, S2, S3, and S4– O.F. = 20A + 10C + 0S1 + 0S2 + 0S3 + 0S4– Basic coefficients = (0, 0, 0, 0)– Nonbasic columns:

» A column = (1, 1, 1, 0)T

» C column = (2, 1, 0, 1)T

• Transformed EquationsA+2C+S1 = 120 S1 = 120 - A - 2CA+ C +S2 = 90 S2 = 90 - A - CA +S3 = 70 S3 = 70 - A C +S4 = 50 S4 = 50 - C

18

Indirect Effect (Potential Loss)• Caused by the induced changes in the current basic

variables (due to the 1-unit increase of the non-basic variable).• Zj = Indirect decrease in O.F. value = [Basic coefficients] • [A column] = “Contribution” due only to basic variable change.• (e.g.) = (0, 0, 0, 0) • (1, 1, 1, 0)T

= 0·(1) + 0·(1) + 0·(1) + 0·(0) = 0. (for nonbasic variable A)

19

Direct Effect (Potential Gain)

• Caused by the change in the current nonbasic variables.• Cj = Direct increase in O.F. value

= Contribution due to the 1-unit increase

of the nonbasic variable

= Cost coefficient of the nonbasic

variable• (e.g.) = 20 (for nonbasic variable A)

20

Opportunity Cost (Net Potential Gain)

• The opportunity cost of NOT having the 1-unit increase of the nonbasic variable.

• Cj - Zj = Opportunity cost

= Potential gain - Potential loss

= Direct effect - Indirect effect

• (e.g.) = 20 - 0

= 20 (for nonbasic variable A)

21

Enter and Exit Rules

• Enter Rule -- Set the nonbasic variable with the most positive opportunity cost Cj-Zj to be the enter variable to the current basis (assuming maximization problems).

• Exit Rule -- Set the basic variable that strictly confines the increase of the enter variable to be the exit variable from the current basis. (Note: The purpose is to keep the new basic solution feasible.)

22

Example• Enter Rule

– (Cj-Zj)A=20 & (Cj-Zj)C=10.– A is the enter variable.

• Exit Rule -- assuming C is the enter variable for demo– S1 = 120 -A -2C >= 0 (i.e. C <= 120/2 = 60)

S2 = 90 -A - C >= 0 (i.e. C <= 90/1 = 90)

S3 = 70 -A >= 0 (i.e. C <= 70/0 = )S4 = 50 - C >= 0 (i.e. C <= 50/1 = 50) Min

– C <= 50 (corresponding to S4).– S4 is the exit variable.

23

Simplex Tableau• A simplex tableau is a table-style representation of the

corresponding transformed equations.• There is a simplex tableau corresponding to each basic

feasible solution (extreme point).• A simplex tableau also contains the opportunity cost

calculations.• A simplex-tableau transformation brings us to another

simplex tableau that represents an adjacent extreme point (basic feasible solution).

24

Simplex Tableau -- SetupMax 20A + 10C

st A + 2C + S1 = 120

A + C + S2 = 90

A + S3 = 70

C + S4 = 50

A, C, S1, S2, S3, S4 >= 0

Basic Basic NB NB B B B BCoefficient Variable 20 10 0 0 0 0 Value Ratio

C B A C S1 S2 S3 S4 b0 S1 1 2 1 0 0 0 1200 S2 1 1 0 1 0 0 900 S3 1 0 0 0 1 0 700 S4 0 1 0 0 0 1 50

Zj 0 O.V.Cj-Zj

25

Simplex Tableau -- Setup (cont.)

Basic Basic NB NB B B B BCoefficient Variable 20 10 0 0 0 0 Value Ratio

C B A C S1 S2 S3 S4 b0 S1 1 2 1 0 0 0 120 120/10 S2 1 1 0 1 0 0 90 90/10 S3 1 0 0 0 1 0 70 70/1 Min0 S4 0 1 0 0 0 1 50 -

Zj 0 0 0 0 0 0 0 O.V.Cj-Zj 20 10 0 0 0 0

Max

• Zj = [Basic Coefficient] [jth Column]

• Cj - Zj = (jth Cost Coefficient) - Zj

• O.V. = [Basic Coefficient] [b Column]• Ratio = [b Column] / [Enter-Variable Column] (ratio of corresponding elements!)

26

Simplex Tableau -- Transformation20 10 0 0 0 0

C B A C S1 S2 S3 S4 b Ratio0 S1 1 2 1 0 0 0 1200 S2 1 1 0 1 0 0 90

20 A 1 0 0 0 1 0 700 S4 0 1 0 0 0 1 50

Zj O.V.Cj-Zj

20 10 0 0 0 0C B A C S1 S2 S3 S4 b Ratio0 S1 0 2 1 0 -1 0 500 S2 0 1 0 1 -1 0 20

20 A 1 0 0 0 1 0 700 S4 0 1 0 0 0 1 50

Zj O.V.Cj-Zj

27

Simplex Tableau -- Transformation20 10 0 0 0 0

C B A C S1 S2 S3 S4 b Ratio0 S1 0 2 1 0 -1 0 50 50/20 S2 0 1 0 1 -1 0 20 20/1 Min

20 A 1 0 0 0 1 0 70 -0 S4 0 1 0 0 0 1 50 50/1

Zj 20 0 0 0 20 0 1400 O.V.Cj-Zj 0 10 0 0 -20 0

Max

20 10 0 0 0 0C B A C S1 S2 S3 S4 b Ratio0 S1 0 0 1 -2 1 0 10

10 C 0 1 0 1 -1 0 2020 A 1 0 0 0 1 0 700 S4 0 0 0 -1 1 1 30

Zj 20 10 0 10 10 0 1600 O.V.Cj-Zj 0 0 0 -10 -10 0

28

Optimality Condition

• For a maximization problem, if all opportunity costs are nonpositive (i.e. Cj-Zj<=0 for all j’s) , an optimal extreme point (basic feasible solution) has been reached.

20 10 0 0 0 0C B A C S1 S2 S3 S4 b Ratio0 S1 0 0 1 -2 1 0 10

10 C 0 1 0 1 -1 0 2020 A 1 0 0 0 1 0 700 S4 0 0 0 -1 1 1 30

Zj 20 10 0 10 10 0 1600 O.V.Cj-Zj 0 0 0 -10 -10 0 Optimal

29

Artificial Variables• Motivation -- No straight-forward initial solution

after adding slacks/surpluses.• Example

3X1 + 4X2 >= 6 3X1 + 4X2 - S1 = 62X1 - 6X2 <= 4 2X1 - 6X2 + S2 = 4Nonbasic variables: X1=X2=0Basic variables: S1 = -6, S2 = 4 (infeasible!!)

• Artificial Variable -- with very negative cost coefficient MAX ………………… -mA1 (Note: m is a very large number) ST 3X1 + 4X2 - S1 + A1 = 6 2X1 - 6X2 + S2 = 4

30

ExampleMax 12X1 + 20 X2 + 8X3 12X1 + 20 X2 + 8X3 s.t. 3X1 - 2X3 >= -2 3X1 - 2X3 - S1 = -2 -4X1 - X2 + 12X3 >= 4 -4X1 - X2 + 12X3 - S2 = 4 - X1 + 3X3 <= -6 - X1 + 3X3 + S3 = -6 3X1 + 4X2 - 6X3 = -12 3X1 + 4X2 - 6X3 = -12 X2 + 9X3 = 31 X2 + 9X3 = 31 X1>=0 X2>=0 X3>=0

12X1 + 20 X2 + 8X3 12X1 + 20 X2 + 8X3 -mA1-mA2-mA3-mA4-3X1 + 2X3 + S1 = 2 -3X1 + 2X3 + S1 = 2 -4X1 - X2 + 12X3 - S2 = 4 -4X1 - X2 + 12X3 - S2 +A1 = 4 X1 - 3X3 - S3 = 6 X1 - 3X3 - S3 +A2 = 6 -3X1 - 4X2 + 6X3 = 12 -3X1 - 4X2 + 6X3 +A3 = 12 X2 + 9X3 = 31 X2 + 9X3 +A4 = 31

31

Possible Outcomes

• Optimal -- All opportunity costs are nonpositive.

• Infeasible -- One or more artificial variables remain in the solution when optimality conditions hold.

• Unbounded -- There exists a column with a positive opportunity cost, whose entries in that column are all nonpositive.

• Alternative Optima -- One of the nonbasic column has opportunity cost Cj-Zj=0 in the optimal tableau. (Assuming the optimal solution is nondegenerate.)

32

Treatments for Minimization

• Change O.F. from Min f(x) to Max -f(x).or• Switch to the following rules:

– Use +M for artificial coefficients.– Set the nonbasic variable with the most negative

opportunity cost to be the enter variable.– Optimality conditions = all opportunity costs are

nonnegative.