Operations Research 51 Dr. Emad Elbeltagi CHAPTER 3 THE SIMPLEX METHOD In this chapter, Students will be introduced to solve linear programming models using the simplex method. This will give them insights into what commercial linear programming software packages actually do. Such an understanding can be useful in several ways. For example, students will be able to identify when a problem has alternate optimal solutions, unbounded solution, etc. 3.1 Gauss-Jordan Elimination for Solving Linear Equations The Gauss-Jordan elimination procedure is a systematic method for solving systems of linear equations. It works one variable at a time, eliminating it in all rows but one, and then moves on to the next variable. We illustrate the procedure on three examples. Example 3.1 x 1 + 2x 2 + x 3 = 4 (1) 2x 1 - x 2 + 3x 3 = 3 (2) x 1 + x 2 - x 3 = 3 (3) In the first step of the procedure, we use the first equation to eliminate x 1 from the other two. Specifically, in order to eliminate x 1 from the second equation, we multiply the first equation by 2 and subtract the result from the second equation. Similarly, to eliminate x 1 from the third equation, we subtract the first equation from the third. Such steps are called elementary row operations. We keep the first equation and the modified second and third equations. The resulting equations are: x 1 + 2x 2 + x 3 = 4 (1) - 5x 2 + x 3 = -5 (2) - x 2 - 2x 3 = -1 (3)
40
Embed
CHAPTER 3 THE SIMPLEX METHOD - Mansosp.mans.edu.eg/elbeltagi/ENG345 Ch3.pdf · Let’s recall the Example of Section 2.3 of the previous chapter. The graph model of that example is
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Operations Research 51 Dr. Emad Elbeltagi
CHAPTER 3
THE SIMPLEX METHOD
In this chapter, Students will be introduced to solve linear programming models using the
simplex method. This will give them insights into what commercial linear programming
software packages actually do. Such an understanding can be useful in several ways. For
example, students will be able to identify when a problem has alternate optimal solutions,
unbounded solution, etc.
3.1 Gauss-Jordan Elimination for Solving Linear Equations
The Gauss-Jordan elimination procedure is a systematic method for solving systems of
linear equations. It works one variable at a time, eliminating it in all rows but one, and
then moves on to the next variable. We illustrate the procedure on three examples.
Example 3.1
x1 + 2x2 + x3 = 4 (1)
2x1 - x2 + 3x3 = 3 (2)
x1 + x2 - x3 = 3 (3)
In the first step of the procedure, we use the first equation to eliminate x1 from the other
two. Specifically, in order to eliminate x1 from the second equation, we multiply the first
equation by 2 and subtract the result from the second equation. Similarly, to eliminate x1
from the third equation, we subtract the first equation from the third. Such steps are
called elementary row operations. We keep the first equation and the modified second
and third equations. The resulting equations are:
x1 + 2x2 + x3 = 4 (1)
- 5x2 + x3 = -5 (2)
- x2 - 2x3 = -1 (3)
Operations Research 52 Dr. Emad Elbeltagi
Note that only one equation was used to eliminate x1 in all the others. This guarantees
that the new system of equations has exactly the same solution(s) as the original one. In
the second step of the procedure, we divide the second equation by -5 to make the
coefficient of x2 equal to 1. Then, we use this equation to eliminate x2 from equations 1
and 3. This yields the following new system of equations:
x1 + 7/5x3 = 2 (1)
x2 – 1/5x3 = 1 (2)
- 11/5x3 = 0 (3)
Once again, only one equation was used to eliminate x2 in all the others and that
guarantees that the new system has the same solution(s) as the original one. Finally, in
the last step of the procedure, we use equation 3 to eliminate x3 in equations 1 and 2.
x1 = 2 (1)
x2 = 1 (2)
x3 = 0 (3)
So, there is a unique solution. Note that, throughout the procedure, we were careful to
keep three equations that have the same solution(s) as the original three equations. Why
is it useful? Because, linear systems of equations do not always have a unique solution
and it is important to identify such situations.
Example 3.2
x1 + 2x2 + x3 = 4 (1)
x1 + x2 + 2x3 = 1 (2)
2x1 + 3x2 + 3x3= 2 (3)
First we eliminate x1 from equations 2 and 3.
x1 + 2x2 + x3 = 4 (1)
- x2 + x3 = -3 (2)
- x2 + x3 = -6 (3)
Then we eliminate x2 from equations 1 and 3.
Operations Research 53 Dr. Emad Elbeltagi
x1 + 3x3 = -2 (1)
x2 - x3 = 3 (2)
0 = -3 (3)
Equation 3 shows that the linear system has no solution.
Example 3.3
x1 + 2x2 + x3 = 4 (1)
x1 + x2 + 2x3 = 1 (2)
2x1 + 3x2 + 3x3= 5 (3)
Doing the same as above, we end up with
x1 + 3x3 = -2 (1)
x2 - x3 = 3 (2)
0 = 0 (3)
Now equation 3 is an obvious equality. It can be discarded to obtain
x1 = -2 - 3x3 (1)
x2 = 3 + x3 (2)
The situation where we can express some of the variables (here x1 and x2) in terms of the
remaining variables (here x3) is important. These variables are said to be basic and non-
basic respectively. Any choice of the non-basic variable x3 yields a solution of the linear
system. Therefore the system has infinitely many solutions.
It is generally true that a system of m linear equations in n variables has either:
(a) No solution,
(b) A unique solution,
(c) Infinitely many solutions.
Operations Research 54 Dr. Emad Elbeltagi
The Gauss-Jordan elimination procedure solves the system of linear equations using two
elementary row operations:
- Modify some equation by multiplying it by a nonzero scalar (a scalar is an actual
real number, such as ½ or -2; it cannot be one of the variables in the problem),
- Modify some equation by adding to it a scalar multiplied by another equation.
The resulting system of m linear equations has the same solution(s) as the original
system. If an equation 0 = 0 is produced, it is discarded and the procedure is continued. If
an equation 0 = a is produced where a is a nonzero scalar, the procedure is stopped: in
this case, the system has no solution. At each step of the procedure, a new variable is
made basic: it has coefficient 1 in one of the equations and 0 in all the others.
The procedure stops when each equation has a basic variable associated with it. Say p
equations remain (remember that some of the original m equations may have been
discarded). When m = p, the system has a unique solution. When m > p, then p variables
are basic and the remaining m - p are non-basic. In this case, the system has infinitely
many solutions.
3.2 The Essence of the Simplex Method
Let’s recall the Example of Section 2.3 of the previous chapter. The graph model of that
example is sown in Fig. 3.1. The Five constraints boundaries and their points of
intersection are highlighted in the figure. The points of intersection are the corner-point
solutions of the problem. The four corner points are (0, 0), (0, 7.5), (1, 7), and (6, 0), are
the corner-point feasible solutions (CPF solutions). The other five corner points, (10, 0),
(15, 0), (0, 8), (0, 10), and (5, 5,), are called corner-point infeasible solutions.
Properties of the CPF solutions
- If there is exactly one optimal solution, then it must be a CPF solution.
- If there are multiple optimal solutions, then at least two must be adjacent CPF
feasible solutions.
Operations Research 55 Dr. Emad Elbeltagi
- There are only a finite number of CPF solutions.
- If a CPF solution has no adjacent CPF solution that are better as measured by the
objective function, then there are no better CPF solutions anywhere; i.e., it is
optimal.
In this example, each corner-point solution lies at the intersection of two constraint
boundaries (Table 3.1). For any linear programming problem with n decision variables,
two CPF solutions are adjacent to each other if they share n-1 constraint boundaries. In
the current example since n=2, then two of its CPF solutions are adjacent if they share
one constraint boundary. For example, (0, 0) and (6, 0) are adjacent because they share
the x2=0 constraint boundary. The feasible region of Fig. 3.1 has four edges and each
CPF solution has two adjacent two CPF solutions.
Table 3.1: CPF solutions
CPF Solution Its adjacent CPF solution
(0, 0)
(0, 7.5)
(1, 7)
(6, 0)
(0, 7.5) and (6, 0)
(1, 7) and (0, 0)
(6, 0), and (0, 7.5)
(0, 0) and (1, 7)
Figure 3.1: Feasible region of the example of section 2.3
(1, 7)
(0, 0) (6, 0)
(0, 7.5)
Operations Research 56 Dr. Emad Elbeltagi
Thus, in any linear programming problem that possesses at least one optimal solution, if a
CPF solution has no adjacent CPF solutions that are better (as measured by the objective
function), then it must be an optimal solution. The point (1, 7) is an optimal solution
because its objective equals 7,750,000 is larger than 7,500,000 for (0, 7.5) and 6,000,000
for (6, 0). This is the test that used by the simplex method to determine when an optimal
solution has been reached. The general structure of the simplex method is as follow:
Initialization (Set up to start iterations, finding an initial solution)
Optimality test (Is the current CPF solution optimal?)
Yes (Stop)
No
Iteration
3.3 Setting Up the Simplex Method
Before we start discussing the simplex method, we point out that every linear program
can be converted into “standard" form:
Max c1x1 + c2x2 + ….. + cnxn
subject to a11x1 + a12x2 + ….. + a1nxn = b1
………………….
am1x1 + am2x2 + ….. + amnxn = bm
x1 ≥ 0; …… xn ≥ 0
Where the objective is maximized, the constraints are equalities and the variables are all
nonnegative. This is done as follows:
- If the problem is min z, convert it to max -z.
- If a constraint is ai1x1 + ai2x2 + …. + ainxn ≤ bi, convert it into an equality
constraint by adding a nonnegative slack variable si. The resulting constraint is
ai1x1 + ai2x2 + …. + ainxn + si = bi, where si ≥ 0.
Operations Research 57 Dr. Emad Elbeltagi
- If a constraint is ai1x1 + ai2x2 + …. + ainxn ≥ bi, convert it into an equality
constraint by subtracting a nonnegative surplus variable si. The resulting
constraint is ai1x1 + ai2x2 + …. + ainxn - si = bi, where si ≥ 0.
- If some variable xj is unrestricted in sign, replace it everywhere in the formulation
by x’j –x’’j where x’j ≥ 0 and x’’ j ≥ 0.
- The right side of an equation can always be made nonnegative by multiplying
both sides by -1. For example, -2x1 + 3x2= -5 is mathematically equivalent to
+2x1 - 3x2= + 5.
- The direction of an inequality is reversed when both sides are multiplied by -1.
For example whereas 2 < 4, -2 > -4. Thus the inequality 2x1 – x2 ≤ -5 can be
replaced by -2x1 + x2 ≥ +5.
Example 3.4
Transform the following linear program into standard form.
Min -2x1 + 3x2
x1 - 3x2 + 2x3 ≤ 3
- x1 + 2x2 ≥ 2
x1 is unrestricted; x2 ≥ 0; x3 ≥ 0
Let us first turn the objective into a max and the constraints into equalities.
Max 2x1 - 3x2
x1 - 3x2 + 2x3 + s1 = 3
- x1 + 2x2 - s2 = 2
x1 unrestricted; x2 ≥ 0; x3 ≥ 0; s1 ≥ 0; s2 ≥ 0
The last step is to convert the unrestricted variable x1 into two nonnegative variables;
where: x1 = x’1 – x’’1. Then, the system of equations is transferred as follow:
Max 2x’1 - 2x’’1 - 3x2
x’1 - x’’1 - 3x2 + 2x3 + s1 = 3
- x’1 + x’’1 + 2x2 - s2 = 2
x’1 ≥ 0; x’’1 ≥ 0; x2 ≥ 0; x3 ≥ 0; s1 ≥ 0; s2 ≥ 0
Operations Research 58 Dr. Emad Elbeltagi
These slack variables represent the part that is not used or not available from each
resource in each constraint. Also, these variables represent the difference between the
actual needs and the availability.
This new form of the problem is called the augmented form, because the original form
has been augmented by some supplementary variables needed to apply the simplex
method. If the slack variable equals zero in the current solution, then this solution lies on
the constraint boundary for the corresponding functional constraint. A value greater than
zero means that the solution lies on the feasible side of this constraint boundary, whereas
a value less than zero means that the solution lies on the infeasible side of this constraint
boundary.
An augmented solution is a solution for the original variables (the decision variables)
that has been augmented by the corresponding values of the slack variables. A basic
solution is an augmented corner-point solution. This can be either feasible or infeasible.
Accordingly, a basic feasible (BF) solution is an augmented CPF solution.
3.4 Solution of Linear Programs by the Simplex Method
For simplicity, in this following example we solve the case where the constraints are of
the form ≤ and the right-hand-sides are nonnegative. We will explain the steps of the
simplex method while we progress through an example.
Example 3.5
Max x1 + x2 (0)
2x1 + x2 ≤ 4 (1)
x1 + 2x2 ≤ 3 (2)
x1 ≥ 0; x2 ≥ 0
First, we convert the problem into standard form by adding slack variables s1, s2 ≥ 0.
Max x1 + x2 (0)
2x1 + x2 + s1 = 4 (1)
x1 + 2x2 + s2 = 3 (2)
x1 ≥ 0; x2 ≥ 0 s1 ≥ 0; s2 ≥ 0
Operations Research 59 Dr. Emad Elbeltagi
Let z denote the objective function value. Here, z = x1 + x2 or, equivalently, z - x1 - x2 = 0:
Putting this equation together with the constraints, we get the following system of linear
equations.
z - x1 - x2 = 0 Row ( 0)
2x1 + x2 + s1 = 4 Row (1)
x1 + 2x2 + s2 = 3 Row 2 (2)
Our goal is to maximize z, while satisfying these equations and, in addition, x1 ≥ 0, x2 ≥
0, s1 ≥ 0, s2 ≥ 0. Note that the equations are already in the form that we expect at the last
step of the Gauss-Jordan procedure. Namely, the equations are solved in terms of the
non-basic variables x1, x2. The variables (other than the special variable z) which
appear in only one equation are the basic variables. Here the basic variables are s1 and
s2.
Generally, assume that the standard form has m equations and n variables (m ≤ n). A
solution can be obtained when exactly n – m variables are set equal to zero. The unique
solutions resulting from setting n – m variables equal to zero are called basic solutions. If
a basic solution satisfies the non-negativity restrictions, it is called a feasible basic
solution. The variables set equal to zero are called non-basic variables; the remaining
ones are called basic variables.
The simplex algorithm starts at the origin, which is usually referred to as the starting
solution. It then moves to an adjacent corner point.
Initialization
An initial basic feasible solution is obtained from the system of equations by setting the
non-basic variables to zero (x1 = x2 = 0). This solution maximizes the values of the slack
variables and this is corresponding to the origin point of the graphical solution. Here this
yields x1 = x2 = 0; s1 = 4; s2 = 3; z = 0. This solution is mathematically correct. However,
the question raised here: Is this an optimal solution or can we increase z (Our goal)?
Optimality test
Operations Research 60 Dr. Emad Elbeltagi
The previous solution yields z =0. By looking at Row 0 above, we can increase z by
increasing x1 or x2. This is because these variables have a negative coefficient in Row 0
and none of the basic variables have a nonzero coefficient in the objective function. If all
coefficients in Row 0 had been non-negative, we could have concluded that the current
basic feasible solution is optimum, since there would be no way to increase z (remember
that all variables must remain ≥ 0). Thus, the first rule of the simplex method:
Rule 1: If all variables have a nonnegative coefficient in Row 0, the current basic
feasible solution is optimal. Otherwise, pick a variable with a negative
coefficient in Row 0.
Determining the direction of movement (Step 1)
The choice of which non basic variable to increase is based on how much this variable
influence the objective function, the one that increases the objective function the most
will be chosen. Increasing this non-basic variable from zero will convert it to basic
variable. The variable chosen by Rule 1 is called the entering variable. Therefore, this
variable is called the entering basic variable. Here let us choose, say, x1 as our entering
variable. It really does not matter which variable we choose as long as it has a negative
coefficient in Row 0 and both variables has the same effect on the objective function.
However, we always select the variable with the highest negative coefficient because
such selection is more likely to lead to the optimal solution rapidly.
Determining where to stop (Step 2)
In this step, it is required to determine how far to increase the entering basic variable x1
before stopping. Increasing x1 increases z, so we want to go as far as possible without
leaving the feasible region. So, setting x2 = 0, yields the following solution.
x2 = 0
s1 = 4 - 2x1
s2 = 3 - x1
Operations Research 61 Dr. Emad Elbeltagi
How far x1 can be increased without violating the non-negativity constraint for the basic
and non-basic variables? Thus, x1 can be increased just to 2, at which point s1 dropped to
zero. Increasing x1 beyond 2 would cause s1 to become negative, which would violate
feasibility. This is called the minimum ratio test, which is the ratio of the right hand side
to the coefficient of the entering basic variable (except we ignore any equation where this
coefficient is zero or negative, since this coefficient leads to no upper bound on the
entering basic variable). Decreasing the basic variable to zero will convert it non-basic
variable for the next basic feasible solution. Therefore, this variable is called the leaving
basic variable because it is leaving the basic variables.
Solving for the new basic feasible solution (Step 3)
Increasing x1 = 0 to x1 = 2 moves us from the initial basic feasible solution on the left to
the new basic feasible solution on the right.
Initial basic feasible solution New basic feasible solution