Linear Programming Learning Notes (2) The Simplex Method
All the resources come from Linear Programming: Foundations and Extensions by Professor Robert J. Vanderbei.
Explore the link below for further information:
LP Book Resources
Part 1
Basic Theory: The Simplex Method and Duality
Chapter 2 The Simplex Method
An Example
maximize
5x1+4x2+3x3
s.t.
2x1+3x2+x3≤5
4x1+x2+2x3≤11
3x1+4x2+2x3≤8
x1,x2,x3≥0
(2.1)
Step1: Rewrite with Slack Variables
maximize
ζ=5x1+4x2+3x3
s.t.
w1=5−2x1−3x2−x3
w2=11−4x1−x2−2x3
w3=8−x1−4x2−2x3
x1,x2,x3,w1,w2,w3≥0
(2.2)
Step2: Iteration
(2.2) is a so-called Feasible Dicionary Solution.
- To start the iterative process, we need an initial feasible solution. Here, we simply set all the original variables to zero and determine the slack variables using the defining equations:
x1=0,x2=0,x3=0,w1=5,w2=11,w3=8 - Improve
ζ
by increasing the decision variables with the biggest positive coefficients, but satisfying the constraints.
Here we increase x1 since its coefficient is biggest 5. Satisfying the w1,w2,w3 constraints, we finally get x1≤5/2 , so our new, improved solution then is :
x1=5/2,x2=0,x3=0,w1=0,w2=1,w3=1/2 - To proceed, we need to have one group of variables that were initially zero and we had the rest explicitly expressed in terms of these. The only thing we need to do is to swap the roles of
x1
and
w1
, by using the equation for
w1
in (2.2) to solve for
x1
:
x1=52−12w1−32x2−12x3 Perform row operations on the equations in (2.2):
ζ=12.5−2.5w1−3.5x2+0.5x3−−−−−−−−−−−−−−−−−−−−−−−−−−
x1=2.5−0.5w1−1.5x2−0.5x3
w2=1+2w1+5x2
w3=0.5+1.5w1+0.5x2−0.5x3 (2.3)Repeat 2-4 steps until all the coefficients of the variables in ζ are negative. The final result is :
ζ=13−w1−3x2−w3−−−−−−−−−−−−−−−−−−−
x1=22−2w1−2x2+x3
w2=1+2w1+5x2
x3=1+3w1+x2−2w3
with x1=2,x2=0,x3=1,w1=0,w2=1,w3=0 (2.4)
Theory
1. Dictionaries, Bases, Etc.
The systems of equations(2.2),(2.3)and (2.4) are dictionaries. With the exception of
ζ
, the variables that appear on the left (dependent variables) are called basic variables. Those on the right (independent variables) are called nonbasic variables. The solutions we have obtained by setting the nonbasic variables to zero are called basic feasible solutions.
2. The Simplex Method
Standard Form:
maximize ∑nj=1cjxj
s.t. ∑nj=1aijxj≤bi,i=1,2,...,m
xj≥0,j=1,2,...,nRewrite with Slack Variables:
maximize ζ=∑nj=1cjxj
s.t. wi=b−∑nj=1aijxj≤bi,i=1,2,...,mInterwine the slack variables with the originals:
(x1,...,xn,w1,...,wm)=(x1,...,xn,xn+1,...,xn+m)Iterations of the Simplex Method:
Let B denote the collection of indices from {1,2,...,n+m} corresponding to the basic variables, and let N denote that to the nonbasic ones. Initially, we have N={1,2,...,n} and B={n+1,n+2,...,n+m} , but this changes after iterations.- Pick one variable to transfer it from basic to nonbasic. This one is called the entering variable. It is chosen with the aim of increasing
ζ
: that is, one whose coefficient is positive. There are several selection criteria, here it is sufficed to say that we usually pick an index
k
from
{j∈N:cj¯¯¯>0} having the largest coefficient. The bar put over the coefficient indicate that they change as the algorithm progresses. - Determine the corresponded leaving variable, the one goes from basic to nonbasic. It is chosen to preserve nonnegativity of the current basic variables.
We need to ensure that each of the variables remains nonnegative. Hence, we require that:
b¯i−a¯ikxk=≥0,i∈B
so, we get:
xk=mini∈B:a¯ik>0b¯i/a¯ik
The indice of the leaving variable l is pickl from {i∈B:b¯i/a¯ik is minimal } .
It will change from one dictionary to the next, which is called a pivot. Particular rules that make the entering variable choice unambiguous are called pivot rules. - Repeat step 1 and 2 until there is no coefficients of nonbasic variables in ζ is positive.
3. Initializtion
The right-hand sides of the dictionary being all nonnegative ensures that the initial dictionary is feasible, but this is not always the case. We handle this difficulty by introducing an auxiliary problem for which
(1) A feasible dictionary is easy to find and
(2) The optimal dictionary provides a feasible dictionary for the original problem.
The auxiliary problem is :
maximize −x0
s.t. ∑nj=1aijxj−x0≤bi,i=1,2,...,m
xj≥0,j=0,1,2,...,n
It is best to illustrate how to obtain a feasible dictionary with an example:
maximize −2x1−2x2
s.t. −x1+x2≤−1
−x1−2x2≤−2
x2≤1
x1,x2≥0
The auxiliary problem is
maximize −x0
s.t. −x1+x2−x0≤−1
−x1−2x2−x0≤−2
x2−x0≤1
x0,x1,x2≥0
The initial infeasible dictionary:
ζ=−x0−−−−−−−
w1=−1+x1−x2+x0
w2=−2+x1+2x2+x0
w3=1−x2+x0
To convert it into a feasible one, all we need to do is one pivot with variable x0 entering and the “most infeasible variable”, w2 , leaving the basis: (This is called Phase I)
ζ=−2+x1+2x2−w2−−−−−−−−−−−−−−−−−−−
w1=1−3x2+w2
x0=2−x1−2x2+w2
w3=3−x1−3x2+w2
Using the steps introduced section before, we get the optimal dictionary: (This is called Phase II)
ζ=0−x0−−−−−−−−−
x2=0.33−0.33w1+0.33w2
x1=1.33−x0+0.67w1+0.33w2
w3=0.67+x0+0.33w1−0.33w2
We now drop x0 from the equations and reintroduce the original objective function:
ζ=−2x1−x2=−3−w1−w2
Hence, the starting feasible dictionary for the original problem is :
ζ=−2x1−x2=−3−w1−w2−−−−−−−−−−−−−−−−−−−−−−−−−
x2=0.33−0.33w1+0.33w2
x1=1.33+0.67w1+0.33w2
w3=0.67+0.33w1−0.33w2
4. Unboundedness
If all the ratios a¯ik/b¯i are non positive, then the entering variable can be increased indefinitely to produce an arbitrarily large objective value. In such situations, we say that the problem is unbounded. For example:
ζ=5+x3−x1−−−−−−−−−−−−−
x2=5+2x3−3x1
x4=7−4x1
x5=x1
It is unbounded.- Pick one variable to transfer it from basic to nonbasic. This one is called the entering variable. It is chosen with the aim of increasing
ζ
: that is, one whose coefficient is positive. There are several selection criteria, here it is sufficed to say that we usually pick an index
k
from