本文为《Linear algebra and its applications》的读书笔记
The first example is simple, but it suggests how a problem of linear programming could involve hundreds, if not thousands, of variables and equations.
EXAMPLE 1
A retail sales company has two warehouses and four stores. A particular model of outdoor hot tub is sold at all four stores, and each store has placed an order with company headquarters for a certain number of these hot tubs. Headquarters determines that the warehouses have enough hot tubs and can ship them immediately. The distances from the warehouses to the stores vary, and the cost of transporting a hot tub from a warehouse to a store depends on the distance. The problem is to decide on a shipping schedule that minimizes the total cost of shipping.
Let x i j x_{ij} xij be the number of units (hot tubs) to ship from warehouse i i i to store j j j.
Let
a
1
a_1
a1 and
a
2
a_2
a2 be the numbers of units available at warehouses 1 and 2, and let
r
1
,
.
.
.
,
r
4
r_1, . . . , r_4
r1,...,r4 be the numbers of units requested by the various stores. Then the
x
i
j
x_{ij}
xij must satisfy the equations
and
x
i
j
≥
0
x_{ij} ≥ 0
xij≥0 for
i
=
1
,
2
i = 1, 2
i=1,2 and
j
=
1
,
.
.
.
,
4
j = 1, . . . , 4
j=1,...,4. If the cost of shipping one unit from warehouse
i
i
i to store
j
j
j is
c
i
j
c_{ij}
cij , then the problem is to minimize the function
subject to the four equalities and ten inequalities listed above.
Simplex Method
The simplex method , discussed below, can easily handle problems the size of Example 1. To introduce the method, however, this section focuses mainly on the canonical linear programming problem, in which the objective function must be maximized. Here is an outline of the steps in the simplex method.
- Select an extreme point x \boldsymbol x x of the feasible set F \mathscr F F.
- Consider all the edges of F \mathscr F F that join at x \boldsymbol x x. If the objective function f f f cannot be increased by moving along any of these edges, then x \boldsymbol x x is an optimal solution.
- If f f f can be increased by moving along one or more of the edges, then follow the path that gives the largest increase and move to the extreme point of F \mathscr F F at the opposite end.
- Repeat the process, beginning at step 2.
Since the value of f f f increases at each step, the path will not go through the same extreme point twice. Since there are only a finite number of extreme points, this process will end at an optimal solution (if there is one) in a finite number of steps. If the problem is unbounded, then eventually the path will reach an unbounded edge at step 3 along which f f f increases without bound.
这是单纯形法的主要思想。具体做法会在下面阐述
The next examples concern canonical linear programming problems in which each of the entries in the m m m-tuple b \boldsymbol b b is positive:
Here c \boldsymbol c c and x \boldsymbol x x are in R n \R^n Rn, A A A is an m × n m×n m×n matrix, and b \boldsymbol b b is in R m \R^m Rm.
The simplex method begins by changing each constraint inequality into an equality. This is done by adding one new variable (slack variable) to each inequality.
slack variable: 松弛变量
EXAMPLE 2
Change the inequality
into the equality
by adding the slack variable
x
3
x_3
x3.
If A A A is m × n m×n m×n, the addition of m m m slack variables in A x ≤ b A\boldsymbol x ≤ \boldsymbol b Ax≤b produces a linear system with m m m equations and n + m n + m n+m variables.
- A solution to this system is called a basic solution if no more than m m m of the variables are nonzero.
- A solution to the system is called feasible if each variable is nonnegative.
Thus, in a basic feasible solution, each variable must be nonnegative and at most m m m of them can be positive. Geometrically, these basic feasible solutions correspond to the extreme points of the feasible set.
EXAMPLE 3
Find a basic feasible solution for the system
SOLUTION
Add
s
l
a
c
k
slack
slack
v
a
r
i
a
b
l
e
s
variables
variables to obtain a system of three equations:
A basic solution of (1) has at most three nonzero values for the variables. The following simple solution is called the basic feasible solution associated with (1):
This solution corresponds to the extreme point
0
\boldsymbol 0
0 in the feasible set (in
R
3
\R^3
R3).
It is customary to refer to the nonzero variables x 4 , x 5 x_4, x_5 x4,x5, and x 6 x_6 x6 in system (1) as basic variables because each has a coefficient of 1 and occurs in only one equation. The basic variables are said to be “in” the solution of (1). The variables x 1 , x 2 x_1, x_2 x1,x2, and x 3 x_3 x3 are said to be “out” of the solution. In a linear programming problem, this particular solution would probably not be optimal since only the slack variables are nonzero.
A standard procedure in the simplex method is to change the role a variable plays in a solution.
(通过这个方法在不同的极端点处求得目标函数的最优解)
For example, although x 2 x_2 x2 is out of the solution in (1), it can be introduced “into” a solution by using elementary row operations. The goal is to pivot on the x 2 x_2 x2 entry in the third equation of (1) to create a new system in which x 2 x_2 x2 appears only in the third equation.
To “pivot” on a particular term here means to transform its coefficient into a 1 and then use it to eliminate corresponding terms in all the other equations, not just the equations below it, as was done in Section 1.2.
First, divide the third equation in (1) by the coefficient of x 2 x_2 x2 to obtain a new third equation:
Second, to equations 1 and 2 of (1) add multiples of this new equation that will eliminate
x
2
x_2
x2 from those equations. This produces the system
The basic solution associated with this new system is
The variable
x
2
x_2
x2 has come into the solution, and the variable
x
6
x_6
x6 has gone out. Unfortunately, this basic solution is not feasible since
x
4
<
0
x_4 < 0
x4<0. This lack of feasibility was caused by an improper choice of a pivot equation. The next paragraph shows how to avoid this problem.
In general, consider the system
and suppose the next step is to bring the variable
x
k
x_k
xk into the solution by using equation
p
p
p to pivot on entry
a
p
k
x
k
a_{pk} x_k
apkxk . The basic solution corresponding to the resulting system will be feasible if the following two conditions are satisfied:
- The coefficient
a
p
k
a_{pk}
apk of
x
k
x_k
xk must be positive.
(When the p p pth equation is divided by a p k a_{pk} apk, the new b p b_p bp term must be positive.) - The ratio
b
p
/
a
p
k
b_p/a_{pk}
bp/apk must be the smallest among all the ratios
b
i
/
a
i
k
b_i /a_{ik}
bi/aik for which
a
i
k
>
0
a_{ik} > 0
aik>0.
(This will guarantee that when the p p pth equation is used to eliminate the x k x_k xk term from the i i i th equation, the resulting b i b_i bi term will be positive.)
EXAMPLE 4
Determine which row to use as a pivot in order to bring
x
2
x_2
x2 into the solution in Example 3.
SOLUTION
Compute the ratios
b
i
/
a
i
2
b_i /a_{i2}
bi/ai2:
Since the first ratio is the smallest, pivot on the
x
2
x_2
x2 term in the first equation. This produces the system
Now the basic feasible solution is
A matrix format greatly simplifies calculations of this type. For instance, system (1) in Example 3 is represented by the augmented matrix
The circled 3 in the
x
2
x_2
x2 column indicates that this entry will be used as a pivot to bring
x
2
x_2
x2 into the solution. Complete row reduction in column 2 produces the new matrix that corresponds to the new system in Example 4:
As in Example 4, the new basic feasible solution is
读到现在大概能理解为什么说一个基本可行解就对应一个极端点了:
如果引入的松弛变量为 0,则表明某个不等式取得了相等关系,也就是说解在该等式表示的超平面上;而如果 x i = 0 x_i=0 xi=0 ( x 1 x_1 x1 非松弛变量),则表明解在 x i = 0 x_i=0 xi=0 这个超平面上。
所以当取得基本可行解时,至少有 n n n 个变量为 0,表明解(该解在 n n n 维空间内) 至少在 n n n 个不同的表示凸包边界的超平面的交点上,该交点一定为顶点,也就一定为极端点
The preceding discussion has prepared the way for a full demonstration of the simplex method, based on the constraints in Example 3. At each step, the objective function in Example 5 will drive the choice of which variable to bring into the solution of the system.
EXAMPLE 5
Maximize
25
x
1
+
33
x
2
+
18
x
3
25x_1 + 33x_2 + 18x_3
25x1+33x2+18x3
SOLUTION
First, add slack variables. Then change the objective function
25
x
1
+
33
x
2
+
18
x
3
25x_1 + 33x_2 + 18x_3
25x1+33x2+18x3 into an
e
q
u
a
t
i
o
n
equation
equation by introducing a new variable
M
M
M given by
M
=
25
x
1
+
33
x
2
+
18
x
3
M =25x_1 + 33x_2 + 18x_3
M=25x1+33x2+18x3. Now the goal is to maximize the variable
M
M
M , where
M
M
M satisfies the equation
The original problem is now restated as follows:
find a solution for which
x
j
≥
0
(
j
=
1
,
.
.
.
,
6
)
x_j ≥ 0 (j = 1, . . . , 6)
xj≥0(j=1,...,6) and for which
M
M
M is as large as possible.
The augmented matrix for this new system is called the initial simplex tableau (初始单纯形表). It is written with two ruled lines in the matrix:
The horizontal line above the bottom row isolates the equation corresponding to the objective function. This last row will play a special role in what follows. (The bottom row is used only to decide which variable to bring into the solution. Pivot positions are never chosen from the bottom row.)
The column headings for the slack variables are in color, to remind us at the end of the calculations that only the original variables are part of the final solution of the problem.
Look in rows 1 to 3 of the tableau above to find the basic feasible solution. The c o l u m n s columns columns o f of of t h e the the 3 × 3 3×3 3×3 i d e n t i t y identity identity m a t r i x matrix matrix in these three rows i d e n t i f y identify identify the basic variables—namely, x 4 , x 5 x_4, x_5 x4,x5, and x 6 x_6 x6. The basic solution is
This solution is not optimal, however, since only the slack variables are nonzero. However, the bottom row implies that
The value of
M
M
M will rise when any of the variables
x
1
,
x
2
x_1, x_2
x1,x2, or
x
3
x_3
x3 rises. Since the coefficient of
x
2
x_2
x2 is the largest of the three coefficients, bringing
x
2
x_2
x2 into the solution will cause the greatest increase in
M
M
M.
To bring x 2 x_2 x2 into the solution, follow the pivoting procedure outlined earlier. The pivot should be the entry 3 that is circled in the first row.
The result of the pivot operation is
Now the columns of the
3
×
3
3 ×3
3×3 identity matrix are in columns 2, 5, and 6 of the tableau. So the basic feasible solution is
Thus
M
M
M has increased from 0 to 660. To see if
M
M
M can be increased further, look at the bottom row of the tableau and solve the equation for
M
M
M :
The value of
M
M
M will increase only if
x
1
x_1
x1 increases (from 0). So
x
1
x_1
x1 needs to come into the solution:
After pivoting, the resulting tableau is
The corresponding basic feasible solution is
The bottom row shows that
The negative coefficients of the variables here show that
M
M
M can be no larger than
4854
7
\frac{4854}{7}
74854, so the solution is optimal. The maximum occurs when
x
1
=
78
7
,
x
2
=
88
7
,
x_1 = \frac{78}{7} , x_2 = \frac{88}{7},
x1=778,x2=788, and
x
3
=
0
x_3 = 0
x3=0.
The fact that the slack variables x 4 x_4 x4 and x 5 x_5 x5 are zero means that the first two inequalities listed at the beginning of this example are both equalities at the optimal values of x 1 , x 2 x_1, x_2 x1,x2, and x 3 x_3 x3.
The goal of step 3 is to produce the greatest increase possible in the value of M M M . This happens when only one variable x k x_k xk satisfies the conditions. Suppose, however, that the most negative entry in the bottom row appears in both columns j j j and k k k . Step 3 says that either x j x_j xj or x k x_k xk should be brought into the solution, and that is correct. Occasionally, a few computations can be avoided by first using step 4 to compute the “smallest ratio” for both columns j j j and k k k , and then choosing the column for which this “smallest ratio” is larger. This situation will arise in Section 9.4.
Two things can go wrong in the simplex algorithm.
-
At step 4, there might be a negative entry in the bottom row of the x k x_k xk column, but no positive entry a i k a_{ik} aik above it. In this case, it will not be possible to find a pivot to bring x k x_k xk into the solution. This corresponds to the case where the objective function is unbounded and no optimal solution exists.
-
The second potential problem also occurs at step 4. The smallest ratio b i / a i k b_i /a_{ik} bi/aik may occur in more than one row. When this happens, the next tableau will have at least one basic variable equal to zero, and in subsequent tableaus the value of M M M may remain constant. Theoretically it is possible for an infinite sequence of pivots to occur and fail to lead to an optimal solution. Such a phenomenon is called cycling. Fortunately, cycling occurs only rarely in practical applications. In most cases, one may arbitrarily choose either row with a minimum ratio as the pivot.
Minimization Problems
So far, each canonical maximizing problem involved a vector b \boldsymbol b b whose coordinates were positive. But what happens when some of the coordinates of b \boldsymbol b b are zero or negative? And what about a minimizing problem?
If some of the coordinates of
b
\boldsymbol b
b are zero, then it is possible for cycling to occur and the simplex method to fail to terminate at an optimal solution.
(Fortunately, cycling does not generally happen in practical applications.)
The case when one of the coordinates of b \boldsymbol b b is negative can occur in practice and requires some special consideration. The difficulty is that all the b i b_i bi terms must be nonnegative in order for the slack variables to provide an initial basic feasible solution.
One way to change a negative b i b_i bi term into a positive term would be to multiply the inequality by − 1 −1 −1 (before introducing slack variables). But this would change the direction of the inequality. Thus a negative b i b_i bi term causes the same problem as a reversed inequality. The following example discusses this case.
EXAMPLE 7
SOLUTION
The minimum of
f
(
x
1
,
x
2
)
f (x_1, x_2)
f(x1,x2) over a set is the same as the maximum of
−
f
(
x
1
,
x
2
)
−f (x_1, x_2)
−f(x1,x2) over the same set. However, in order to use the simplex algorithm, the canonical description of the feasible set must use
≤
≤
≤ signs. So the first inequality above must be rewritten. Thus the original problem is equivalent to the following:
To solve this, let M = − x 1 − 2 x 2 M = −x_1 − 2x_2 M=−x1−2x2 and add slack variables to the inequalities. This creates the linear system
To find a nonnegative solution to this system for which
M
M
M is a maximum, construct the initial simplex tableau:
The corresponding basic solution is
However, since
x
3
x_3
x3 is negative, this basic solution is not feasible.
In order to replace a negative b i b_i bi entry by a positive number, find another negative entry in the same row. (If all the other entries in the row are nonnegative, then the problem has no feasible solution.) This negative entry is in the column corresponding to the variable that should now come into the solution.
In this example, the first two columns both have negative entries, so either x 1 x_1 x1 or x 2 x_2 x2 should be brought into the solution. For example, to bring x 2 x_2 x2 into the solution, select as a pivot the entry a i 2 a_{i2} ai2 in column 2 for which the ratio b i / a i 2 b_i /a_{i2} bi/ai2 is the smallest non-negative number. In this case, only the ratio ( − 14 ) / ( − 1 ) (−14)/(−1) (−14)/(−1) is nonnegative, so the −1 in the first row must be the pivot. After the pivot operations on column 2, the resulting tableau is
Now each entry in the augmented column (except the bottom entry) is positive, and the simplex method can begin. (Sometimes it may be necessary to pivot more than once in order to make each of these terms nonnegative.) The next tableau turns out to be optimal:
The maximum feasible value of
−
x
1
−
2
x
2
−x_1 − 2x_2
−x1−2x2 is
−
20
−20
−20, when
x
1
=
8
x_1 = 8
x1=8 and
x
2
=
6
x_2 = 6
x2=6. So the minimum value of
x
1
+
2
x
2
x_1 + 2x_2
x1+2x2 is
20
20
20.
The “Simplex” in the Simplex Algorithm
The simplex algorithm focuses on the columns of A A A instead of the rows. Suppose that A A A is m × n m×n m×n and denote the columns by a 1 , . . . , a m \boldsymbol a_1, . . . , \boldsymbol a_m a1,...,am. The addition of m m m slack variables creates an m m m by n + m n + m n+m system of equations of the form
where x 1 , . . . , x n + m x_1, . . . , x_{n+m} x1,...,xn+m are nonnegative and { e 1 , . . . , e m } \{\boldsymbol e_1, . . . , \boldsymbol e_m\} {e1,...,em} is the standard basis for R m \R^m Rm.
The initial basic feasible solution is obtained when x 1 , . . . , x n x_1, . . . , x_n x1,...,xn are zero and b 1 e 1 + ⋅ ⋅ ⋅ + b m e m = b b_1\boldsymbol e_1 + · · · + b_m\boldsymbol e_m = \boldsymbol b b1e1+⋅⋅⋅+bmem=b. If s = b 1 + ⋅ ⋅ ⋅ + b m s = b_1 + · · · + b_m s=b1+⋅⋅⋅+bm, then the equation
If v 1 , . . . , v m \boldsymbol v_1 , . . . , \boldsymbol v_m v1,...,vm are linearly independent vectors in R m \R^m Rm, then the convex hull of the set { 0 , v 1 , . . . , v m } \{\boldsymbol 0, \boldsymbol v_1 , . . . , \boldsymbol v_m\} {0,v1,...,vm} is an m m m-dimensional simplex, S S S. A typical vector in S S S has the form c 0 0 + c 1 v 1 + ⋅ ⋅ ⋅ + c m v m c_0\boldsymbol 0 + c_1 \boldsymbol v_1 + · · · + c_m\boldsymbol v_m c00+c1v1+⋅⋅⋅+cmvm, where the weights are nonnegative and sum to one.
shows that b \boldsymbol b b is in what is called the s i m p l e x simplex simplex generated by 0 , s e 1 , . . . , s e m \boldsymbol 0, s\boldsymbol e_1, . . . , s\boldsymbol e_m 0,se1,...,sem. For simplicity, we say that “ b \boldsymbol b b is in an m m m-dimensional simplex determined by e 1 , . . . , e m \boldsymbol e_1, . . . , \boldsymbol e_m e1,...,em.” This is the first simplex in the simplex algorithm.
In general, if v 1 , . . . , v m \boldsymbol v_1, . . . , \boldsymbol v_m v1,...,vm is any basis of R m \R^m Rm, selected from the columns of the matrix P = [ a 1 ⋅ ⋅ ⋅ a n e 1 ⋅ ⋅ ⋅ e m ] P = \begin{bmatrix}\boldsymbol a_1&· · ·&\boldsymbol a_n&\boldsymbol e_1&· · ·&\boldsymbol e_m\end{bmatrix} P=[a1⋅⋅⋅ane1⋅⋅⋅em], and if b \boldsymbol b b is a linear combination of these vectors with nonnegative weights, then b \boldsymbol b b is in an m m m-dimensional simplex determined by v 1 , . . . , v m \boldsymbol v_1, . . . , \boldsymbol v_m v1,...,vm.
A b a s i c basic basic feasible solution of the linear programming problem corresponds to a particular basis from the columns of P P P. The simplex algorithm changes this basis and hence the corresponding simplex that contains b \boldsymbol b b, one column at a time. The various ratios computed during the algorithm drive the choice of columns. Since row operations do not change the linear dependence relations among the columns, each basic feasible solution tells how to build b \boldsymbol b b from the corresponding columns of P P P.