高级优化理论与方法（十）

Simplex

Given a basis $B$, $x=[x_B^T,0{]}^T$, $x_B=B^{T}b$

Goal: new basis $B^{\prime }=B\to a_p,a_q$

$[A,b]=[B,D,b]\Rightarrow [I,Y,y_0]$

$[A,b]=[B^{\prime },D^{\prime },b]\Rightarrow [I,Y^{\prime },y_0^{\prime }]$

$y_0=x_B,y_0^{\prime }=x_B^{\prime }$

注：$[B^{\prime },D^{\prime }]$是把原矩阵中的第$p$列和第$q$列交换后的结果。

Claim: $y_{1j},\cdots ,y_{mj},m+1\le j\le n$, satisfy $a_j=y_{1j}{a}_1+\cdots +y_{mj}{a}_m$

$a_q=y_{1q}{a}_1+\cdots +y_{pq}{a}_p+\cdots +y_{mq}{a}_m$

$a_p=\frac{a_q}{y_{pq}}-{\sum }_{i=1,i\ne p}^m\frac{y_{iq}}{y_{pq}}{a}_i$

Claim: $a_1,\cdots ,a_{p-1},a_{p+1},a_m,a_q$ linearly independent$\iff y_{pq}=0$

$a_p=y_{1q}^{\prime }{a}_1+\cdots +y_{pq}^{\prime }{a}_q+\cdots +y_{mq}^{\prime }{a}_m$

$y_{pq}^{\prime }=\frac{1}{y_{pq}},y_{iq}^{\prime }=-\frac{y_{iq}}{y_{pq}},\forall 1\le i\le m,i\ne p$

$\forall j\ge m+1,j\ne q:\{\begin{array}{l}{y}_{ij}-\frac{y_{pj}{y}_{iq}}{y_{pq}},i\ne p\\ \frac{y_{pj}}{y_{pq}},i=p\end{array}$

$y_{i0}^{\prime }=\{\begin{array}{l}{y}_{i0}-\frac{y_{p0}{y}_{iq}}{y_{pq}},i\ne q\\ \frac{y_{p0}}{y_{pq}},i=p\end{array}$

Claim: $x^{\prime }=[{y_0^{\prime }}^T,0{]}^T$ satisfies $Ax=b$

How to determine p and q?

determine p

Known $q:\Rightarrow ?p$

basis: $B,[I,Y,y_0]$

$a_q=y_{1q}{a}_1+\cdots +y_{mq}{a}_m$

$\{\begin{array}{l}\epsilon a_q=\epsilon y_{1q}{a}_1+\cdots +\epsilon y_{mq}{a}_m\\ b=y_{10}{a}_1+\cdots +y_{m0}{a}_m\end{array}$

$\Rightarrow b=(y_{10}-\epsilon y_{1q})a_1+\cdots +(y_{m0}-\epsilon y_{mq})a_m+\epsilon a_q$

$x^{\prime }=[y_{10}-\epsilon y_{1q},\cdots ,y_{m0}-\epsilon y_{mq},0,\cdots ,0,\epsilon ,0,\cdots ,0]$

A x ′ = b , x ′ ≥ 0 ⇒ ε = m i n { y i 0 y i q : y i q > 0 } ⇒ p = a r g m i n 1 ≤ i ≤ m { y i 0 y i q : y i q > 0 } Ax'=b,x'\geq 0\Rightarrow \varepsilon=min\{\frac{y_{i0}}{y_{iq}}:y_{iq}>0\}\Rightarrow p=argmin_{1\leq i\leq m}\{\frac{y_{i0}}{y_{iq}}:y_{iq}>0\} Ax′=b,x′≥0⇒ε=min{yiq​yi0​​:yiq​>0}⇒p=argmin1≤i≤m​{yiq​yi0​​:yiq​>0}

new basis $B^{\prime }\stackrel{a_p}{\underset{a_q}{⟷}}B,p$ satisfies $y_{p0}-\epsilon y_{pq}=0$

What if $y_{iq}\le 0$ for all $1\le i\le m$?
$\Rightarrow$ for arbitrarily large $\epsilon \ge 0,x^{\prime }$ satisfies $A{x}^{\prime }=b,x^{\prime }\ge 0$
$\Rightarrow$ no solution [unbounded]

determine q

Goal: $c^{T}x\ge c^T{x}^{\prime }$

Given $B,x=[y_{10},\cdots ,y_{m0},0,\cdots ,0{]}^T$

$z_0=c^{T}x=c_1{y}_{10}+\cdots +c_m{y}_{m0}$

$$\begin{gathered} z=c^T{x}^{\prime } \\ =c_1(y_{10}-\epsilon y_{1q})+\cdots +c_m(y_{m0}-\epsilon y_{mq})+c_q\epsilon \\ =z_0-c_p{y}_{p0}+c_q\epsilon -(c_1\epsilon y_{1q}+\cdots +c_m\epsilon y_{mq}) \\ =z_0-c_p\epsilon y_{pq}+c_q\epsilon -(c_1\epsilon y_{1q}+\cdots +c_m\epsilon y_{mq}) \\ =z_0+\epsilon (c_q-(c_1{y}_{1q}+\cdots +c_p{y}_{pq}+\cdots +c_m{y}_{mq})) \\ =z_0+\epsilon (c_q-z_q) \end{gathered}$$

$\Rightarrow z-z_0=\epsilon (c_q-z_q)=\epsilon r_q$

$\because \forall m+1\le j\le n:c_j-z_j$ is computable
$\therefore z-z_0<0\iff \exists m+1\le q\le n:r_q=c_q-z_q<0$

Claim: If $\forall m+1\le j\le n:r_j=c_j-z_j\ge 0$, then $x$ optimal.

Algorithm

Simplex: Given an initial basis B

1. Compute: $[A,b]=[B,D,b]\to [I,Y,y_0]$
2. Compute: $\forall m+1\le j\le n:r_j=c_j-z_j$
3. If $\forall j:r_j\ge 0$, then return $x=[y_0^T,0{]}^T$
4. else pick an $q$ with $r_q<0$
5. If no $y_{iq}>0$ for $1\le i\le m$, then return “no solution”
   else p = a r g m i n { y i 0 y i q : y i q > 0 } p=argmin\{\frac{y_{i0}}{y_{iq}}:y_{iq}>0\} p=argmin{yiq​yi0​​:yiq​>0}
6. Update $[I,Y,y_0]$, goto 2

Example

min $-x_1-2x_2-x_3$
s.t. $$\begin{gathered} 2x_1+x_2-x_3\le 2 \\ 2x_1-x_2+5x_3\le 6 \\ 4x_1+x_2+x_3\le 6 \\ {x}_1,x_2,x_3\ge 0 \end{gathered}$$

$$\begin{gathered} \Rightarrow 2x_1+x_2-x_3+x_4=2 \\ 2x_1-x_2+5x_3+x_5=6 \\ 4x_1+x_2+x_3+x_6=6 \\ {x}_1,\cdots ,x_6\ge 0 \end{gathered}$$

$A=\left[\begin{array}{cccccc}2& 1& -1& 1& 0& 0\\ 2& -1& 5& 0& 1& 0\\ 4& 1& 1& 0& 0& 1\end{array}\right]$

$b=\left[\begin{array}{c}2\\ 6\\ 6\end{array}\right]$

$c=\left[\begin{array}{c}-1\\ -2\\ -1\\ 0\\ 0\\ 0\end{array}\right]$

$B=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]$

$\left[\begin{array}{ccccccc}{x}_4& x_5& x_6& x_1& x_2& x_3& b\\ 1& 0& 0& 2& 1& -1& 2\\ 0& 1& 0& 2& -1& 5& 6\\ 0& 0& 1& 4& 1& 1& 6\end{array}\right]$

$\left[\begin{array}{cccccc}{x}_4& x_5& x_6& x_1& x_2& x_3\\ 2& 6& 6& 0& 0& 0\end{array}\right]$

$z=[0,0,0,-1,-2,-1]\left[\begin{array}{c}2\\ 6\\ 6\\ 0\\ 0\\ 0\end{array}\right]=0$

$z_j={\sum }_{i=1}^m{y}_{ij}{c}_i=0$ ,for $4\le j\le 6$

$r_j=c_j-z_j=\{\begin{array}{l}-1,j=4\\ -2,j=5\\ -1,j=6\end{array}$
$\Rightarrow q=5$（取最小的）

$\frac{y_{i0}}{y_{iq}}=\{\begin{array}{l}2,i=1\\ /,i=2\\ 6,i=3\end{array}$
$\Rightarrow p=1$（取正数中最小的）

$y_{ij}^{\prime }=\{\begin{array}{l}{y}_{ij}^{\prime }-\frac{y_{pj}{y}_{iq}}{y_{pq}},m+1\le j\le n,j\ne q,i\ne q\\ \frac{y_{pj}}{y_{pq}},i=p\end{array}$

$y_{iq}^{\prime }=\{\begin{array}{l}\frac{1}{y_{pq}},i=p\\ -\frac{y_{iq}}{y_{pq}},i\ne p\end{array}$

$\left[\begin{array}{ccccccc}1& 0& 0& 2& 1& -1& 2\\ 0& 1& 0& 4& 1& 3& 8\\ 0& 0& 1& 2& -1& 2& 4\end{array}\right]$

$B^{\prime }=[a_2,a_5,a_6],Y^{\prime }=[a_1,a_4,a_3]$

$x^{\prime }=\left[\begin{array}{cccccc}{x}_2& x_5& x_6& x_1& x_4& x_3\\ 2& 8& 4& 0& 0& 0\end{array}\right]$

$z^{\prime }=-4$

$c=[-2,0,0,-1,0,-1]$

$z_j=\{\begin{array}{l}-4,j=4\\ -2,j=5\\ 2,j=6\end{array}$

$r_j=c_j-z_j=\{\begin{array}{l}3,j=4\\ 2,j=5\\ -3,j=6\end{array}$
$\Rightarrow q=6$

由于步骤比较繁琐，样例不再继续计算。

Initial Basic Feasible Solution

min $c^{T}x$
s.t. $Ax=b$
$x\ge 0$

$\Rightarrow$ Artificial LP
min $y_1+\cdots +y_m\ge 0$
s.t. $[A,I]\left[\begin{array}{c}x\\ y\end{array}\right]=b$
$x\ge 0,y\ge 0$

Theorem

Thm: The initial LP has a basic feasible solution iff the artificial LP has an optimal solution with the objective function equal to 0.

Two-Phases Simplex

min $c^{T}x$
s.t. $Ax=b$
$x\ge 0$
Assume $b\ge 0$

Phase $\mathrm{I}$

1. Construct artifitial LP
2. Apply simplex to artifitial LP with an initial basic feasible solution $[0,\cdots ,0,b]$
3. If the optimal solution of the artifitial LP returns a value $\sum y_i\ne 0$, then “no solution”
4. else Phase $\mathrm{II}$

Phase $\mathrm{II}$

Apply Simplex to the initial LP with the optimal solution of the artificial LP.

Example

min $2x_1+3x_2$
s.t. $4x_1+2x_2-x_3=12$
$x_1+x_2-x_4=6$
$x_1,\cdots ,x_4\ge 0$

Artifitial LP:
min $y_1+y_2$
s.t. $4x_1+2x_2-x_3+y_1=12$
$x_1+x_2-x_4+y_2=6$
$x_1,x_2,x_3,x_4,y_1,y_2\ge 0$

$A=\left[\begin{array}{ccccccc}4& 2& -1& 0& 1& 0& 12\\ 1& 1& 0& -1& 0& 1& 6\end{array}\right]$

$x=0,y=[12,6{]}^T,z=18$

$p=5,q=2$

$A=\left[\begin{array}{ccccccc}\frac{7}{2}& 0& -1& \frac{1}{2}& 1& -\frac{1}{2}& 9\\ \frac{1}{4}& 1& 0& -\frac{1}{4}& 0& \frac{1}{4}& \frac{3}{2}\end{array}\right]$

$y_1=9,x_2=\frac{3}{2},x_1=x_3=x_4=y_2=0,z=9$

$r_j=\{\begin{array}{l}-\frac{7}{2},j=1\\ 1,j=3\\ -\frac{1}{2},j=4\\ \frac{3}{2},j=6\end{array}$
$\Rightarrow q=1$

$\frac{y_{i0}}{y_{iq}}=\{\begin{array}{l}2,i=1\\ 6,i=2\end{array}$
$\Rightarrow p=1$
注：此处$p=1,q=1$表示用$x_1$的那一列和$y_1$的那一列进行交换。

$A=\left[\begin{array}{ccccccc}1& 0& -\frac{2}{7}& \frac{1}{7}& \frac{2}{7}& -\frac{1}{7}& \frac{18}{7}\\ 0& 1& \frac{1}{14}& -\frac{2}{7}& -\frac{1}{14}& \frac{2}{7}& \frac{6}{7}\end{array}\right]$

$x_1=\frac{18}{7},x_2=\frac{6}{7},x_3=x_4=y_1=y_2=0,z=0$

注：此时，$z=0$表示第一阶段已经达到了最优，接下来开始做第二阶段。

$\left[\begin{array}{ccccc}1& 0& -\frac{2}{7}& \frac{1}{7}& \frac{18}{7}\\ 0& 1& \frac{1}{14}& -\frac{2}{7}& \frac{6}{7}\end{array}\right]$

$x=[\frac{18}{7},\frac{6}{7},0,0],z=8$

总结

本节课着重解决上次课提出的关于线性规划的三个问题，即如何找到初始可行解、如何确定$x$达到最优、如何从$x$构造新的可行解。这节课通过一系列的推导，推导出了单纯形法，并给出了单纯形法的算法和样例。对于如何寻找初始可行解的问题，又提出了两阶段的单纯形法。下节课将讨论单纯形法的时间复杂度以及一些优化。