高级优化理论与方法（十一）

Simplex

Matrix form of Simplex

min $c^{T}x$
s.t. $Ax=b$
$x\ge 0$

Assume: the first $m$ columns of $A$ form a basis $B$.
$A=[B,D],c=[c_B,c_D{]}^T,x=[x_B,x_D{]}^T$

$\Rightarrow$ min $c_B^T{x}_B+c_D^T{x}_D$
s.t. $B{x}_B+D{x}_D=b$
$x_B,x_D\ge 0$

For basic solution $x=[x_B,0{]}^T$
$\Rightarrow x_B=B^{-1}b,x=[B^{-1}b,0{]}^T,z_0=c_B^T{B}^{-1}b$

If $x_D\ne 0$, then $x_B=B^{-1}b-B^{-1}D{x}_D$
$\Rightarrow z=c_B^T(B^{-1}b-B^{-1}D{x}_D)+c_D^T{x}_D=z_0+(c_D^T-c_B^T{B}^{-1}b)x_D$

Define $r_D^T=c_D^T-c_B^T{B}^{-1}b\Rightarrow z=z_0+r_D^T{x}_D$

$\left[\begin{array}{cc}A& b\\ c^T& 0\end{array}\right]=\left[\begin{array}{ccc}B& D& b\\ c_B^T& c_D^T& 0\end{array}\right]$

$\left[\begin{array}{cc}{B}^{-1}& 0\\ 0^T& 1\end{array}\right]\cdot \left[\begin{array}{ccc}B& D& b\\ c_B^T& c_D^T& 0\end{array}\right]=\left[\begin{array}{ccc}{I}_m& Y& y_0\\ c_B^T& c_D^T& 0\end{array}\right]=\left[\begin{array}{ccc}{B}^{-1}B& B^{-1}D& B^{-1}b\\ c_B^T& c_D^T& 0\end{array}\right]$

$\left[\begin{array}{cc}{I}_m& 0\\ -c_B^T& 1\end{array}\right]\cdot \left[\begin{array}{ccc}{I}_m& Y& y_0\\ 0& c_D^T-c_B^T{B}^{-1}D& -c_B^T{B}^{-1}b\end{array}\right]=\left[\begin{array}{ccc}{I}_m& Y& y_0\\ 0& r_D^T& -z_0\end{array}\right]$

Example

max $7x_1+6x_2$
s.t. $2x_1+x_2\le 3$
$x_1+4x_2\le 4$
$x_1,x_2\ge 0$

$\Rightarrow$ min $-7x_1-6x_2$
s.t. $2x_1+x_2+x_3=3$
$x_1+4x_2+x_4=4$
$x_1,x_2,x_3,x_4\ge 0$

$\left[\begin{array}{cc}A& b\\ c^T& 0\end{array}\right]=\left[\begin{array}{ccccc}2& 1& 1& 0& 3\\ 1& 4& 0& 1& 4\\ -7& -6& 0& 0& 0\end{array}\right]$

$B=[a_3,a_4]$

$x=\left[\begin{array}{c}0\\ 0\\ 3\\ 4\end{array}\right]$

$r_B^T=[-7,-6]\Rightarrow q=1$

$z_0=0$

p = a r g m i n { y i 0 y i q : y i q > 0 } p=argmin\{\frac{y_{i0}}{y_{iq}}:y_{iq}>0\} p=argmin{yiq​yi0​​:yiq​>0}
$\frac{3}{2}$ v.s. $\frac{4}{1}\Rightarrow p=1$

$B^{\prime }=[a_1,a_4]$

$\left[\begin{array}{ccccc}1& \frac{1}{2}& \frac{1}{2}& 0& \frac{3}{2}\\ 0& 3\frac{1}{2}& -\frac{1}{2}& 1& \frac{5}{2}\\ 0& -\frac{5}{2}& \frac{7}{2}& 0& \frac{21}{2}\end{array}\right]$

$x=[\frac{3}{2},0,0,\frac{5}{2}{]}^T$

$z_0=-\frac{21}{2}$

$r_D^T=[-\frac{5}{2},\frac{7}{2}]\Rightarrow q=2$

$\frac{y_{i0}}{y_{iq}}:\frac{\frac{3}{2}}{\frac{1}{2}}=3$ v.s. $\frac{\frac{5}{2}}{\frac{7}{2}}=\frac{5}{7}\Rightarrow p=2$

$\left[\begin{array}{ccccc}1& 0& \frac{4}{7}& -\frac{1}{7}& \frac{8}{7}\\ 0& 1& -\frac{1}{7}& \frac{2}{7}& \frac{5}{7}\\ 0& 0& \frac{22}{7}& \frac{5}{7}& \frac{86}{7}\end{array}\right]$

$B=[a_1,a_2],x=[\frac{8}{7},\frac{5}{7},0,0{]}^T,z=-\frac{86}{7},r_D^T=[\frac{22}{7},\frac{5}{7}]\ge 0$

$x$ optimal !

Degenerated Basic Solutions

Example

min $-\frac{3}{4}{x}_1+20x_2-\frac{1}{2}{x}_3+6x_4$
s.t. $\frac{1}{4}{x}_1-8x_2-x_3+9x_4+x_5=0$
$\frac{1}{2}{x}_1-12x_2-\frac{1}{2}{x}_3+3x_4+x_6=0$
$x_3+x_7=1$
$x_1,\cdots ,x_7\ge 0$

$\left[\begin{array}{cccccccc}\frac{1}{4}& -8& -1& 9& 1& 0& 0& 0\\ \frac{1}{2}& -12& -\frac{1}{2}& 3& 0& 1& 0& 0\\ 0& 0& 1& 0& 0& 0& 1& 1\\ -\frac{3}{4}& 20& -\frac{1}{2}& 6& 0& 0& 0& 0\end{array}\right]$

$B=[a_5,a_6,a_7],x=[0,0,0,0,0,0,1{]}^T,z=0,q=1,p=1$
注：此时，选$p=1$和$p=2$都可以，这里不妨选$p=1$

$\left[\begin{array}{cccccccc}1& -32& -4& 36& 4& 0& 0& 0\\ 0& 4& \frac{3}{2}& -15& -2& 1& 0& 0\\ 0& 0& 1& 0& 0& 0& 1& 1\\ 0& -4& -\frac{7}{2}& 33& 3& 0& 0& 0\end{array}\right]$

$B=[a_1,a_6,a_7],x=[0,0,0,0,0,0,1{]}^T,z=0,q=2,p=2$

$\left[\begin{array}{cccccccc}1& 0& 8& -84& -12& 8& 0& 0\\ 0& 1& \frac{3}{8}& -\frac{15}{4}& -\frac{1}{2}& \frac{1}{4}& 0& 0\\ 0& 0& 1& 0& 0& 0& 1& 1\\ 0& 0& -2& 18& 1& 1& 0& 0\end{array}\right]$

$B=[a_1,a_2,a_7],x=[0,0,0,0,0,0,1{]}^T,z=0,q=3,p=1$

$\left[\begin{array}{cccccccc}\frac{1}{8}& 0& 1& -\frac{21}{2}& -\frac{3}{2}& 1& 0& 0\\ -\frac{3}{64}& 1& 0& \frac{3}{16}& \frac{1}{16}& -\frac{1}{8}& 0& 0\\ -\frac{1}{8}& 0& 0& \frac{21}{2}& \frac{3}{2}& -1& 1& 1\\ \frac{1}{4}& 0& 0& -3& -2& 3& 0& 0\end{array}\right]$

$B=[a_3,a_2,a_7],x=[0,0,0,0,0,0,1{]}^T,z=0,q=4,p=2$

$\left[\begin{array}{cccccccc}-\frac{5}{2}& 56& 1& 0& 2& -6& 0& 0\\ -\frac{1}{4}& \frac{16}{3}& 0& 1& \frac{1}{3}& -\frac{2}{3}& 0& 0\\ \frac{5}{2}& -56& 0& 0& -2& 6& 1& 1\\ -\frac{1}{2}& 16& 0& 0& -1& 1& 0& 0\end{array}\right]$

$B=[a_3,a_4,a_7],x=[0,0,0,0,0,0,1{]}^T,z=0,q=5,p=1$

$\left[\begin{array}{cccccccc}-\frac{5}{4}& 28& \frac{1}{2}& 0& 1& -3& 0& 0\\ \frac{1}{6}& -4& -\frac{1}{6}& 1& 0& \frac{1}{3}& 0& 0\\ 0& 0& 1& 0& 0& 0& 1& 1\\ -\frac{7}{4}& 44& \frac{1}{2}& 0& 0& -2& 0& 0\end{array}\right]$

$B=[a_5,a_4,a_7],x=[0,0,0,0,0,0,1{]}^T,z=0,q=6,p=2$

$\left[\begin{array}{cccccccc}\frac{1}{4}& -8& -1& 9& 1& 0& 0& 0\\ \frac{1}{2}& -12& -\frac{1}{2}& 3& 0& 1& 0& 0\\ 0& 0& 1& 0& 0& 0& 1& 1\\ -\frac{3}{4}& 20& -\frac{1}{2}& 6& 0& 0& 0& 0\end{array}\right]$

$B=[a_5,a_6,a_7],x=[0,0,0,0,0,0,1{]}^T,z=0,q=1,p=1$
注：此时回到了初始的结果，算法陷入了死循环。这个例子用于解释退化基本解会导致单纯形法陷入死循环的情况。

Bland’s Method

①The entering variable should be the lowest index variable with negative $r_j$.

②The leaving variable (in case $p$ a tie in the min ratio) should be the lowest index variable.

注：该方法给出了避免产生死循环的方法，即当$p$和$q$有多个选法时，选择下标较小的变量。Bland声称自己有证明，但是找不到了。该方法在实践中证明有效，但是缺乏理论证明。其理论证明是一个数学界的公开问题。

Remarks

若$n>>m$，那么用上面的算法计算起来计算量非常大，为了减小计算量，我们要找到一些简便方法。

Claim

Claim: $E{B}^{-1}=B_{new}^{-1}$, where $B_{new}=B[a_p\leftrightarrow a_q]$

Revised Simplex

1. Form the matrix $[B^{-1},y_0]([B,I_m,b]\to [I_m,B^{-1},y_0])$
2. Calculate $r_D^T=c_D^T-{\lambda }^{T}D,{\lambda }^T=c_B^T{B}^{-1}$
3. If $\forall j\ge m+1:r_j\ge 0$, then return the current basic solution.
4. Select $q$ with $r_q<0$
5. Compute $y_q=B^{-1}{a}_q$
6. If no $y_{iq}>0$, then return “no solution”
   else p = a r g m i n { y i 0 y i q : y i q > 0 } p=argmin\{\frac{y_{i0}}{y_{iq}}:y_{iq}>0\} p=argmin{yiq​yi0​​:yiq​>0}
7. Form $[B^{-1},y_0,y_q]$ and compute $E{B}^{-1},E{y}_0$
8. Goto 2

Example

min $-3x_1-5x_2$
s.t. $x_1+x_2+x_3=4$
$5x_1+3x_2-x_4=8$
$x_1,x_2,x_3,x_4\ge 0$

Phase $\mathrm{I}$

min $x_5$
s.t. x_1+x_2+x_3=4$
$5x_1+3x_2-x_4+x_5=8$
$x_1,\cdots ,x_5\ge 0$

$\left[\begin{array}{ccccc}1& 1& 1& 0& 0\\ 5& 3& 0& -1& 1\end{array}\right]$

$B=[a_3,a_5],B^{-1}=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right],c_B=[0,1],[B^{-1},y_0]=\left[\begin{array}{ccc}1& 0& 4\\ 0& 1& 8\end{array}\right]$

Compute $r:{\lambda }^T=c_B^T{B}^{-1}=[0,1]$
$r^T=c_B^T-{\lambda }^{T}D=[0,0,0]-[0,1]\left[\begin{array}{ccc}1& 1& 0\\ 5& 3& -1\end{array}\right]=\left[\begin{array}{c}5\\ -3\\ 1\end{array}\right]$

$q=1,y_1=B^{-1}{a}_1=\left[\begin{array}{c}1\\ 5\end{array}\right]$

$\frac{y_{i0}}{y_{iq}}=4$ v.s. $\frac{8}{5}$
$p=2$

$E=\left[\begin{array}{cc}1& -\frac{1}{5}\\ 0& \frac{1}{5}\end{array}\right]$

$y_0^{\prime }=E{y}_0=\left[\begin{array}{c}\frac{12}{5}\\ \frac{8}{5}\end{array}\right]$

$B_{new}^{-1}=E{B}^{-1}=\left[\begin{array}{cc}1& -\frac{1}{5}\\ 0& \frac{1}{5}\end{array}\right]$

$B_{new}=[a_3,a_1],c_B=[0,0]$

${\lambda }_{new}^T=c_B^T{B}_{new}^{-1}=[0,0]$

$r_{new}=c_B^T-{\lambda }_{new}^{T}D=[0,0,1]-[0,0]\left[\begin{array}{ccc}1& 0& 0\\ 3& -1& 1\end{array}\right]=[0,0,1]\Rightarrow$ optimal

Phase $\mathrm{II}$

$B=[a_3,a_1],y_0=[\frac{12}{5},\frac{8}{5}],c_B=[0,-3]$

$B^{-1}=\left[\begin{array}{cc}1& -\frac{1}{5}\\ 0& \frac{1}{5}\end{array}\right]$

$\left[\begin{array}{ccccc}1& 1& 1& 0& 4\\ 5& 3& 0& -1& 8\\ -3& -5& 0& 0& 0\end{array}\right]$

$c_D=[-5,0]$

Compute $r:{\lambda }^T=c_B^T{B}^{-1}=[0,-3]\left[\begin{array}{cc}1& -\frac{1}{5}\\ 0& \frac{1}{5}\end{array}\right]=[0,-\frac{3}{5}]$
$r_D^T=c_D^T-{\lambda }^{T}D=[-5,0]-[0,-\frac{3}{5}]\left[\begin{array}{cc}1& 0\\ 3& -1\end{array}\right]=\left[\begin{array}{c}-\frac{16}{5}\\ -\frac{3}{5}\end{array}\right]$
$q=2$

$y_2=B^{-1}{a}_2=\left[\begin{array}{cc}1& -\frac{1}{5}\\ 0& \frac{1}{5}\end{array}\right]\left[\begin{array}{c}1\\ 3\end{array}\right]=\left[\begin{array}{c}\frac{2}{5}\\ \frac{3}{5}\end{array}\right]$

$\frac{y_{i0}}{y_{iq}}:\frac{\frac{12}{5}}{\frac{2}{5}}$ v.s. $\frac{\frac{8}{5}}{\frac{3}{5}}\Rightarrow p=2$

$E=\left[\begin{array}{cc}1& -\frac{2}{3}\\ 0& \frac{5}{3}\end{array}\right]$

$B_{new}^{-1}=E{B}^{-1}=\left[\begin{array}{cc}1& -\frac{1}{3}\\ 0& \frac{1}{3}\end{array}\right],y_{new}=E{y}_0=\left[\begin{array}{c}\frac{4}{3}\\ \frac{8}{3}\end{array}\right],B_{new}=[a_3,a_2]$

Compute $r_{new}:r_{new}^T=\left[\begin{array}{c}\frac{16}{3}\\ -\frac{5}{3}\end{array}\right]\Rightarrow q=4$

$y_4=B_{new}^{-1}{a}_4=\left[\begin{array}{cc}1& -\frac{1}{3}\\ 0& \frac{1}{3}\end{array}\right]\left[\begin{array}{c}0\\ 1\end{array}\right]=\left[\begin{array}{c}\frac{1}{3}\\ -\frac{1}{3}\end{array}\right]\Rightarrow p=1$

$B_{new}^{\prime }=[a_4,a_2]$

$E=\left[\begin{array}{cc}3& 0\\ 1& 1\end{array}\right]$

${B_{new}^{\prime }}^{-1}=E{B}_{new}=\left[\begin{array}{cc}3& -1\\ 1& 0\end{array}\right],y_{new}^{\prime }=E{y}_{new}=\left[\begin{array}{c}4\\ 4\end{array}\right]$

Compute $r_{new}^{\prime }=[2,5]>0$

OPT: $[0,4,0,4]$

Duality of LP

Normal Form

Primal LP:
min $c^{T}x$
s.t. $Ax\ge b$
$x\ge 0$

Dual LP:
min $y^{T}b$
s.t. $y^{T}A\le c^T$
$y\ge 0$
symmatric form of duality（对偶的对称形式）

Claim

Claim: Dual of dual = primal

Standard Form

primal:
min $c^{T}x$
s.t. $Ax=b$
$x\ge 0$

$\Rightarrow$ min $c^{T}x$
s.t. $Ax\ge b$
$-Ax\ge -b$
$x\ge 0$

$\Rightarrow$ min $c^{T}x$
s.t.$\left[\begin{array}{c}A\\ -A\end{array}\right]x\ge \left[\begin{array}{c}b\\ -b\end{array}\right]$
$x\ge 0$

$\Rightarrow$ max $u^{T}b-v^{T}b$
s.t. ${\left[\begin{array}{c}u\\ v\end{array}\right]}^T\left[\begin{array}{c}A\\ -A\end{array}\right]\le c^T$
$u,v\ge 0$

Define $y=u-v$
$\Rightarrow$ max $y^{T}b$
s.t. $y^{T}A\le c^T$

Asymmetric form of duality（对偶的不对称形式）

Example

max $2x_1+5x_2+x_3$
s.t. $2x_1-x_2+7x_3\le 6$
$x_1+3x_2+4x_3\le 9$
$3x_1+6x_2+x_3\le 3$

$\Rightarrow$ min $[6,9,3]y$
s.t. $\left[\begin{array}{ccc}2& -1& 7\\ 1& 3& 4\\ 3& 6& 1\end{array}\right]y\ge \left[\begin{array}{c}2\\ 5\\ 1\end{array}\right]$
$y\ge 0$

Theorem

Thm(Week LP duality): If $x$ is any feasible solution to the primal LP and $y$ is any feasible solution to the dual $c^{T}x\ge y^{T}b$.

总结

前几节课讲的单纯形法侧重于对方法的推导，这节课比较侧重于实践。一上来先规范了单纯形法的解法，并给出例子予以说明。然后讨论了退化基本解的问题，给出例子说明按照之前推导的方法不一定收敛，可能产生死循环。于是给出了Bland’s Method，可有效避免死循环的问题，但是该方法的正确性在理论上还没有得到证明。接着，对于$n$远大于$m$的情况，为了减少计算量，又介绍了修正单纯形法。至此，单纯形法基本介绍完毕。后面开始介绍关于线性规划的对偶性问题，不加证明地给出了两种形式下的对偶线性规划，并且两种对偶体现为对偶的对称形式和对偶的不对称形式。最后给出了线性规划的弱对偶定理。下节课将进一步介绍对偶线性规划的相关知识。