高级优化理论与方法（九）

Linear Programming

min $c^{T}x$
s.t. $Ax=b$
$x\ge 0$

$c,x\in {\mathbb{R}}^n,b\in {\mathbb{R}}^m,A\in {\mathbb{R}}^{m\times n},m\le n,rankA=m$

Terms & Notations

Constraints

Ω = { x ∈ R n : A x = b , x ≥ 0 } \Omega=\{x\in\mathbb{R}^n:Ax=b,x\geq 0\} Ω={x∈Rn:Ax=b,x≥0}

${\sum }_{i=1}^n{a}_{ij}{x}_i=b_j$: hyperplane

${\sum }_{i=1}^n{a}_{ij}{x}_i\ge b_j$: halfspace

Convex

Def: A set $\Omega$ is convex, if $\forall a,b\in \Omega ,\forall \epsilon \in [0,1]:\epsilon a+(1-\epsilon )b\in \Omega$

convex combination: $\epsilon a+(1-\epsilon )b$

Ω = { x ∈ R n : A x = b , x ≥ 0 } \Omega=\{x\in\mathbb{R}^n:Ax=b,x\geq 0\} Ω={x∈Rn:Ax=b,x≥0}: convex polytype

Extreme Point

$S$ convex, $x\in S$ is an extreme point of $S$, if $\forall y,z\in S,y\ne z$, and $\forall \epsilon \in [0,1]:x\ne \epsilon y+(1-\epsilon )z$.

Basic Solution

Def: Let $B$ be a matrix whose columns are $m$ linearly independent column of $A$. Then, w.l.o.g., $A=[B,0],D\in {\mathbb{R}}^{m\times (n-m)}$. Let $x_B=B^{-1}b\in {\mathbb{R}}^m$. So, $x=[x_B,0{]}^T$ is a solution of $Ax=b$

$Ax=[B,0]\left[\begin{array}{c}{x}_B\\ 0\end{array}\right]=B{x}_B+0=b$

$[x_B^T,0^T{]}^T$ is a “basic solution” to $Ax=b$ with respect to $B$. The components of $x_B$ are called nasic variables. The column of $B$ are basic columns.

feasible solution: $x_B\ge 0$

Example

$\left[\begin{array}{cccc}1& 1& -1& 4\\ 1& -2& -1& 1\end{array}\right]x=\left[\begin{array}{c}8\\ 2\end{array}\right]$

$B=\left[\begin{array}{cc}1& 1\\ 1& -2\end{array}\right]:x_B=\left[\begin{array}{c}6\\ 2\end{array}\right],x=[6,2,0,0{]}^T$ basic solution & feasible solution

$B=\left[\begin{array}{cc}-1& 4\\ -1& 1\end{array}\right]:x_B=\left[\begin{array}{c}0\\ 2\end{array}\right],x=[0,0,0,2{]}^T$ basic solution & feasible solution

$B=\left[\begin{array}{cc}1& 4\\ 1& 1\end{array}\right]:x_B=\left[\begin{array}{c}0\\ 2\end{array}\right],x=[0,0,0,2{]}^T$ basic solution & feasible solution

$B=\left[\begin{array}{cc}1& 4\\ -2& 1\end{array}\right]:x_B=\left[\begin{array}{c}0\\ 2\end{array}\right],x=[0,0,0,2{]}^T$ basic solution & feasible solution

$B=\left[\begin{array}{cc}1& -1\\ -2& -1\end{array}\right]:x_B=\left[\begin{array}{c}2\\ -6\end{array}\right],x=[0,2,-6,0{]}^T$ basic solution, but not feasible.

$x=[3,1,0,1{]}^T$ is a solution of $Ax=b,x\ge 0$, but not a basic solution.

Theorem

$x\in \Omega$ is an extreme point of $\Omega \iff x$ basic feasible solution.

Theorem

Theorem[Fundamental Theorem of LP]:

1. $\exists$ feasible solution$\Rightarrow \exists$ basic feasible solution
2. $\exists$ optimal solution$\Rightarrow \exists$ basic optimal solution

Algorithm

1. Find an initial basic feasible solution $x$
2. while $x$ is not optimal do
   move to another basic feasible solution

Brute-force:
Enumerate all basic solutions, find the feasible ones, out put the minimum.

注：该暴力算法的时间复杂度特别大，每次搜索需要在$n$列中，选$m$列来计算基本解，于是时间复杂度为$O(C_n^m)\approx O(n^m)$。

Problems

①How to find an initial basic feasible solution?
②How to decide the optimality of $x$?
③How to construct a new basic feasible solution from $x$?

Problem③

Input: $Ax=b，A=[B,D],x=[x_B^T,0^T{]}^T$
$B=[a_1,\cdots ,a_p,\cdots ,a_m],A=[a_1,\cdots ,a_m,\cdots ,a_q,\cdots ,a_n]$

解决的核心思路为把$B$的$a_p$列换成$a_q$列，从而形成$B^{\prime }$。

$[A,b]=[B,D,b]\Rightarrow [I,Y,x_B]$
注：$A=[D,B]$,$I$表示单位矩阵，上面的过程为高斯消元。

Claim: $a_j=y_{1j}{a}_1+\cdots +y_{mj}{a}_m$

Claim: $a_1,\cdots ,a_{p-1},a_{p+1},a_m,a_q$ linearly independent$\iff y_{pq}=0$

总结

本节课在研究线性规划问题。先给出了线性规划中的一些基础定义，其中最重要的是关于基本解的定义。接下来我们的目标就是如何找到最优的基本解。我们先给出了一种暴力求解算法作为抛砖引玉。同时给出了算法的一个大致框架，但是这个框架存在三个问题，我们将在下次课中一一解决。