高级优化理论与方法(九)
Linear Programming
min
c
T
x
c^Tx
cTx
s.t.
A
x
=
b
Ax=b
Ax=b
x
≥
0
x\geq 0
x≥0
c , x ∈ R n , b ∈ R m , A ∈ R m × n , m ≤ n , r a n k A = m c,x\in \mathbb{R}^n, b\in \mathbb{R}^m,A\in \mathbb{R}^{m\times n},m\leq n, rankA=m c,x∈Rn,b∈Rm,A∈Rm×n,m≤n,rankA=m
Terms & Notations
Constraints
Ω = { x ∈ R n : A x = b , x ≥ 0 } \Omega=\{x\in\mathbb{R}^n:Ax=b,x\geq 0\} Ω={x∈Rn:Ax=b,x≥0}
∑ i = 1 n a i j x i = b j \sum_{i=1}^n a_{ij}x_i=b_j ∑i=1naijxi=bj: hyperplane
∑ i = 1 n a i j x i ≥ b j \sum_{i=1}^n a_{ij}x_i\geq b_j ∑i=1naijxi≥bj: halfspace
Convex
Def: A set Ω \Omega Ω is convex, if ∀ a , b ∈ Ω , ∀ ε ∈ [ 0 , 1 ] : ε a + ( 1 − ε ) b ∈ Ω \forall a,b\in\Omega, \forall \varepsilon \in [0,1]: \varepsilon a+(1-\varepsilon)b\in\Omega ∀a,b∈Ω,∀ε∈[0,1]:εa+(1−ε)b∈Ω
convex combination: ε a + ( 1 − ε ) b \varepsilon a+(1-\varepsilon)b εa+(1−ε)b
Ω = { x ∈ R n : A x = b , x ≥ 0 } \Omega=\{x\in\mathbb{R}^n:Ax=b,x\geq 0\} Ω={x∈Rn:Ax=b,x≥0}: convex polytype
Extreme Point
S S S convex, x ∈ S x\in S x∈S is an extreme point of S S S, if ∀ y , z ∈ S , y ≠ z \forall y,z \in S, y\neq z ∀y,z∈S,y=z, and ∀ ε ∈ [ 0 , 1 ] : x ≠ ε y + ( 1 − ε ) z \forall \varepsilon\in[0,1]: x\neq \varepsilon y+(1-\varepsilon)z ∀ε∈[0,1]:x=εy+(1−ε)z.
Basic Solution
Def: Let B B B be a matrix whose columns are m m m linearly independent column of A A A. Then, w.l.o.g., A = [ B , 0 ] , D ∈ R m × ( n − m ) A=[B,0], D\in \mathbb{R}^{m\times (n-m)} A=[B,0],D∈Rm×(n−m). Let x B = B − 1 b ∈ R m x_B=B^{-1}b\in \mathbb{R}^m xB=B−1b∈Rm. So, x = [ x B , 0 ] T x=[x_B,0]^T x=[xB,0]T is a solution of A x = b Ax=b Ax=b
A x = [ B , 0 ] [ x B 0 ] = B x B + 0 = b Ax=[B,0]\begin{bmatrix} x_B\\ 0 \end{bmatrix}=Bx_B+0=b Ax=[B,0][xB0]=BxB+0=b
[ x B T , 0 T ] T [x_B^T,0^T]^T [xBT,0T]T is a “basic solution” to A x = b Ax=b Ax=b with respect to B B B. The components of x B x_B xB are called nasic variables. The column of B B B are basic columns.
feasible solution: x B ≥ 0 x_B\geq 0 xB≥0
Example
[ 1 1 − 1 4 1 − 2 − 1 1 ] x = [ 8 2 ] \begin{bmatrix} 1 &1&-1&4\\ 1&-2&-1&1 \end{bmatrix}x=\begin{bmatrix} 8\\ 2 \end{bmatrix} [111−2−1−141]x=[82]
B = [ 1 1 1 − 2 ] : x B = [ 6 2 ] , x = [ 6 , 2 , 0 , 0 ] T B=\begin{bmatrix} 1&1\\ 1&-2 \end{bmatrix}:x_B=\begin{bmatrix} 6\\ 2 \end{bmatrix},x=[6,2,0,0]^T B=[111−2]:xB=[62],x=[6,2,0,0]T basic solution & feasible solution
B = [ − 1 4 − 1 1 ] : x B = [ 0 2 ] , x = [ 0 , 0 , 0 , 2 ] T B=\begin{bmatrix} -1&4\\ -1&1 \end{bmatrix}:x_B=\begin{bmatrix} 0\\ 2 \end{bmatrix},x=[0,0,0,2]^T B=[−1−141]:xB=[02],x=[0,0,0,2]T basic solution & feasible solution
B = [ 1 4 1 1 ] : x B = [ 0 2 ] , x = [ 0 , 0 , 0 , 2 ] T B=\begin{bmatrix} 1&4\\ 1&1 \end{bmatrix}:x_B=\begin{bmatrix} 0\\ 2 \end{bmatrix},x=[0,0,0,2]^T B=[1141]:xB=[02],x=[0,0,0,2]T basic solution & feasible solution
B = [ 1 4 − 2 1 ] : x B = [ 0 2 ] , x = [ 0 , 0 , 0 , 2 ] T B=\begin{bmatrix} 1&4\\ -2&1 \end{bmatrix}:x_B=\begin{bmatrix} 0\\ 2 \end{bmatrix},x=[0,0,0,2]^T B=[1−241]:xB=[02],x=[0,0,0,2]T basic solution & feasible solution
B = [ 1 − 1 − 2 − 1 ] : x B = [ 2 − 6 ] , x = [ 0 , 2 , − 6 , 0 ] T B=\begin{bmatrix} 1&-1\\ -2&-1 \end{bmatrix}:x_B=\begin{bmatrix} 2\\ -6 \end{bmatrix},x=[0,2,-6,0]^T B=[1−2−1−1]:xB=[2−6],x=[0,2,−6,0]T basic solution, but not feasible.
x = [ 3 , 1 , 0 , 1 ] T x=[3,1,0,1]^T x=[3,1,0,1]T is a solution of A x = b , x ≥ 0 Ax=b, x\geq 0 Ax=b,x≥0, but not a basic solution.
Theorem
x ∈ Ω x\in\Omega x∈Ω is an extreme point of Ω ⇔ x \Omega \Leftrightarrow x Ω⇔x basic feasible solution.
Theorem
Theorem[Fundamental Theorem of LP]:
- ∃ \exist ∃ feasible solution ⇒ ∃ \Rightarrow \exist ⇒∃ basic feasible solution
- ∃ \exist ∃ optimal solution ⇒ ∃ \Rightarrow \exist ⇒∃ basic optimal solution
Algorithm
- Find an initial basic feasible solution x x x
- while
x
x
x is not optimal do
move to another basic feasible solution
Brute-force:
Enumerate all basic solutions, find the feasible ones, out put the minimum.
注:该暴力算法的时间复杂度特别大,每次搜索需要在 n n n列中,选 m m m列来计算基本解,于是时间复杂度为 O ( C n m ) ≈ O ( n m ) O(C_n^m)\approx O(n^m) O(Cnm)≈O(nm)。
Problems
①How to find an initial basic feasible solution?
②How to decide the optimality of
x
x
x?
③How to construct a new basic feasible solution from
x
x
x?
Problem③
Input:
A
x
=
b
,
A
=
[
B
,
D
]
,
x
=
[
x
B
T
,
0
T
]
T
Ax=b,A=[B,D],x=[x_B^T,0^T]^T
Ax=b,A=[B,D],x=[xBT,0T]T
B
=
[
a
1
,
⋯
,
a
p
,
⋯
,
a
m
]
,
A
=
[
a
1
,
⋯
,
a
m
,
⋯
,
a
q
,
⋯
,
a
n
]
B=[a_1,\cdots,a_p,\cdots,a_m],A=[a_1,\cdots,a_m,\cdots,a_q,\cdots,a_n]
B=[a1,⋯,ap,⋯,am],A=[a1,⋯,am,⋯,aq,⋯,an]
解决的核心思路为把 B B B的 a p a_p ap列换成 a q a_q aq列,从而形成 B ′ B' B′。
[
A
,
b
]
=
[
B
,
D
,
b
]
⇒
[
I
,
Y
,
x
B
]
[A,b]=[B,D,b]\Rightarrow [I,Y,x_B]
[A,b]=[B,D,b]⇒[I,Y,xB]
注:
A
=
[
D
,
B
]
A=[D,B]
A=[D,B],
I
I
I表示单位矩阵,上面的过程为高斯消元。
Claim: a j = y 1 j a 1 + ⋯ + y m j a m a_j=y_{1j}a_1+\cdots+y_{mj}a_m aj=y1ja1+⋯+ymjam
Claim: a 1 , ⋯ , a p − 1 , a p + 1 , a m , a q a_1,\cdots,a_{p-1},a_{p+1},a_m,a_q a1,⋯,ap−1,ap+1,am,aq linearly independent ⇔ y p q = 0 \Leftrightarrow y_{pq}=0 ⇔ypq=0
总结
本节课在研究线性规划问题。先给出了线性规划中的一些基础定义,其中最重要的是关于基本解的定义。接下来我们的目标就是如何找到最优的基本解。我们先给出了一种暴力求解算法作为抛砖引玉。同时给出了算法的一个大致框架,但是这个框架存在三个问题,我们将在下次课中一一解决。