对偶线性规划 - 维基百科
单凭这么一段文字,很难想象到底发生了什么,只知道以一定的方式将原线性规划问题变为了另一个线性规划问题。
下表提供了对偶问题的具体构建方法,这应该是一个结论,但我们不知所以然。
当然,照着做确实能把一个线性规划问题转换为其对偶问题,但 why?
简单推导,参考《拉格朗日乘子法》
一般形式的线性规划问题:
min
x
∈
R
n
c
⊺
x
s
u
b
j
u
c
t
t
o
A
x
=
b
G
x
≤
h
\begin{aligned} \min\limits_{\bm{x} \in \mathbb{R}^n} & \bm{c}^\intercal \bm{x} \\ subjuct~to~ & A\bm{x} = \bm{b} \\ & G\bm{x} \le \bm{h} \end{aligned}
x∈Rnminsubjuct to c⊺xAx=bGx≤h 其中
b
∈
R
m
,
h
∈
R
r
\bm{b} \in \mathbb{R}^m, \bm{h} \in \mathbb{R}^r
b∈Rm,h∈Rr。现在,对其条件进行一些操作:
(1) 对于等式条件
A
x
=
b
A\bm{x} = \bm{b}
Ax=b,两边同乘以
u
∈
R
m
\bm{u} \in \mathbb{R}^m
u∈Rm,有
u
⊺
A
x
=
u
⊺
b
(1)
\bm{u}^\intercal A\bm{x} = \bm{u}^\intercal\bm{b} \tag{1}
u⊺Ax=u⊺b(1)
(2) 对于不等式条件
G
x
≤
h
G\bm{x} \le \bm{h}
Gx≤h,两边同乘以
v
≤
0
∈
R
r
\bm{v} \le 0 \in \mathbb{R}^r
v≤0∈Rr,有
v
⊺
G
x
≥
v
⊺
h
(2)
\bm{v}^\intercal G\bm{x} \ge \bm{v}^\intercal\bm{h} \tag{2}
v⊺Gx≥v⊺h(2) 执行
(
1
)
+
(
2
)
(1) + (2)
(1)+(2),有
u
⊺
A
x
+
v
⊺
G
x
≥
b
⊺
u
+
h
⊺
v
(
A
⊺
u
+
G
⊺
v
)
⊺
x
≥
b
⊺
u
+
h
⊺
v
\begin{aligned} \bm{u}^\intercal A\bm{x} + \bm{v}^\intercal G\bm{x} &\ge \bm{b}^\intercal \bm{u} + \bm{h}^\intercal \bm{v} \\ (A^\intercal \bm{u} + G^\intercal \bm{v})^\intercal \bm{x} &\ge \bm{b}^\intercal \bm{u} + \bm{h}^\intercal \bm{v} \end{aligned}
u⊺Ax+v⊺Gx(A⊺u+G⊺v)⊺x≥b⊺u+h⊺v≥b⊺u+h⊺v 令
c
=
A
⊺
u
+
G
⊺
v
\bm{c} = A^\intercal \bm{u} + G^\intercal \bm{v}
c=A⊺u+G⊺v,则
c
⊺
x
≥
b
⊺
u
+
h
⊺
v
\bm{c}^\intercal \bm{x} \ge \bm{b}^\intercal \bm{u} + \bm{h}^\intercal \bm{v}
c⊺x≥b⊺u+h⊺v 即
(
b
⊺
u
+
h
⊺
v
)
(\bm{b}^\intercal \bm{u} + \bm{h}^\intercal \bm{v})
(b⊺u+h⊺v) 是
c
⊺
x
\bm{c}^\intercal \bm{x}
c⊺x 的下界。
我们就可以获得原问题的对偶问题:
max
u
∈
R
m
,
v
∈
R
r
b
⊺
u
+
h
⊺
v
s
u
b
j
u
c
t
t
o
A
⊺
u
+
G
⊺
v
=
c
v
≤
0
\begin{aligned} \max\limits_{\bm{u} \in \mathbb{R}^m, \bm{v} \in \mathbb{R}^r} & \bm{b}^\intercal \bm{u} +\bm{h}^\intercal \bm{v} \\ subjuct~to~ & A^\intercal \bm{u} + G^\intercal \bm{v} = \bm{c} \\ & \bm{v} \le 0 \end{aligned}
u∈Rm,v∈Rrmaxsubjuct to b⊺u+h⊺vA⊺u+G⊺v=cv≤0 这只是对标准形式的推导,稍微变化就可以对照上表中的各种情况。
【本质上是拉格朗日对偶】
例子
下面以推导方式和对表方式将这个表中的问题甲转换为问题乙:
推导
∀ y 1 ∈ R \forall y_1 \in \mathbb{R} ∀y1∈R,有 ( 5 x 1 + 6 x 2 ) y 1 = 7 y 1 (5x_1 + 6x_2)y_1 = 7y_1 (5x1+6x2)y1=7y1; ∀ y 2 , y 3 < 0 \forall y_2, y_3 < 0 ∀y2,y3<0,有 y 2 x 1 + y 3 x 2 ≤ 0 y_2x_1 + y_3x_2 \le 0 y2x1+y3x2≤0,则 ( 5 x 1 + 6 x 2 ) y 1 + ( y 2 x 1 + y 3 x 2 ) ≤ 7 y 1 ( 5 y 1 + y 2 ) x 1 + ( 6 y 1 + y 3 ) x 2 ≤ 7 y 1 \begin{aligned} (5x_1 + 6x_2)y_1 + (y_2x_1 + y_3x_2) \le 7y_1 \\ (5y_1 + y_2)x_1 + (6y_1 + y_3)x_2 \le 7y_1 \end{aligned} (5x1+6x2)y1+(y2x1+y3x2)≤7y1(5y1+y2)x1+(6y1+y3)x2≤7y1 令 c = ( 5 y 1 + y 2 , 6 y 1 + y 3 ) = ( 3 , 4 ) \bm{c} = (5y_1 + y_2, 6y_1 + y_3) = (3, 4) c=(5y1+y2,6y1+y3)=(3,4),那么 ( 5 y 1 + y 2 ) x 1 + ( 6 y 1 + y 3 ) x 2 = 3 x 1 + 4 x 2 ≤ 7 y 1 \begin{aligned} (5y_1 + y_2)x_1 + (6y_1 + y_3)x_2 = 3x_1 + 4x_2 \le 7y_1 \end{aligned} (5y1+y2)x1+(6y1+y3)x2=3x1+4x2≤7y1 即 7 y 1 7y_1 7y1 是 ( 3 x 1 + 4 x 2 ) (3x_1 + 4x_2) (3x1+4x2) 的一个上限。且由于 y 2 , y 3 < 0 y_2, y_3 < 0 y2,y3<0,故而 5 y 1 > 3 , 6 y 1 > 4 5y_1 > 3, 6y_1 > 4 5y1>3,6y1>4,就得到了表右边对偶问题。
对照
c = ( 3 , 4 ) \bm{c} = (3, 4) c=(3,4), A = [ 5 6 ] A = \begin{bmatrix} 5 & 6 \end{bmatrix} A=[56], b = ( 7 ) \bm{b} = (7) b=(7),那么按照构建表,对偶问题应该是 max y ∈ R m b ⊺ y s u b j u c t t o A ⊺ y = > < c \begin{aligned} \max\limits_{\bm{y} \in \mathbb{R}^m} & \bm{b}^\intercal \bm{y} \\ subjuct~to~ & A^\intercal \bm{y} \xlongequal[>]{<} \bm{c} \end{aligned} y∈Rmmaxsubjuct to b⊺yA⊺y<>c 代入,即 max y ∈ R m 7 y 1 s u b j u c t t o 5 y 1 ≥ 3 6 y 1 ≥ 4 y 1 ∈ R \begin{aligned} \max\limits_{\bm{y} \in \mathbb{R}^m} & 7y_1 \\ subjuct~to~ & 5y_1 \ge 3 \\ & 6y_1 \ge 4 \\ & y_1 \in \mathbb{R} \end{aligned} y∈Rmmaxsubjuct to 7y15y1≥36y1≥4y1∈R 这可能经过练习熟练后,才适合这么做,否则一头雾水,也容易出错。
最优传输的对偶问题
d M ( r , c ) = min P ∈ U ( r , c ) ∑ i , j P i j M i j U ( r , c ) = { P ∈ R > 0 n × m ∣ P 1 m = r , P ⊺ 1 n = c } \begin{aligned} d_M(\bm{r}, \bm{c}) &= \min_{ P \in U(\bm{r}, \bm{c}) } \sum_{i,j} P_{ij} M_{ij} \\ U(\bm{r}, \bm{c}) &= \{ P\in \mathbb{R}_{>0}^{n \times m} \mid P\bm{1}_m = \bm{r}, P^\intercal\bm{1}_n = \bm{c}\} \end{aligned} dM(r,c)U(r,c)=P∈U(r,c)mini,j∑PijMij={P∈R>0n×m∣P1m=r,P⊺1n=c} 写得更正式一点就是 min P ∑ i , j P i j M i j s u b j u c t t o P 1 m = r P ⊺ 1 n = c P > 0 \begin{aligned} \min_{P} &\sum_{i,j} P_{ij} M_{ij} \\ subjuct~to~ &P\bm{1}_m = \bm{r} \\ &P^\intercal\bm{1}_n = \bm{c} \\ &P > 0 \end{aligned} Pminsubjuct to i,j∑PijMijP1m=rP⊺1n=cP>0 仿照上面的推导步骤,设 α ∈ R n , β ∈ R m \bm{\alpha} \in \mathbb{R}^n, \bm{\beta} \in \mathbb{R}^m α∈Rn,β∈Rm,有 α ⊺ P 1 m = α ⊺ r β ⊺ P ⊺ 1 n = β ⊺ c \begin{aligned} \bm{\alpha}^\intercal P\bm{1}_m = \bm{\alpha}^\intercal \bm{r} \\ \bm{\beta}^\intercal P^\intercal\bm{1}_n = \bm{\beta}^\intercal \bm{c} \end{aligned} α⊺P1m=α⊺rβ⊺P⊺1n=β⊺c 两行相加,得 α ⊺ P 1 m + β ⊺ P ⊺ 1 n = α ⊺ r + β ⊺ c \begin{aligned} \bm{\alpha}^\intercal P\bm{1}_m + \bm{\beta}^\intercal P^\intercal\bm{1}_n = \bm{\alpha}^\intercal \bm{r} + \bm{\beta}^\intercal \bm{c} \end{aligned} α⊺P1m+β⊺P⊺1n=α⊺r+β⊺c 因为最优传输解问题是最小化问题,我们就找一个比之更小的式子,然后最大化,那么令 α ⊺ P 1 m + β ⊺ P ⊺ 1 n = α ⊺ r + β ⊺ c ≤ ⟨ P , M ⟩ α ⊺ P 1 m + β ⊺ P ⊺ 1 n ≤ ⟨ P , M ⟩ \begin{aligned} \bm{\alpha}^\intercal P\bm{1}_m + \bm{\beta}^\intercal P^\intercal\bm{1}_n = \bm{\alpha}^\intercal \bm{r} + \bm{\beta}^\intercal \bm{c} \le \langle P, M \rangle \\ \bm{\alpha}^\intercal P\bm{1}_m + \bm{\beta}^\intercal P^\intercal\bm{1}_n \le \langle P, M \rangle \end{aligned} α⊺P1m+β⊺P⊺1n=α⊺r+β⊺c≤⟨P,M⟩α⊺P1m+β⊺P⊺1n≤⟨P,M⟩ 即 ∑ i , j P i j ( α i + β j ) ≤ ∑ i , j P i j M i j \sum_{i,j} P_{ij} (\alpha_{i} + \beta_{j}) \le \sum_{i,j} P_{ij} M_{ij} i,j∑Pij(αi+βj)≤i,j∑PijMij 如果最大化 ∑ i , j P i j ( α i + β j ) = α ⊺ P 1 m + β ⊺ P ⊺ 1 n = α ⊺ r + β ⊺ c \sum_{i,j} P_{ij} (\alpha_{i} + \beta_{j}) = \bm{\alpha}^\intercal P\bm{1}_m + \bm{\beta}^\intercal P^\intercal\bm{1}_n = \bm{\alpha}^\intercal \bm{r} + \bm{\beta}^\intercal \bm{c} i,j∑Pij(αi+βj)=α⊺P1m+β⊺P⊺1n=α⊺r+β⊺c 就能获得原问题的最优解。由于 P > 0 P > 0 P>0 是变量,要想使 ∑ i , j P i j ( α i + β j ) ≤ ∑ i , j P i j M i j \sum_{i,j} P_{ij} (\alpha_{i} + \beta_{j}) \le \sum_{i,j} P_{ij} M_{ij} ∑i,jPij(αi+βj)≤∑i,jPijMij 始终成立,就必须 α i + β j ≤ M i j \alpha_{i} + \beta_{j} \le M_{ij} αi+βj≤Mij,于是,对偶问题: max α ∈ R m , β ∈ R n α ⊺ r + β ⊺ c s u b j u c t t o α i + β j ≤ M i j \begin{aligned} \max_{\bm{\alpha} \in \mathbb{R}^m, \bm{\beta} \in \mathbb{R}^n} &\bm{\alpha}^\intercal \bm{r} + \bm{\beta}^\intercal \bm{c} \\ subjuct~to~ &\alpha_{i} + \beta_{j} \le M_{ij} \end{aligned} α∈Rm,β∈Rnmaxsubjuct to α⊺r+β⊺cαi+βj≤Mij