Legendre transform
设区
I
⊂
R
I \subset \mathbb{R}
I⊂R,
f
:
I
→
R
‾
f:I\to \overline{\mathbb{R}}
f:I→R是一个凸函数,则
f
f
f的Legendre transform为
f
∗
:
I
∗
→
R
f^*:I^*\to \mathbb{R}
f∗:I∗→R
f
∗
(
x
∗
)
=
sup
x
∈
I
(
x
∗
x
−
f
(
x
)
)
,
x
∗
∈
I
∗
f^*\left(x^*\right)=\sup_{x\in I}\left(x^*x-f\left(x\right)\right),\quad x^*\in I^*
f∗(x∗)=x∈Isup(x∗x−f(x)),x∗∈I∗
其中
I
∗
=
{
x
∗
∈
R
:
f
∗
(
x
∗
)
<
∞
}
I^* = \left\{x^*\in \mathbb{R}:f^*\left(x^*\right)<\infty\right\}
I∗={x∗∈R:f∗(x∗)<∞}
类似地
定义在凸集
x
⊂
R
n
\mathbf{x}\subset \mathbb{R}^n
x⊂Rn的凸函数
f
:
X
→
R
f:X\to \mathbb{R}
f:X→R,则
f
∗
:
X
∗
→
R
f^*:X^*\to \mathbb{R}
f∗:X∗→R
f
∗
(
x
∗
)
=
sup
x
∈
X
(
⟨
x
∗
,
x
⟩
−
f
(
x
)
)
,
x
∗
∈
X
∗
f^*\left(\mathbf{x}^*\right)=\sup_{\mathbf{x}\in X}\left(\langle \mathbf{x}^*,\mathbf{x}\rangle - f\left(x\right)\right), \mathbf{x}^*\in X^*
f∗(x∗)=x∈Xsup(⟨x∗,x⟩−f(x)),x∗∈X∗
其中
X
∗
=
{
x
∗
∈
R
n
:
sup
x
∈
X
(
⟨
x
∗
,
x
⟩
−
f
(
x
)
)
<
∞
}
X^*=\left\{\mathbf{x}^* \in \mathbb{R}^n: \sup_{\mathbf{x} \in X}\left(\left\langle \mathbf{x}^*, \mathbf{x}\right\rangle-f(\mathbf{x})\right)<\infty\right\}
X∗={x∗∈Rn:supx∈X(⟨x∗,x⟩−f(x))<∞}
Fenchel’s duality theorem
设
f
f
f是定义在
R
n
\mathbb{R}^n
Rn上的适当凸函数,
g
g
g是定义在
R
n
\mathbb{R}^n
Rn上的 适当凹函数,则
inf
x
(
f
(
x
)
−
g
(
x
)
)
=
sup
p
(
g
∗
(
p
)
−
f
∗
(
p
)
)
\inf_{\mathbf{x}}\left(f\left(\mathbf{x}\right) - g\left(\mathbf{x}\right)\right) = \sup_{\mathbf{p}}\left(g^{*}\left(\mathbf{p}\right) - f^*\left(\mathbf{p}\right)\right)
xinf(f(x)−g(x))=psup(g∗(p)−f∗(p))
其中
f
∗
(
x
∗
)
:
=
sup
{
⟨
x
∗
,
x
⟩
−
f
(
x
)
∣
x
∈
R
n
}
g
∗
(
x
∗
)
:
=
inf
{
⟨
x
∗
,
x
⟩
−
g
(
x
)
∣
x
∈
R
n
}
f^*\left(\mathbf{x}^*\right) :=\sup\left\{\left\langle \mathbf{x}^*,\mathbf{x}\right\rangle - f\left(\mathbf{x}\right)|\mathbf{x}\in\mathbb{R}^n\right\}\\ g^*\left(\mathbf{x}^*\right) :=\inf\left\{\left\langle \mathbf{x}^*,\mathbf{x}\right\rangle - g\left(\mathbf{x}\right)|\mathbf{x}\in\mathbb{R}^n\right\}\\
f∗(x∗):=sup{⟨x∗,x⟩−f(x)∣x∈Rn}g∗(x∗):=inf{⟨x∗,x⟩−g(x)∣x∈Rn}
证明:
原问题可以写为
inf
x
,
y
(
f
(
x
)
−
g
(
y
)
)
s.t.
y
=
x
\begin{aligned} \inf_{\mathbf{x},\mathbf{y}}&\left(f\left(\mathbf{x}\right) - g\left(\mathbf{y}\right)\right)\\ \text{s.t.}& \mathbf{y}=\mathbf{x} \end{aligned}
x,yinfs.t.(f(x)−g(y))y=x
于是
L
(
x
,
y
,
λ
)
=
f
(
x
)
−
g
(
y
)
+
λ
T
(
y
−
x
)
L\left(\mathbf{x},\mathbf{y},\mathbf{\lambda}\right) = f\left(\mathbf{x}\right) - g\left(\mathbf{y}\right) + \mathbf{\lambda}^T\left(\mathbf{y}-\mathbf{x}\right)
L(x,y,λ)=f(x)−g(y)+λT(y−x)
对偶
inf
x
,
y
L
=
−
sup
x
(
λ
T
x
−
f
(
x
)
)
+
inf
y
(
λ
T
y
−
g
(
y
)
)
=
g
∗
(
λ
)
−
f
∗
(
λ
)
\begin{aligned} \inf_{\mathbf{x},\mathbf{y}}L &= -\sup_{\mathbf{x}} \left(\mathbf{\lambda}^T\mathbf{x}-f\left(\mathbf{x}\right)\right) + \inf_{\mathbf{y}}\left(\mathbf{\lambda}^T\mathbf{y}-g\left(\mathbf{y}\right)\right)\\ &=g^*\left(\mathbf{\lambda}\right)-f^*\left(\mathbf{\lambda}\right) \end{aligned}
x,yinfL=−xsup(λTx−f(x))+yinf(λTy−g(y))=g∗(λ)−f∗(λ)
因此对偶问题就是
sup
p
(
g
∗
(
p
)
−
f
∗
(
p
)
)
\sup_{\mathbf{p}}\left(g^{*}\left(\mathbf{p}\right) - f^*\left(\mathbf{p}\right)\right)
psup(g∗(p)−f∗(p))
参考
https://en.wikipedia.org/wiki/Fenchel%27s_duality_theorem