Optimization Week 9: Convex conjugate (Fenchel Conjugate)

Every function has something known as the convex conjugate, or differential conjugate. And, this is a very important concept in convexity and in duality.

1 Definition

Given function f : R n → R f: \mathbb{R}^n\rightarrow \mathbb{R} f:RnR, its conjugate f ∗ f^* f is:
f ∗ ( y ) = max ⁡ x y T x − f ( x ) f^*(y)=\max_x y^Tx-f(x) f(y)=xmaxyTxf(x)
f ∗ f^* f is always convex because it is max of affine functions.

2 Properties

Sum of conjugate

f ( x ) + f ∗ ( y ) ≥ y T x f(x)+f^*(y)\geq y^Tx f(x)+f(y)yTx

Decompose conjugate

If f ( u , v ) = f 1 ( u ) + f 2 ( v ) f(u,v)=f_1(u)+f_2(v) f(u,v)=f1(u)+f2(v),
then f ∗ ( w , z ) = f 1 ∗ ( w ) + f 2 ∗ ( z ) f^*(w,z)=f_1^*(w)+f_2^*(z) f(w,z)=f1(w)+f2(z)

Double conjugate

f ∗ ∗ ( z ) ≤ f ( z ) f^{**}(z)\leq f(z) f(z)f(z)
Always less than the original function, and is convex. Convex envelope.
If f f f is convex, then f ∗ ∗ ( z ) = f ( z ) f^{**}(z) = f(z) f(z)=f(z).

Convex original

  • f ∗ ∗ ( z ) = f ( z ) f^{**}(z) = f(z) f(z)=f(z).
  • f ∗ ( y ) = max ⁡ x y T x − f ( x ) = y T x ˉ − f ( x ˉ ) f^*(y)=\max_x y^Tx-f(x)=y^T\bar x-f(\bar x) f(y)=maxxyTxf(x)=yTxˉf(xˉ), where y ∈ ∂ f ( x ˉ ) y\in \partial f(\bar x) yf(xˉ)
  • The derivatives of f f f and f ∗ f^* f are inverse of each other: y ∈ ∂ f ( x ) ⇔ x ∈ ∂ f ∗ ( y ) y\in \partial f(x)\Leftrightarrow x\in \partial f^*(y) yf(x)xf(y)

3 Examples-todo

Uses: duality 2

已标记关键词 清除标记
©️2020 CSDN 皮肤主题: 1024 设计师:白松林 返回首页