# Chapter3: Convex Functions

## 3.1 Basic properties and examples

### 3.1.1 Definition

A function f : R n → R f:\mathbf{R}^n\rightarrow\mathbf{R} is a convex function if d o m   f \mathbf{dom}\space f is convex set and for x , y ∈ d o m   f , θ ∈ [ 0 , 1 ] x,y\in \mathbf{dom}\space f,\theta\in[0,1] , we have
f ( θ x + ( 1 − θ ) y ) ≤ θ f ( x ) + ( 1 − θ ) f ( y ) f(\theta x+(1-\theta)y)\leq \theta f(x)+(1-\theta)f(y)

f f is concave if − f -f is convex.

f f is convex if and only if for all x ∈ d o m   f x\in \mathbf{dom}\space f , and for all v v , the function g ( t ) = f ( x + t v ) g(t)=f(x+tv) is convex.

### 3.1.2 Extended-value extensions

If f is convex define its extended-value extension f ~ : R n → R ∪ { ∞ } \tilde{f}:\mathbf{R^n}\rightarrow\mathbf{R}\cup \{\infty\} :
f ~ = { f ( x ) x ∈ d o m   f ∞ x ∉ d o m   f \tilde{f}=\left\{ \begin{array}{rcl} f(x) &x \in \mathbf{dom}\space f \\ \infty &x \notin \mathbf{dom}\space f \end{array}\right.

### 3.1.3 First-order conditions

Suppose f f is a convex, then
f ( y ) ≥ f ( x ) + ∇ f ( x ) T ( y − x ) f(y)\geq f(x)+\nabla f(x)^T(y-x)
holds for all x , y ∈ d o m f x,y\in \mathbf{dom}f .

### 3.1.4 Second-order conditions

f f is convex if and only if d o m f \mathbf{dom}f is convex and for all x ∈ d o m f x\in \mathbf{dom}f , ∇ 2 f ⪰ 0 \nabla^2f\succeq0 .

### 3.1.5 Examples

Powers of absolute value. ∣ x ∣ p |x|^p for p ≥ 1 p\geq1 is convex.

Negative entropy. x l o g x x log x (either on R + + \mathbf{R}_{++} , or on R + \mathbf{R}_+ , defined as 0 0 for x = 0 x = 0 ) is convex.

Norms. Every norm on R n \mathbf{R}^n is convex.

Log-sum-exp. The function f ( x ) = log ⁡ ( e x 1 + ⋅ ⋅ ⋅ + e x n ) f(x) = \log(e^{x_1} +···+e^{x_n} ) is convex on R .

Geometric mean. The geometric mean f ( x ) = ( ∏ i n x i ) 1 n f(x) = (\prod_i^{n}x_i)^{\frac{1}{n}} is concave on d o m f = R + + n \mathbf{dom}f=\mathbf{R}_{++}^n .

Log-determinant. The function$f (X ) = \log \det X$ is concave on d o m f = S + + n \mathbf{dom}f=\mathbf{S}_{++}^n .

### 3.1.6 Sublevel sets

The α \alpha -sublevel set of a function f : R n → R f:\mathbf{R}^n\rightarrow\mathbf{R} is
{ x ∣ x ∈ d o m f ; f ( x ) ≤ α } \{ x\mid x\in \mathbf{dom}f;f(x)\leq \alpha \}
Sublevel sets of a convex function are convex.

### 3.1.7 Epigraph

A Epigraph of a function f : R n → R f:\mathbf{R}^n\rightarrow\mathbf{R} is
e p i f = { ( x , t ) ∣ x ∈ d o m f ; f ( x ) ≤ t } \mathbf{epi}f=\{ (x,t) \mid x\in \mathbf{dom}f;f(x)\leq t \}
is a subset of R n + 1 \mathbf{R}^{n+1} .

### 3.1.8 Jensen’s inequality and extensions

The basic inequality can be extended to
f ( θ 1 x 1 + ⋯ + θ n x n ) ≤ θ 1 f ( x 1 ) + ⋯ + θ n f ( x n ) f(\theta_1x_1+\cdots+\theta_nx_n)\leq \theta_1f(x_1)+\cdots+\theta_nf(x_n)
where ∑ i θ i = 1 , θ i > 0 , x i ∈ d o m f , f \sum_i\theta_i=1,\theta_i>0,x_i \in \mathbf{dom}f,f is convex function. For intergral, that is
f ( ∫ S p ( x ) x ) ≤ ∫ S p ( x ) f ( x ) f\left(\int_Sp(x)x\right)\leq \int_Sp(x)f(x)
It is the expectation inequality
f ( E ( x ) ) ≤ E ( f ( x ) ) f(\mathbb{E}(x))\leq\mathbb{E}(f(x))

## 3.2 Operations that preserve convexity

### 3.2.1 Nonnegative weighted sums

Suppose f i f_i is convex and w i > 0 w_i>0 ,
g x ( x ) = ∑ i w i f i ( x ) gx(x) = \sum_iw_if_i(x)
is convex.

### 3.2.2 Composition with an affine mapping

Suppose f : R n → R , A ∈ R n × m , b ∈ R n f:\mathbf{R}^n\rightarrow\mathbf{R}, A\in \mathbf{R}^{n\times m},b\in \mathbf{R}^{n} , define g : R m → R g:\mathbf{R}^m\rightarrow\mathbf{R} :
g ( x ) = f ( A x + b ) , x ∈ { x ∣ A x + b ∈ d o m f } g(x)=f(Ax+b),x\in \{ x\mid Ax+b\in \mathbf{dom}f \}
its convexity is same with f f .

### 3.2.3 Pointwise maximum and supremum

If f 1 f_1 and f 2 f_2 are convex functions then their pointwise maximum f f , defined by
f ( x ) = max ⁡ { f 1 ( x ) , f 2 ( x ) } , d o m f = d o m f 1 ∩ d o m f 2 f(x)=\max\{ f_1(x),f_2(x) \}, \mathbf{dom}f=\mathbf{dom}f_1\cap\mathbf{dom}f_2
is convex.

For each y ∈ A , f ( x , y ) y\in \mathcal{A},f(x,y) is convex in x x , the pointwise supremum
g ( x ) = sup ⁡ y ∈ A f ( x , y ) g(x)=\sup_{\mathcal{y\in A}}f(x,y)
is convex. Where d o m   g = { x ∣ ( x , y ) ∈ d o m   g , sup ⁡ y ∈ A f ( x , y ) < ∞ } \mathbf{dom}\space g=\{ x\mid (x,y)\in \mathbf{dom} \space g, \sup_{\mathcal{y\in A}}f(x,y) <\infty \} .

### 3.2.5 Minimization

If f f is convex in ( x , y ) (x,y) , and C C is a convex nonempty set, then the function
g ( x ) = inf ⁡ y ∈ C f ( x , y ) g(x)=\inf_{y\in C}f(x,y)
is convex.

### 3.2.6 Perspective of a function

If f : R n → R , f:\mathbf{R}^n\rightarrow\mathbf{R}, the perspective of f f is g : R n + 1 → R g:\mathbf{R}^{n+1}\rightarrow\mathbf{R} :
g ( x , t ) = t f ( x / t ) g(x,t)=tf(x/t)
with domian
d o m g = { ( x , t ) ∣ x / t ∈ d o m f , t > 0 } \mathbf{dom}g=\{ (x,t)\mid x/t\in \mathbf{dom}f,t>0\}
It is convex if f f is convex.

## 3.3 The conjugate function

### 3.3.1 Definition and examples

If KaTeX parse error: Undefined control sequence: \mbox at position 37: …rrow\mathbf{R},\̲m̲b̲o̲x̲{the function }… is called conjugate function if
f ∗ ( y ) = sup ⁡ x ∈ d o m f ( y T x − f ( x ) ) f^*(y) = \sup_{x\in \mathbf{dom}f}(y^Tx-f(x))
The domain of the conjugate function consists of for which the supremum is finite.

### 3.3.2 Basic properties

Fenchel’s inequality
f ( x ) + f ∗ ( y ) ≥ x T y f(x)+f^*(y)\geq x^Ty
Conjugate of the conjugate

The conjugate of the conjugate of a convex function is the original function.

Differentiable functions

Let z ∈ R n , y = ∇ f ( z ) z\in \mathbf{R}^n,y=\nabla f(z)
f ∗ ( y ) = z T ∇ f ( z ) − f ( z ) f^*(y)=z^T\nabla f(z)-f(z)
Scaling and composition with affine transformation

Conjugate of g ( x ) = a f ( x ) + b g(x)=af(x)+b is g ∗ ( y ) = a f ∗ ( y / a ) − b g^*(y)=af^*(y/a)-b .

Conjugate of g ( x ) = f ( A x + b ) g(x)=f(Ax+b) is g ∗ ( y ) = f ∗ ( A − 1 y ) − b T A − T y g^*(y)=f^*(A^{-1}y)-b^TA^{-T}y .

Sums of independent functions

If f ( x , y ) = f 1 ( x ) + f 2 ( y ) f(x,y)=f_1(x)+f_2(y) , then f ∗ ( u , v ) = f 1 ∗ ( u ) + f 2 ∗ ( v ) f^*(u,v)=f_1^*(u)+f_2^*(v) .

## 3.4 Quasiconvex functions

### 3.4.1 Definition and examples

A function f : R n → R f:\mathbf{R}^n\rightarrow\mathbf{R} is called quasiconvex (or unimodal) if its domain and all its sublevel sets
S α = { x ∈ d o m f ∣ f ( x ) < α } S_{\alpha}=\{ x\in \mathbf{dom} f \mid f(x)<\alpha \}
Are convex.

### 3.4.2 Basic properties

The extension of Jenson’s equality is: A function f f is quasiconvex if d o m f \mathbf{dom}f is convex and for x , y ∈ d o m f x,y\in \mathbf{dom}f , f ( θ x + ( 1 − θ ) y ) ≤ max ⁡ { f ( x ) , f ( y ) } f(\theta x+(1-\theta)y)\leq\max\{f(x),f(y)\}

### 3.4.3 Differentiable quasiconvex functions

First-order conditions

Suppose f : R n → R f:\mathbf{R}^n\rightarrow\mathbf{R} is differentiable. Then f f is quasiconvex if and only if dom f f is convex and for all x , y ∈ d o m f x, y ∈ \mathbf{dom}f
f ( y ) ≤ f ( x ) ⇒ ∇ f ( x ) T ( y − x ) ≤ 0 f(y)\leq f(x) \Rightarrow\nabla f(x)^T(y-x)\leq0
Second-order conditions

If f is quasiconvex, then for all x ∈ d o m f x ∈ \mathbf{dom}f , and all y ∈ R n y ∈ \mathbf{R}^n , we have
y T ∇ f ( x ) = 0 ⇒ y T ∇ 2 f ( x ) y ≥ 0 y^T\nabla f(x)=0\Rightarrow y^T \nabla ^2f(x)y\geq0

### 3.4.4 Operations that preserve quasiconvexity

Nonnegative weighted maximum

If w i > 0 , f i ( x ) w_i>0,f_i(x) is quasiconvex,
f = max ⁡ { ∑ i w i f ( x ) } f=\max\{\sum_iw_if(x) \}
is quasi convex.
f ( x ) = sup ⁡ y ∈ C { w ( y ) f ( x , y ) } f(x)=\sup_{y\in C}\{ w(y)f(x,y) \}
is quasi convex.

Minimization

If f ( x , y ) f(x,y) is quasiconvex jointly in x x and y y and C C is a convex set, then the function
g ( x ) = inf ⁡ y ∈ C f ( x , y ) g(x) = \inf_{y\in C}f(x,y)
is quasi convex.

### 3.4.5 Representation via family of convex functions

We seek a family of convex functions ϕ t : R n → R \phi_t : \mathbf{R}^n\rightarrow\mathbf{R} , indexed by t ∈ R t ∈ \mathbf{R} , with
f ( x ) ≤ t ⇔ ϕ t ≤ 0 f(x)\leq t \Leftrightarrow\phi_t\leq0

## 3.5 Log-concave and log-convex functions

### 3.5.1 Definition

A function f : R n → R f : \mathbf{R}^n\rightarrow\mathbf{R} is logarithmically concave or log-concave if $f(x) > 0$ for all x ∈ d o m f x ∈ \mathbf{dom}f and log ⁡ f \log f is concave.

if for all x , y ∈ d o m f x, y∈\mathbf{dom}f and 0 ≤ θ ≤ 1 0≤θ≤1 ,we have
f ( θ x + ( 1 − θ ) y ) ≤ f ( x ) θ f ( y ) 1 − θ f(\theta x+(1-\theta)y)\leq f(x)^{\theta}f(y)^{1-\theta}

### 3.5.2 Properties

Twice differentiable log-convex/concave functions

We conclude that f f is log-convex if and only if for all x ∈ d o m f x ∈ \mathbf{dom} f ,
f ( x ) ∇ 2 f ( x ) ⪰ ∇ f ( x ) ∇ f ( x ) T f(x)\nabla^2f(x) \succeq \nabla f(x)\nabla f(x)^T
and log-concave if and only if for all x ∈ d o m f x ∈ \mathbf{dom} f ,
f ( x ) ∇ 2 f ( x ) ⪯ ∇ f ( x ) ∇ f ( x ) T f(x)\nabla^2f(x) \preceq \nabla f(x)\nabla f(x)^T

Log-convexity and log-concavity are closed under multiplication and positive scaling.

The sum of two log-convex functions is log-convex. If f ( x , y ) f(x,y) is log-convex in x x for every y y , then
g ( x ) = ∫ f ( x , y ) d y g(x)=\int f(x,y)dy
is log-convex.

Integration of log-concave functions

If f : R n × R m → R f : \mathbf{R}^n \times \mathbf{R}^m\rightarrow\mathbf{R} is log-concave, then g ( x ) = ∫ f ( x , y ) d y g(x)=\int f(x,y)dy is log-concave.

## 3.6 Convexity with respect to generalized inequalities

### 3.6.1 Monotonicity with respect to a generalized inequality

Suppose K ⊆ R n K ⊆ \mathbf{R}^n is a proper cone with associated generalized inequality ⪯ K \preceq_K . A function f : R n → R f : \mathbf{R}^n\rightarrow\mathbf{R} is called K K -nondecreasing if
x ⪯ K y ⟹ f ( x ) ≤ f ( y ) x\preceq_Ky \Longrightarrow f(x)\leq f(y)
and K K -increasing if
x ⪯ K y , x ≠ y ⟹ f ( x ) < f ( y ) x\preceq_Ky,x\neq y \Longrightarrow f(x)< f(y)

### 3.6.2 Convexity with respect to a generalized inequality

Suppose K ⊆ R m K ⊆ \mathbf{R}^m is a proper cone with associated generalized inequality ⪯ K \preceq_K . We say f : R n → R m f : \mathbf{R}^n\rightarrow\mathbf{R}^m is K K -convex if for all KaTeX parse error: Undefined control sequence: \mbox at position 7: x, y, \̲m̲b̲o̲x̲{and}\space 0 ≤…,
f ( θ x + ( 1 − θ ) y ) ⪯ K θ f ( x ) + ( 1 − θ ) f ( y ) f(\theta x+(1-\theta)y)\preceq_K \theta f(x)+(1-\theta)f(y)
Differentiable K-convex functions

A differentiable function f f is K K -convex if and only if its domain is convex, and for all x , y ∈ d o m f x, y∈\mathbf{dom}f ,
f ( y ) ⪰ K f ( x ) + D f ( x ) ( y − x ) f(y)\succeq_Kf(x)+Df(x)(y-x)
Here D f ( x ) ∈ R m × n Df(x)\in\mathbf{R}^{m\times n} is the Jacobian matrix of f f at x x .

• 点赞
• 评论
• 分享
x

海报分享

扫一扫，分享海报

• 收藏
• 打赏

打赏

来碗拿铁️

你的鼓励将是我创作的最大动力

C币 余额
2C币 4C币 6C币 10C币 20C币 50C币
• 举报
• 一键三连

点赞Mark关注该博主, 随时了解TA的最新博文
10-10 4048

01-17 154