Directory
集合与函数(Sets and functions)
凸优化问题(Convex Optimization problem)
A conex optimization problem is of the form:
min
x
∈
D
f
(
x
)
\min_{x \in D} f(x)
x∈Dminf(x) subject to
g
i
(
x
)
<
0
,
i
=
1
,
.
.
.
,
m
h
j
(
x
)
=
0
,
j
=
1
,
.
.
.
,
r
\begin{array}{ll} g_i(x)<0, ~i=1,...,m \\ h_j(x)=0, ~j=1,...,r \end{array}
gi(x)<0, i=1,...,mhj(x)=0, j=1,...,r where
f
f
f and
g
i
g_i
gi are all convex, and
h
j
h_j
hj are affine. Any local minimizer of a convex optimization problem is a global minimizer.
凸集(Convex Sets)
定义(Definition )
Definition 2.1 凸集(Convex set): a set
C
⊆
R
n
C \subseteq \mathbb{R}^n
C⊆Rn is a convex set if for any
x
,
y
∈
C
x,y\in C
x,y∈C, we have
t
x
+
(
1
−
t
)
y
∈
C
,
∀
0
≤
t
≤
1.
tx+(1-t)y \in C, ~\forall 0\le t \le 1.
tx+(1−t)y∈C, ∀0≤t≤1.
In other words, line segment joining any two elements lies entirly in set as following picture
Then, we assume that the black line respresents vector t x + ( 1 − t ) y tx+(1-t)y tx+(1−t)y with t ∈ [ 0 , 1 ] t \in [0,1] t∈[0,1]; the vector always locates the set space composed of the linear combination of x x x and y y y, for any t t t.
Definition 2.2 Convex combination of
x
1
,
.
.
.
,
x
k
∈
R
n
:
x_1,...,x_k \in \mathbb{R}^n:
x1,...,xk∈Rn: any linear combination
θ
1
x
1
+
.
.
.
+
θ
k
x
k
,
\theta_1x_1+...+\theta_kx_k,
θ1x1+...+θkxk, with
θ
i
≥
0
,
\theta_i \ge 0,
θi≥0, and
∑
i
=
1
k
θ
i
\sum_{i=1}^{k}\theta_i
∑i=1kθi=1.
Definition 2.3 集合的凸包(Convex hull of set )
C
C
C: all convex combinations of elements in
C
C
C. The convex hull is always convex.
Definition 2.4 锥(Cone): a set
C
⊆
R
n
C\subseteq \mathbb{R}^n
C⊆Rn is a cone if for any
x
∈
C
x \in C
x∈C, we have
t
x
∈
C
tx \in C
tx∈C for all
t
≥
0
t \geq 0
t≥0.
Definition 2.5 凸锥(Convex cone): a cone that is also convex, i.e.,
x
1
,
x
2
∈
C
→
t
1
x
1
+
t
2
x
2
∈
C
f
o
r
a
l
l
t
1
,
t
2
≥
0
x_1,x_2 \in C\rightarrow t_1x_1+t_2x_2 \in C ~ for ~ all ~ t_1,t_2\geq 0
x1,x2∈C→t1x1+t2x2∈C for all t1,t2≥0
Definition 2.6 圆锥组合 Conic combination of
x
1
,
.
.
.
,
x
k
∈
R
x_1,...,x_k \in \mathbb{R}
x1,...,xk∈R: any linear combination
θ
1
x
1
+
.
.
.
+
θ
k
x
k
,
w
i
t
h
θ
≥
0
\theta_1x_1+...+\theta_kx_k, ~ with ~ \theta \geq 0
θ1x1+...+θkxk, with θ≥0
Definition 2.7 Conic hull of set
C
C
C; all conic combinations of elements in
C
C
C.
凸集的例子(Examples of convex sets)
- Empty set, point, and line are convex set.
- 范数球(Norm ball): { x : ∥ x ∥ ≤ r } \{x:\|x\| \le r\} {x:∥x∥≤r}, for given norm ∥ ⋅ ∥ \|\cdot\| ∥⋅∥, radius r r r.
- 超平面(Hyperplane): { x : a T x = b } \{x:a^Tx=b\} {x:aTx=b}, for given a , b a,b a,b.
- 半空间(Halfspace): { x : a T x ≤ b } \{ x: a^Tx \leq b\} {x:aTx≤b}.
- 仿射空间(Affine space): { x : A x = b } \{x:Ax=b\} {x:Ax=b}
- 多面体(Polyhedron): { x : A x ≤ b } \{x:Ax\leq b \} {x:Ax≤b},下面为多面体
等价问题
注意:两个问题的最优值相同,则这两个问题等价。
A. 增加约束:
min f 0 ( x ) s . t . l i ≤ x i ≤ u i , i = 1 , . . . , n . \begin{array}{ll} & \min ~ f_0(x) \\ & s.t. ~~~ l_i \le x_i \le u_i, i=1,...,n. \end{array} min f0(x)s.t. li≤xi≤ui,i=1,...,n.
The above inequality constraint can be swith as the following inequations:
s
.
t
.
l
i
−
x
i
≤
0
,
i
=
1
,
.
.
.
,
n
.
x
i
−
u
i
≤
,
i
=
1
,
.
.
.
,
n
.
\begin{array}{ll} & s.t. & l_i - x_i \le 0, &i=1,...,n. \\ & & x_i - u_i \le, &i=1,...,n. \end{array}
s.t.li−xi≤0,xi−ui≤,i=1,...,n.i=1,...,n.
B. 增加次方:
min x ∥ A x − b ∥ 2 \min_x \| \mathbf{A} x - b \|_2 xmin∥Ax−b∥2 由于 x x x在函数内单调递增,故可等价于: min x ∥ A x − b ∥ 2 2 \min_x \| \mathbf{A} x - b \|_2^2 xmin∥Ax−b∥22
C. 消除约束等式约束:
Theoretically, the reduction of the constraints of equations makes higher efficiency, so we need to eliminate the equation constraints as possible as we can.
An typical example:
Assume that
z
∈
R
k
,
ζ
:
R
k
→
R
n
z\in \mathbb{R}^k, \zeta:\mathbb{R}^k \rightarrow\mathbb{R}^n
z∈Rk,ζ:Rk→Rn,
min
f
0
(
x
)
s
.
t
.
f
i
(
x
)
≤
0
,
i
=
1
,
.
.
.
,
n
x
=
ζ
(
z
)
\begin{array}{lll} & \min ~ f_0(x) \\ & s.t. ~~ f_i(x) \le 0, i=1,...,n \\ & \quad ~~~ x = \zeta(z) \end{array}
min f0(x)s.t. fi(x)≤0,i=1,...,n x=ζ(z) Then, the above problem can be reformulate as the function of variable
z
z
z,i.e.,
min
f
0
(
ζ
(
z
)
)
s
.
t
.
f
i
(
ζ
(
z
)
)
≤
0
,
i
=
1
,
.
.
.
,
n
\begin{array}{ll} & \min ~f_0(\zeta(z)) \\ & s.t. ~~ f_i(\zeta(z)) \le 0, i=1,...,n \end{array}
min f0(ζ(z))s.t. fi(ζ(z))≤0,i=1,...,n Finally,
x
∗
x^*
x∗ can be obtained from the
x
∗
=
ζ
(
z
∗
)
x^* = \zeta(z^*)
x∗=ζ(z∗).