Chapter2 Convex Set
2.1 Affine, Convex Set
2.1.1 Line, Line Segment
Suppose x 1 , x 2 x_1,x_2 x1,x2 are two points in R n \mathbf{R}^n Rn,
y = θ x 1 + ( 1 − θ ) x 2 , θ ∈ R y=\theta x_1+(1-\theta)x_2,\theta\in\mathbf{R} y=θx1+(1−θ)x2,θ∈R
is a line. If θ ∈ [ 0 , 1 ] \theta\in[0,1] θ∈[0,1], y y y is a line segment.
2.1.2 Affine Set
A set C ⊆ R n C\subseteq\mathbf{R}^n C⊆Rn is affine if for any x 1 , x 2 ∈ C , θ ∈ R x_1,x_2\in C,\theta \in \mathbf{R} x1,x2∈C,θ∈R, θ x 1 + ( 1 − θ ) x 2 ∈ C \theta x_1+(1-\theta)x_2\in C θx1+(1−θ)x2∈C.
Affine combination means for x 1 , x 2 , … , x k x_1,x_2,\ldots,x_k x1,x2,…,xk and ∑ i θ i = 1 \sum_i \theta_i=1 ∑iθi=1, ∑ i k θ i x i ∈ C \sum_i^k{\theta_ix_i}\in C ∑ikθixi∈C. If x i ∈ C x_i \in C xi∈C, then its affine combination ∈ C \in C ∈C.
A affine set C C C can be expressed as C = V + x 0 = { v + x 0 ∣ v ∈ V } C=V+x_0=\{v+x_0|v\in V\} C=V+x0={v+x0∣v∈V} ( a subspace plus an offset). The dimension of an affine set is the dimension of subspace V = C − x 0 , x 0 ∈ C V=C-x_0, x_0\in C V=C−x0,x0∈C.
Affine hull of a set
C
C
C is the set of all affine combination of
x
i
∈
C
x_i\in C
xi∈C, denoted as
a
f
f
C
=
{
∑
i
k
θ
i
x
i
∣
x
i
∈
C
,
∑
i
θ
i
=
1
}
{\rm \mathbf{aff}}\space C=\{\sum_i^k{\theta_ix_i}|x_i \in C, \sum_i\theta_i=1\}
aff C={i∑kθixi∣xi∈C,i∑θi=1}
affine hull is the smallest affine set that contains
C
C
C, which is if set
S
S
S is a affine set that contains
C
C
C, the
a
f
f
C
⊆
S
{\mathbf{aff}} \space C \subseteq S
aff C⊆S.
2.1.3 Affine dimension, relative interior
Affine dimension of a set C C C is the dimension of its affine hul,
Relative interior is
r
e
l
i
n
t
C
=
{
x
∈
C
∣
B
(
x
,
r
)
∩
a
f
f
C
⊆
C
f
o
r
s
o
m
e
r
>
0
}
\mathbf{relint}\space C=\{x\in C\mid B(x,r)\cap\mathbf{aff}\space C\subseteq {\rm C \space for\space some \space} r>0\}
relint C={x∈C∣B(x,r)∩aff C⊆C for some r>0}
where
B
(
x
,
r
)
=
{
y
∣
∣
∣
y
−
x
∣
∣
≤
r
}
B(x,r)=\{ y\mid ||{y-x}|| \leq r \}
B(x,r)={y∣∣∣y−x∣∣≤r}
2.1.4 Convex sets
Set C C C is convex if for x 1 , x 2 ∈ C , θ ∈ [ 0 , 1 ] x_1,x_2\in C, \theta \in [0,1] x1,x2∈C,θ∈[0,1], we have θ x 1 + ( 1 − θ ) x 2 ∈ C \theta x_1+(1-\theta)x_2\in C θx1+(1−θ)x2∈C.
A convex combination means the form ∑ i k θ i x i \sum_i^k{\theta_ix_i} ∑ikθixi, where ∑ i θ i = 1 , θ i > 0 \sum_i\theta_i=1, \theta_i>0 ∑iθi=1,θi>0.
Convex hull of a set
C
C
C is the set of all convex combination of
x
i
∈
C
x_i\in C
xi∈C, denoted as
c
o
n
v
C
=
{
∑
i
k
θ
i
x
i
∣
x
i
∈
C
,
∑
i
θ
i
=
1
,
θ
i
>
0
}
{\rm \mathbf{conv}}\space C=\{\sum_i^k{\theta_ix_i}|x_i \in C, \sum_i\theta_i=1,\theta_i>0\}
conv C={i∑kθixi∣xi∈C,i∑θi=1,θi>0}
Convex hull is the smallest convex set that contains
C
C
C, which is if set
S
S
S is a convex set that contains
C
C
C, the
c
o
n
v
C
⊆
S
{\mathbf{conv}} \space C \subseteq S
conv C⊆S.
2.1.5 Cones
A set C C C is a cone if for x ∈ C , θ > 0 x\in C,\theta>0 x∈C,θ>0, we have θ x ∈ C \theta x\in C θx∈C. A set is convex cone if for x 1 , x 2 ∈ C ; θ 1 , θ 2 ≥ 0 x_1,x_2 \in C;\theta_1,\theta_2\geq0 x1,x2∈C;θ1,θ2≥0, we have θ 1 x 1 + θ 2 x 2 ∈ C \theta_1x_1+\theta_2x_2\in C θ1x1+θ2x2∈C.
A conic combination of x i ∈ C x_i\in C xi∈C is ∑ i θ i x i \sum_i\theta_ix_i ∑iθixi with θ i ≥ 0 \theta_i\geq0 θi≥0.
2.2 Important examples
2.2.1 hyperplane, halfspaces
A hyperplane is a set of form
{
x
∣
a
T
x
=
b
}
\{x\mid a^Tx=b\}
{x∣aTx=b}
where
a
∈
R
n
,
a
≠
0
,
b
∈
R
a\in \mathbf{R}^n,a\neq0,b\in \mathbf{R}
a∈Rn,a=0,b∈R. It can be represented as
{
x
∣
a
T
(
x
−
x
0
)
=
0
}
=
x
0
+
a
⊥
\{ x \mid a^T(x-x_0) =0 \}=x_0 + a^{\bot}
{x∣aT(x−x0)=0}=x0+a⊥
Where
a
⊥
a^{\bot}
a⊥ means
{
x
∣
a
T
x
=
0
}
\{ x\mid a^Tx =0 \}
{x∣aTx=0}. Shown as Fig 2.6.
A hyperplane devide space into 2 halfspaces:
{
x
∣
a
T
x
≤
b
}
\{ x\mid a^Tx\leq b\}
{x∣aTx≤b}
or
{
x
∣
a
T
(
x
−
x
0
)
≤
0
}
\{ x\mid a^T(x-x_0)\leq 0\}
{x∣aT(x−x0)≤0}
2.2.2 Euclidean balls, ellipsoids
A Euclidean balls in
R
n
\mathbf{R}^n
Rn is
B
(
x
c
,
r
)
=
{
x
∣
∣
∣
x
−
x
c
∣
∣
2
≤
r
}
=
{
x
∣
(
x
−
x
c
)
T
(
x
−
x
c
)
≤
r
2
}
=
{
x
c
+
r
u
∣
∣
∣
u
∣
∣
2
≤
1
}
B(x_c,r)=\{ x \mid ||x-x_c||_2\leq r\}=\{ x \mid (x-x_c)^T(x-x_c)\leq r^2\}=\{x_c+ru\mid ||u||_2\leq1\}
B(xc,r)={x∣∣∣x−xc∣∣2≤r}={x∣(x−xc)T(x−xc)≤r2}={xc+ru∣∣∣u∣∣2≤1}
where
r
>
0
,
∣
∣
u
∣
∣
2
=
(
u
T
u
)
1
2
r>0, ||u||_2=(u^Tu)^{\frac{1}{2}}
r>0,∣∣u∣∣2=(uTu)21.
A Ellipsoid is
E
=
{
x
∣
(
x
−
x
c
)
T
P
−
1
(
x
−
x
c
)
≤
1
}
=
{
x
c
+
A
u
∣
∣
∣
u
∣
∣
2
≤
1
}
\mathcal{E}=\{ x|(x-x_c)^TP^{-1}(x-x_c)\leq 1 \}=\{ x_c+Au\mid ||u||_2\leq1 \}
E={x∣(x−xc)TP−1(x−xc)≤1}={xc+Au∣∣∣u∣∣2≤1}
where
P
=
P
T
,
P
≻
0
,
A
P=P^T,P\succ0, A
P=PT,P≻0,A square and nonsingular.
2.2.3 Norm ball, Norm cone
A norm ball is { x ∣ ∣ ∣ x − x c ∣ ∣ ≤ r } \{ x\mid ||x-x_c|| \leq r \} {x∣∣∣x−xc∣∣≤r}, where ∣ ∣ ⋅ ∣ ∣ ||\cdot|| ∣∣⋅∣∣ is any norm.
A norm cone is { ( x , t ) ∣ ∣ ∣ x ∣ ∣ ≤ t } ⊆ R n + 1 \{ (x,t) \mid ||x|| \leq t \} \subseteq\mathbf{R}^{n+1} {(x,t)∣∣∣x∣∣≤t}⊆Rn+1, where ∣ ∣ ⋅ ∣ ∣ ||\cdot|| ∣∣⋅∣∣ is any norm.
2.2.4 Polyhedra
A polyhedron is thus the intersection of a finite number of halfspaces and hyperplanes:
P
=
{
a
j
T
x
≤
b
j
,
j
=
1
,
⋯
,
m
;
c
i
T
x
=
d
i
,
i
=
1
,
⋯
,
p
}
\mathcal{P}=\{ a^T_jx\leq b_j,j=1,\cdots,m;c^T_ix =d_i, i=1,\cdots,p \}
P={ajTx≤bj,j=1,⋯,m;ciTx=di,i=1,⋯,p}
It can be represented as compact form:
P
=
{
x
∣
A
T
x
⪯
b
;
C
T
x
=
d
}
\mathcal{P}=\{ x\mid A^Tx\preceq b;C^Tx=d \}
P={x∣ATx⪯b;CTx=d}
where
A
=
[
a
1
T
,
⋯
,
a
m
T
]
,
C
=
[
c
1
T
,
⋯
,
c
p
T
]
A = [a_1^T, \cdots,a_m^T],C=[c_1^T,\cdots,c_p^T]
A=[a1T,⋯,amT],C=[c1T,⋯,cpT].
Simplexes are special polyhedron:
C
=
c
o
n
v
{
v
0
,
⋯
,
v
k
}
=
{
∑
i
=
0
k
θ
i
v
i
∣
θ
⪰
0
;
1
θ
=
1
}
C={\mathbf{conv} \space }\{ v_0,\cdots,v_k \}=\{ \sum_{i=0}^k\theta_iv_i \mid \theta \succeq0;\mathbf{1}\theta=1 \}
C=conv {v0,⋯,vk}={i=0∑kθivi∣θ⪰0;1θ=1}
where
v
i
−
x
0
v_i-x_0
vi−x0 are linearly independent.
A polyhedron can be expressed in the form of convex hull add conic hull:
{
∑
i
k
θ
i
x
i
∣
∑
i
m
θ
i
=
1
;
θ
i
>
0
}
\{ \sum_i^{k}\theta_ix_i\mid\sum_i^m\theta_i=1;\theta_i>0 \}
{i∑kθixi∣i∑mθi=1;θi>0}
where
m
<
k
m<k
m<k.
2.2.5 positive semidefinite cone
S n = { X ∈ R n × n ∣ X = X T } S + n = { X ∈ S n ∣ X ⪰ 0 } S + + n = { X ∈ S + n ∣ X ≻ 0 } \begin{aligned} \mathbf{S}^n &= \{ X\in \mathbf{R}^{n\times n} \mid X=X^T \} \\ \mathbf{S}^n_+ &= \{ X\in \mathbf{S}^{n} \mid X\succeq0 \} \\ \mathbf{S}^n_{++} &= \{ X\in \mathbf{S}^{n}_+ \mid X\succ0\} \end{aligned} SnS+nS++n={X∈Rn×n∣X=XT}={X∈Sn∣X⪰0}={X∈S+n∣X≻0}
S + n \mathbf{S}^n_+ S+n is a convex cone.
2.3 Operations that preserve convexity
2.3.1 Intersection
If
S
1
S_1
S1 and
S
2
S_2
S2 are convex, then
S
1
∩
S
2
S_1\cap S_2
S1∩S2 is convex. It holds for infinite intersections:
S
α
∈
S
⇒
⋂
S
α
∈
S
S_{\alpha} \in S\Rightarrow\bigcap S_{\alpha} \in S
Sα∈S⇒⋂Sα∈S
2.3.2 Affine
Let the function f : R n → R m f:\mathbf{R}^n\rightarrow\mathbf{R}^m f:Rn→Rm is affine, S S S is convex set, then { f ( x ) ∣ x ∈ S } \{ f(x)\mid x\in S \} {f(x)∣x∈S} is convex.
2.3.3 Linear fractional, perspective functions
Define perspective function P : R n + 1 → R n P:\mathbf{R}^{n+1}\rightarrow\mathbf{R}^{n} P:Rn+1→Rn, with d o m P = R n × R + + , P ( x , t ) = x t \mathbf{dom} P=\mathbf{R}^n\times\mathbf{R}_{++},P(x,t)=\frac{x}{t} domP=Rn×R++,P(x,t)=tx.
A linear-fractional function is formed by composing the perspective function with an affine function. Suppose
g
(
x
)
=
[
A
c
T
]
x
+
[
b
d
]
g(x)= \left[ \begin{array}{cc} A \\ c^T \end{array} \right]x+\left[ \begin{array}{cc} b \\ d \end{array} \right]
g(x)=[AcT]x+[bd], where
A
∈
R
m
×
n
,
c
∈
R
n
,
b
∈
R
m
,
d
∈
R
A\in \mathbf{R}^{m\times n},c\in\mathbf{R}^{n},b\in\mathbf{R}^{m},d\in\mathbf{R}
A∈Rm×n,c∈Rn,b∈Rm,d∈R. Then
f
f
f is
P
∘
f
P\circ f
P∘f:
f
=
A
x
+
b
c
T
x
+
d
,
d
o
m
f
=
{
x
∣
c
T
x
+
d
>
0
}
f=\frac{Ax+b}{c^Tx+d}, \space \mathbf{dom}f = \{ x| c^Tx+d>0\}
f=cTx+dAx+b, domf={x∣cTx+d>0}
it can be represented as
f
(
x
)
=
P
−
1
(
x
)
(
Q
P
(
x
)
)
f(x)=\mathcal{P}^{-1}(x)(Q\mathcal{P}(x))
f(x)=P−1(x)(QP(x)), where
P
(
x
)
=
{
t
(
x
,
1
)
∣
t
>
0
}
\mathcal{P}(x)=\{ t(x,1) \mid t>0 \}
P(x)={t(x,1)∣t>0}.
2.4 Generalized inequalities
2.4.1 Proper cones, generalized inequalities
A proper cone is a cone convex, close, solid and pointed.
Generalized inequalities y ⪯ K x y\preceq_Kx y⪯Kx means that x − y ∈ K x-y\in K x−y∈K. When K = R + n K=\mathbf{R}_+^n K=R+n, we drop symbol K K K. So as K = S + n K=\mathbf{S}_+^n K=S+n.
2.4.2 Minimum, minimal elements
x ∈ S x\in S x∈S is the minimum element if for y ∈ S y\in S y∈S, then x ⪯ K y x\preceq_Ky x⪯Ky.
x ∈ S x\in S x∈S is the minimal element if y ∈ S , x ⪯ K y y\in S,x\preceq_Ky y∈S,x⪯Ky only if y = x y=x y=x.
2.5 Separating and supporting hyperplanes
2.5.1 Separating hyperplane theorem
Separating hyperplane theorem: Suppose C C C and D D D are nonempty convex sets and C ∩ D = ∅ C∩D=∅ C∩D=∅. Then there exist a ≠ 0 a\neq0 a=0 and b b b such that. a T x ≤ b a^Tx≤b aTx≤b for all x ∈ C x∈C x∈C and a T x ≥ b a^Tx≥b aTx≥b for all x ∈ D x∈D x∈D.
The strict seperation is θ T x < b \theta^Tx<b θTx<b for all x ∈ C x∈C x∈C and a T x > b a^Tx>b aTx>b for all x ∈ D x∈D x∈D.
2.5.2 Supporting hyperplane
Suppose
C
⊆
R
n
C ⊆ R_n
C⊆Rn, and
x
0
x0
x0 is a point in its boundary
b
d
C
\mathbf{bd}\space C
bd C,
x
0
∈
b
d
C
=
c
l
C
\
i
n
t
C
x_0 ∈ \mathbf{bd}\space C = \mathbf{cl}\space C \backslash \mathbf{int}\space C
x0∈bd C=cl C\int C
If.
a
≠
0
a\neq0
a=0 satisfies
a
T
x
≤
a
T
x
0
a^Tx≤a^Tx_0
aTx≤aTx0 for all
x
∈
C
x∈C
x∈C,then the hyperplane
{
x
∣
a
T
x
=
a
T
x
0
}
\{x|a^Tx=a^Tx_0\}
{x∣aTx=aTx0} is called a supporting hyperplane to
C
C
C at the point
x
0
x_0
x0.
2.6 Dual cones, generalized inequalities
2.6.1 Dual cones
If K K K is a cone, then K ∗ = { y ∣ x T y ≥ 0 , x ∈ K } K^*=\{ y\mid x^Ty\geq0,x\in K \} K∗={y∣xTy≥0,x∈K} is the dual cone of K K K, it is always convex.
2.6.2 Dual generalized inequalities
Dual generalized inequalities is the one of K ∗ K^* K∗.