Convex Optimization 读书笔记 (7)

Chapter8: Geometric problems

8.1 Projection on a set

The distance of a point $x_0\in {\mathbf{R}}^n$ to a closed set $C\subseteq {\mathbf{R}}^n$, in the norm $\Vert \cdot \Vert$, is defined as
d i s t ( x 0 , C ) = inf ⁡ { ∣ ∣ x 0 − x ∣ ∣ ∣ x ∈ C } {\rm \bold{dist}}(x_0,C)=\inf\{ ||x_0-x|| \mid x\in C \} dist(x0​,C)=inf{∣∣x0​−x∣∣∣x∈C}We refer to any point $z\in C$ which is closest to $x_0 $ satisfies $\Vert z-x_0\Vert =dist(x_0,C)$, as a projection of $x_0$ on $C$.

8.1.1 Projecting a point on a convex set

If $C$ is convex, then we can compute the projection $P_C(x_0)$ and the distance $\mathbf{d}\mathbf{i}\mathbf{s}\mathbf{t}(x_0,C)$ by solving a convex optimization problem:
$$\begin{array}{rl}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& ||x-x_0||\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& f_i(x)\le 0,i=1,...m\\ & Ax=b\end{array}$$

8.1.2 Separating a point and a convex set

If $P_C(x_0)$ denotes the Euclidean projection of $x_0$ on $C$, where $x_0\notin C$, then the hyperplane
$$(P_C(x_0)-x_0{)}^T(x-\frac{1}{2}(x_0+P_C(x_0)))=0$$(strictly) separates $x_0$ from $C$.

In other norms, the clearest link between the projection problem and the separating hyperplane problem is via Lagrange duality.

8.1.3 Projection and separation via indicator and support functions

Define indicator function $I_C$ and the support function $S_C$ of the set $C$:
$$\begin{gathered} S_C(x)=\underset{y\in C}{\sup }{x}^{T}y \\ {I}_C(x)=\{\begin{array}{rc}0,& x\in C\\ \infty ,& x\notin C\end{array} \end{gathered}$$The problem of projecting $x_0$ on a closed convex set $C$ can be expressed compactly as
$$\begin{array}{rl}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& ||x-x_0||\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& I_C(x)\le 0\end{array}$$The dual function of this problem is
$$g(z,\lambda )=\{\begin{array}{rc}{z}^T{x}_0-S_C(z),& ||z|{|}_{\ast }\le 1,\lambda \ge 0\\ -\infty ,& \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}$$so we obtain the dual problem
$$\begin{array}{rl}\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& z^T{x}_0-S_C(z)\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& ||z|{|}_{\ast }\le 1\end{array}$$

8.2 Distance between sets

The distance between two sets $C$ and $D$, in a norm $\Vert \cdot \Vert$, is defined as
d i s t ( C , D ) = inf ⁡ { ∣ ∣ x − y ∣ ∣ ∣ x ∈ C , y ∈ D } {\rm \bold{dist}}(C,D)=\inf\{ ||x-y||\mid x\in C,y\in D \} dist(C,D)=inf{∣∣x−y∣∣∣x∈C,y∈D}

8.2.1 Computing the distance between convex sets

We can find $\mathbf{d}\mathbf{i}\mathbf{s}\mathbf{t}(C,D)$ by solving the convex optimization problem
$$\begin{array}{rl}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& ||x-y||\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& f_i(x)\le 0,i=1,...m\\ & g_i(y)\le 0,i=1,...p\end{array}$$

8.2.2 Separating convex sets

First express the distance between convex sets in the following equivalent form:
$$\begin{array}{rl}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& ||w||\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& f_i(x)\le 0,i=1,...m\\ & g_i(y)\le 0,i=1,...p\\ & x-y=w\end{array}$$The dual function is
$$\begin{array}{rl}g(\lambda ,z,\mu )& =\underset{x,y,w}{\inf }(||w||+\sum _{i=1}^m{\lambda }_i{f}_i(x)+\sum _{i=1}^p{\mu }_i{g}_i(x)+z^T(x-y-z))\\ & =\{\begin{array}{ll}{\inf }_x({\sum }_{i=1}^m{\lambda }_i{f}_i(x)+z^{T}x)+{\inf }_y({\sum }_{i=1}^p{\mu }_i{g}_i(x)-z^{T}y)& ||z|{|}_{\ast }\le 1,\\ -\infty & \mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}\end{array}$$
which results in the dual problem
$$\begin{array}{rl}\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& \underset{x}{\inf }(\sum _{i=1}^m{\lambda }_i{f}_i(x)+z^{T}x)+\underset{y}{\inf }(\sum _{i=1}^p{\mu }_i{g}_i(x)-z^{T}y)\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& ||z|{|}_{\ast }\le 1\\ & \lambda ⪰0,\mu ⪰0\end{array}$$

8.2.3 Distance and separation via indicator and support functions

The problem of finding the distance between two convex sets can be posed as the convex problem
$$\begin{array}{rl}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& ||x-y||\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& I_C(x)\le 0\\ & I_D(x)\le 0\end{array}$$The dual of this problem is
$$\begin{array}{rl}\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& -S_C(-z)-S_D(z)\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& ||z|{|}_{\ast }\le 1\end{array}$$

8.3 Euclidean distance and angle problems

Suppose $a_1,...,a_n$ is a set of vectors in ${\mathbf{R}}^n$, which we assume (for now) have known Euclidean lengths
$$l_1=||a_1|{|}_2,...,l_n=||a_n|{|}_2$$We will refer to the set of vectors as a configuration, or, when they are independent, a basis.

8.3.1 Gram matrix and realizability

The Gram matrix of vectors { a 1 , . . . , a n } \{ a_1,...,a_n \} {a1​,...,an​} is
$$G=A^{T}A,A=[a_1\cdots a_n]$$which is $G_{ij}=a_i^T{a}_j,G_{ii}=l_i^2$. The distance between $a_i$ and $a_j$ is
$$d_{ij}=||a_i-a_j|{|}_2=(G_{ii}+G_{jj}-2G_{ij}{)}^{\frac{1}{2}}$$The correlation coefficient ${\rho }_{ij}$ between (nonzero) $a_i$ and $a_j$ is given by
$${\rho }_{ij}=\frac{a_i^T{a}_j}{||a_i|{|}_2||a_j|{|}_2}=\frac{G_{ij}}{\sqrt{G_{ii}}\sqrt{G_{jj}}}$$The angel ${\theta }_{ij}$ is
$${\theta }_{ij}={\cos }^{-1}{\rho }_{ij}$$A set of lengths, distances, and angles (or correlation coefficients) is realizable if and only if the associated Gram matrix $G$ is positive semidefinite, and has diagonal elements $l_1^2,...,l_n^2$.

8.3.2 Problems involving angles only

Suppose we only care about the angles (or correlation coefficients) between the vectors, and do not specify the lengths or distances between them. Then the gram matrix
$$G=\mathbf{d}\mathbf{i}\mathbf{a}\mathbf{g}(l)C\mathbf{d}\mathbf{i}\mathbf{a}\mathbf{g}(l)$$where $C_{ij}=\cos {\theta }_{ij}$.

8.3.3 Euclidean distance problems

In a Euclidean distance problem, we are concerned only with the distances between the vectors, $d_{ij}$ , and do not care about the lengths of the vectors, or about the angles between them.
A Euclidean distance matrix is $D\in {\mathbf{S}}^n$ with nonnegative elements, zero diagonal and satisfied
$$G=(z{1}^T+1z^T-D)/2⪰0\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{s}\mathrm{o}\mathrm{m}\mathrm{e}z⪰0$$where $D_{ij}=d_{ij}^2$.
In summary, a matrix $D\in {\mathbf{S}}^n$ is a Euclidean distance matrix if and only if
$$\begin{gathered} D_{ii}=0,i=1,...,n{D}_{ij}\ge 0,i,j=1,...,n \\ (I-\frac{1}{n}11^T)D(I-\frac{1}{n}11^T)⪯0 \end{gathered}$$

8.4 Extremal volume ellipsoids

8.4.1 The Lowner-John ellipsoid

The minimum volume ellipsoid that contains a set $C$ is called the Lowner-John ellipsoid of the set $C$, and is denoted ${\mathcal{E}}_{lj}$. First denote ellipsoid as
E = { v ∣ ∣ ∣ A v + b ∣ ∣ 2 ≤ 1 } \mathcal{E}=\{ v\mid ||Av+b||_2\leq1 \} E={v∣∣∣Av+b∣∣2​≤1}Computing the minimum volume ellipsoi containing $C$ can be expressed as
$$\begin{array}{rl}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& \log \det A^{-1}\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& \underset{v\in C}{\sup }||Av+b|{|}_2\le 1\end{array}$$

8.4.2 Maximum volume inscribed ellipsoid

First denote ellipsoid as
E = { B u + d ∣ ∣ ∣ u ∣ ∣ 2 ≤ 1 } \mathcal{E}=\{ Bu+d\mid ||u||_2\leq1 \} E={Bu+d∣∣∣u∣∣2​≤1}We now consider the problem of finding the ellipsoid of maximum volume that lies inside a convex set $C$
$$\begin{array}{rl}\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& \log \det B\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& \underset{||u|{|}_2\le 1}{\sup }{I}_C(Bu+d)\le 0\end{array}$$

8.4.3 Affine invariance of extremal volume ellipsoids

If ${\mathcal{E}}_{\mathrm{l}\mathrm{j}}$ is the Lowner-John ellipsoid of $C$, and $T\in {\mathbf{R}}^{n\times n}$ is nonsingular, then the Lowner-John ellipsoid of $TC$ is $T{\mathcal{E}}_{\mathrm{l}\mathrm{j}}$. A similar result holds for the maximum volume inscribed ellipsoid.

8.5 Centering

8.5.1 Chebyshev center

The depth of a point $x\in C$ is defined as
$$\mathbf{d}\mathbf{e}\mathbf{p}\mathbf{t}\mathbf{h}(x,C)=\mathbf{d}\mathbf{i}\mathbf{s}\mathbf{t}(x,{\mathbf{R}}^n\backslash C)$$A Chebyshev center of the set $C$ is defined as any point of maximum depth in $C$:
$$x_{\mathrm{c}\mathrm{h}\mathrm{e}\mathrm{b}}(C)=\arg \max \mathbf{d}\mathbf{e}\mathbf{p}\mathbf{t}\mathbf{h}(x,C)=\arg \max \mathbf{d}\mathbf{i}\mathbf{s}\mathbf{t}(x,{\mathbf{R}}^n\backslash C).$$

8.5.2 Maximum volume ellipsoid center

As an extension of this idea, we define the maximum volume ellipsoid center of $C$, denoted $x_{\mathrm{m}\mathrm{v}\mathrm{e}}$, as the center of the maximum volume ellipsoid that lies in $C$.

8.5.3 Analytic center of a set of inequalities

The analytic center $x_{\mathrm{a}\mathrm{c}}$ of a set of convex inequalities and linear equalities
$$f_i(x)\le 0,i=1,...,m,Fx=g$$is defined as
$$\begin{array}{rl}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}& -\sum _{i=1}^m\log (-f_i(x))\\ \mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}& Fx=g\end{array}$$

8.6 Classification

In pattern recognition and classification problems we are given two sets of points in ${\mathbf{R}}^n$, { x 1 , . . . , x N } \{x_1,...,x_N\} {x1​,...,xN​} and { y 1 , . . . , y M } \{y_1,...,y_M\} {y1​,...,yM​}, define a function
$$\begin{gathered} f(x_i)>0,i=1,...,N \\ f(y_i)<0,i=1,...,M \end{gathered}$$If these inequalities hold, we say that $f$, or its 0-level set { x ∣ f ( x ) = 0 } \{x \mid f(x) = 0\} {x∣f(x)=0}, separates, classifies, or discriminates the two sets of points.

8.6.1 Linear discrimination

In linear discrimination, we seek an affine function $f(x)=a^{T}x-b$ that classifies the points
$$a^T{x}_i-b>0,i=1,...,N,a^T{y}_i-b<0,i=1,...,M.$$Since the strict inequalities are homogeneous in $a$ and $b$, they are feasible if and only if the set of nonstrict linear inequalities
$$a^T{x}_i-b\ge 1,i=1,...,N,a^T{y}_i-b\le -1,i=1,...,M$$

8.6.2 Nonlinear discrimination

We can just as well seek a nonlinear function $f$, from a given subspace of functions, that is positive on one set and negative on another:
$$f(x_i)>0,i=1,...,N,f(y_i)<0,i=1,...,M.$$

8.7 Placement and location

We have N points in ${\mathbf{R}}^2$ or ${\mathbf{R}}^3$, and a list of pairs of points that must be connected by links. The positions of some of the $N$ points are fixed; our task is to determine the positions of the remaining points, i.e., to place the remaining points. The problem is to minimize
$$\sum _{(i,j)\in \mathcal{A}}{f}_{ij}(x_i,x_j)$$where $\mathcal{A}$ is the set of all links in the graph, and $f_{ij}:{\mathbf{R}}^k\times {\mathbf{R}}^k\to \mathbf{R}$ is a cost function associated with arc $(i,j)$.

8.7.1 Linear facility location problems

In the simplest version of the problem the cost associated with arc $(i,j)$ is the distance between nodes $i$ and $j:f_{ij}(x_i,x_j)=\Vert x_i-x_j\Vert$, i.e., we minimize
$$\sum _{(i,j)\in \mathcal{A}}\Vert x_i-x_j||$$

8.7.2 Placement constraints

We can impose a constraint that limits the points $x_1,...,x_p$ (say) to lie in a bounding box with perimeter not exceeding $P_{max}$, by adding the constraints
$$u⪯xi⪯v,i=1,...,p,21^T(v-u)\le P_{max},$$where $u,v$ are additional variables.

8.7.3 Nonlinear facility location problems

More generally, we can associate a cost with each arc that is a nonlinear increasing function of the length
$$\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{z}\mathrm{e}\sum _{i<j}{w}_{ij}h(||x_i-x_j||)$$where $h$ is an increasing (on ${\mathbf{R}}^{+}$) and convex function, and $w_{ij}\ge 0$.

8.7.4 Location problems with path constraints

8.8 Floor planning

A floor planning problem can be considered an extension of a placement problem in two ways:

• The objects to be placed are rectangles or boxes aligned with the axes (as opposed to points), and must not overlap.
• Each rectangle or box to be placed can be reconfigured, within some limits. For example we might fix the area of each rectangle, but not the length and height separately.
  In all floor planning problems, we require that the cells lie inside the bounding rectangle
  $$x_i\ge 0,y_i\ge 0,x_i+w_i\le W,y_i+h_i\le H,i=1,...,N.$$We also require that the cells do not overlap, except possibly on their boundaries:
  $$\mathbf{i}\mathbf{n}\mathbf{t}(C_i\cap C_j)=\varnothing \mathrm{f}\mathrm{o}\mathrm{r}i\ne j.$$
  

8.8.1 Relative positioning constraints

The idea of relative positioning constraints is to specify, for each pair of cells, one of the four possible relative positioning conditions, i.e., left, right, above, or below. One simple method to specify these constraints is to give two relations on { 1 , . . . , N } : L \{1,...,N\}: \mathcal{L} {1,...,N}:L (meaning ‘left of’) and {1,…,N}: \mathcal{B} (meaning ‘below’). We then impose the constraint that $C_i$ is to the left of $C_j$ if $(i,j)\in \mathcal{L}$, and $C_i$ is below $C_j$ if $(i,j)\in \mathcal{B}$. This yields the constraints
$$x_i+w_i\le x_j\mathrm{f}\mathrm{o}\mathrm{r}(i,j)\in \mathcal{L},y_i+h_i\le y_j\mathrm{f}\mathrm{o}\mathrm{r}(i,j)\in \mathcal{B}$$We can use $\mathcal{H},\mathcal{V}$ to represent inequalities:
$$x_i+w_i\le x_j\mathrm{f}\mathrm{o}\mathrm{r}(i,j)\in \mathcal{H},y_i+h_i\le y_j\mathrm{f}\mathrm{o}\mathrm{r}(i,j)\in \mathcal{V}$$

8.8.2 Floor planning via convex optimization

We impose the bounding box constraints and the relative positioning constraints, which are linear inequalities.

8.8.3 Floor planning via geometric programming

The floor planning problem can also be formulated as a geometric program in the variables $x_i,y_i,w_i,h_i,W,H$.