Optimization Week 6: Semidefinite programming (SDP)

1 Linear algebra review

Square matrix eigenvalue and eigenvectors

Square matrix $M$, has eigenvector $v$ and eigenvalue $\lambda$ if
$$Mv=\lambda v$$
Compute $\lambda$
$$det|M-\lambda I|=0$$

Symmetric matrix

$M$ is symmetric $\Rightarrow$ eigen values are real, and eigen vectors are orthogonal. This is consistent with the theory: symmetric matrices have orthogonal eigenvectors, where as non-symmetric matrices do not.

Spectral decomposition of symmetric matrix

$$M=\sum {\lambda }_i{v}_i{v}_i^T$$
$$M=V\Lambda V^T$$
$v_i$ is normalized eigenvector, orthogonal to each other.

2 Positive semidefinite matrix (PSD) (must be Hermitian)

Definition of PSD:

• $\iff$ ${\lambda }_i\ge 0$ and $M=M^T$.
• $\iff$ $x^{T}Mx\ge 0$, $\forall x\in {\mathbb{R}}^n$
• For $\forall A$, $A{A}^T$ and $A^{T}A$ are always PSD

3 Semidefinite programming (SDP)

3.1 Definition

$$\begin{array}{rl}\underset{x}{\min }& c^{T}x\\ s.t.& F_0+\sum _{i=1}^m{x}_i{F}_i\ge 0\end{array}$$ Where the objective is linear function, and $F_i$ are symmetric. Eigenvalues are positive.

3.2 With Linear Programming (LP)

$$\begin{array}{rl}\underset{x}{\min }& c^{T}x\\ s.t.& Ax\ge 0\end{array}$$ $$\begin{array}{rl}\underset{x}{\min }& c^{T}x\\ s.t.& \sum _i{\mathcal{A}}_i{x}_i\ge 0\end{array}$$ Where ${\mathcal{A}}_i=diag(A_i)$, $A_i$ is the $i_{th}$ column of $A$.

3.3 Schur complement

$X=\begin{array}{r}A=\left(\begin{array}{cc}A& B\\ B^T& C\end{array}\right)\end{array}$ is symmetric.
Then,

• $X>0$ $\iff$ $A>0$, and $C-B^T{A}^{-1}B>0$
• $X>0$ $\iff$ $C>0$, and $A-B{C}^{-1}{B}^T>0$
• If $A>0$, then $X>0$ $\iff$ $C-B^T{A}^{-1}B>0$