F范数 低秩矩阵近似高秩矩阵

F范数定义

$$\Vert A\Vert =\sqrt{\sum _{m,n=1,1}^{M,N}{a}_{m,n}^2}$$

SVD-singular value decomposition

$A$ is a $m\times n$ matrix.
$$A=U\Sigma V^{\ast }=U\left[\begin{array}{cc}{\Sigma }_k& 0\\ 0& 0\end{array}\right]V^{\ast }$$
$k$ is the rank of $A$.
Then we have:
$$\begin{array}{rl}\Vert A\Vert & =\Vert U\Sigma V^{\ast }\Vert \\ & =\sqrt{Tr((U\Sigma V^{\ast })(U\Sigma V^{\ast }{)}^T)}\\ & =\sqrt{Tr(U\Sigma V^{\ast }V{\Sigma }^T{U}^T)}\\ & =\sqrt{\sum _{m=1}^k{\sigma }_m^2}=\Vert \Sigma \Vert \end{array}$$
Now we can form a rank $r\le k$ matrix $\hat{A}$ by setting the value ${\delta }_{r+1},\cdots ,{\delta }_k$ in ${\Sigma }_k$ to be zero, namely:
$$\hat{A}=U\left[\begin{array}{cc}{\Sigma }_r& 0\\ 0& 0\end{array}\right]V^{\ast }$$
Then the Frobenius norm ${\epsilon }_r$ of the error matrix $(A-\hat{A})$ is given by:
$$\begin{array}{rl}{\epsilon }_r& =\Vert (A-\hat{A})\Vert \\ & =\Vert U(\Sigma -\hat{\Sigma })V^{\ast })\Vert \\ & =\Vert \Sigma -\hat{\Sigma }\Vert \\ & =\sqrt{\sum _{m=r+1}^k{\sigma }_m^2}\end{array}$$
Assume $B$ is already a matrix giving the minimum value of ${\epsilon }_B$ and its singular value decomposition is given by:
$$B=U_b{\Sigma }_b{V}_b^{\ast }=U_b\left[\begin{array}{cc}{\Sigma }_b& 0\\ 0& 0\end{array}\right]V_b^{\ast }$$
Now we define a new matrix $C$ which is given by:
$$\mathbf{C}={\mathbf{U}}_b^H\mathbf{A}{\mathbf{V}}_b=\left[\begin{array}{cc}{\mathbf{C}}_{11}& {\mathbf{C}}_{12}\\ {\mathbf{C}}_{21}& {\mathbf{C}}_{22}\end{array}\right]$$

Then we have:
$$\begin{array}{rl}{\epsilon }_B& =\Vert \mathbf{A}-\mathbf{B}\Vert \\ & =\Vert {\mathbf{U}}_b^H(\mathbf{A}-\mathbf{B}){\mathbf{V}}_b\Vert \\ & =\Vert \mathbf{C}-{\mathbf{\Sigma }}_b\Vert \\ & =\Vert {\mathbf{C}}_{11}-{\hat{\mathbf{\Sigma }}}_b\Vert +\Vert {\mathbf{C}}_{12}\Vert +\Vert {\mathbf{C}}_{21}\Vert +\Vert {\mathbf{C}}_{22}\Vert \end{array}$$

since $\mathbf{B}$ is already a matrix giving the minimum value of ${\epsilon }_B,$ we must have ${\mathbf{C}}_{12}=0$ Otherwise, we will be able to construct a new rank $r$ matrix $\hat{\mathbf{B}}$, given by:
$$\hat{\mathbf{B}}={\mathbf{U}}_b\left[\begin{array}{cc}{\hat{\mathbf{\Sigma }}}_b& {\mathbf{C}}_{12}\\ 0& 0\end{array}\right]{\mathbf{V}}_b^H$$
so that the new Frobenius norm:
$$\begin{array}{rl}{\epsilon }_{\hat{B}}& =\Vert \mathbf{A}-\hat{\mathbf{B}}\Vert \\ & =\Vert {\mathbf{U}}_b^H(\mathbf{A}-\hat{\mathbf{B}}){\mathbf{V}}_b\Vert \\ & =\Vert {\mathbf{C}}_{11}-{\hat{\mathbf{\Sigma }}}_b\Vert +\Vert {\mathbf{C}}_{21}\Vert +\Vert {\mathbf{C}}_{22}\Vert \end{array}$$
will be smaller than ${\epsilon }_B,$ which contradicts the assumption that $B$ gives the minimum value. In the same way, we have $C_{21}=0$ and $C_{11}={\hat{\mathbf{\Sigma }}}_b.$ Then we have:
$$\mathbf{C}={\mathbf{U}}_b^H\mathbf{A}{\mathbf{V}}_b=\left[\begin{array}{cc}{\hat{\mathbf{\Sigma }}}_b& 0\\ 0& {\mathbf{C}}_{22}\end{array}\right]$$
since ${\hat{\mathbf{\Sigma }}}_b$ is diagonal, it consists of $r$ singular values of $\mathbf{A}.$ We can get:
$$\begin{array}{rl}{\epsilon }_B& =\Vert \mathbf{C}-{\mathbf{\Sigma }}_b\Vert \\ & =\Vert {\mathbf{C}}_{22}\Vert \end{array}$$
since both $U_b$ and $V_b$ are unitary matrices, we have:
$$\Vert \mathbf{A}{\Vert }^2=\Vert \mathbf{C}{\Vert }^2={\Vert {\hat{\mathbf{\Sigma }}}_b\Vert }^2+{\Vert {\mathbf{C}}_{22}\Vert }^2$$
Then:
$$\begin{array}{rl}{\Vert c_x\Vert }^2& =\Vert A{\Vert }^2-{\Vert {\hat{\mathbf{\Sigma }}}_b\Vert }^2\\ & =\sum _{m=1}^k{\sigma }_m^2-{\Vert {\hat{\mathbf{\Sigma }}}_b\Vert }^2\end{array}$$
Obviously, when ${\hat{\mathbf{\Sigma }}}_b$ holds the $r$ largest singular values ${\sigma }_1,\dots ,{\sigma }_r$ of the matrix $\mathbf{A}$ ${\Vert {\mathbf{C}}_{22}\Vert }^2$ and then ${\epsilon }_B$ reaches its minimum value:
$${\epsilon }_B^2={\left|{\mathbf{C}}_{22}\right||}^2=\sum _{m=r+1}^k{\sigma }_m^2={\epsilon }_r^2$$
Therefore, we can draw the conclusion that $\hat{A}$ is the best rank $r$ approximation to $A$ based on minimisation of the error matrix’ Frobenius norm $\Vert \mathbf{A}-\mathbf{B}\Vert$