关于迹、相关系数的矩阵变换推导

1. 问题

对于一个矩阵$D=[d_1;d_2;\dots ;d_n]\in R^{n\times l}$, 其 相关矩阵为 $C\in R^{n\times n}$, 是否有以下表达式成立(经第2部分推导，$基本成立，最正确的见式3$):
$$tr[D(P-C)D^T]=\sum _{i,j}{c}_{ij}\Vert d_i-d_j{\Vert }^2\text{\hspace{1em}(1)}$$

t r [ D ( P − C ) D T ] = ∑ i = 1 n { 1 2 ∑ p , q l C p q [ d i c p − d i c q ] 2 } (2) tr[D(P-C)D^T] = \sum_{i=1}^n \{ \frac{1}{2}\sum_{p,q}^lC_{pq}[d_ic_p-d_ic_q]^2 \} \tag 2 tr[D(P−C)DT]=i=1∑n​{21​p,q∑l​Cpq​[di​cp​−di​cq​]2}(2)

其中C是对称矩阵，$P=diag(C\times 1^{n\times 1})\in R^{n\times n}$表示C按行累加形成的列向量，构成的对角矩阵。

2. 试着推导公式1

令
$$D=\left[\begin{array}{cccc}{d}_{11}& d_{12}& \cdots & d_{1l}\\ d_{21}& d_{22}& \cdots & d_{2l}\\ ⋮& ⋮& \ddots & ⋮\\ d_{n1}& d_{n2}& \cdots & d_{nl}\end{array}\right],C=\left[\begin{array}{cccc}1& c_{12}& \cdots & c_{1n}\\ c_{21}& 1& \cdots & c_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ c_{n1}& c_{n2}& \cdots & 1\end{array}\right]$$
则
$$\begin{gathered} D^T=\left[\begin{array}{cccc}{d}_{11}& d_{21}& \cdots & d_{n1}\\ d_{12}& d_{22}& \cdots & d_{n2}\\ ⋮& ⋮& \ddots & ⋮\\ d_{1l}& d_{2l}& \cdots & d_{nl}\end{array}\right]， \\ P=\left[\begin{array}{cccc}{\sum }_{j=1}^n{c}_{1j}& 0& \cdots & 0\\ 0& {\sum }_{j=1}^n{c}_{2j}& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & {\sum }_{j=1}^n{c}_{nj}\end{array}\right], \end{gathered}$$

从右侧开始：
$$\begin{gathered} \sum _{i,j}{c}_{ij}\Vert d_i-d_j{\Vert }^2=\sum _{i,j}{c}_{ij}\Vert d_i-d_j{\Vert }_2^2 \\ =\sum _{i,j}{c}_{ij}\sum _{k=1}^l(d_{ik}-d_{jk}{)}^2 \\ =\sum _{i=1}^n\sum _{j=1}^n{c}_{ij}\sum _{k=1}^l(d_{ik}-d_{jk}{)}^2 \end{gathered}$$
左侧：
$$tr[D(P-C)D^T]无法直接运算，试着更正为：tr[D^T(P-C)D]$$
则
$$\begin{gathered} tr[D^T(P-C)D] \\ =tr[D^{T}PD-D^{T}CD] \\ =tr(D^{T}PD)-tr(D^{T}CD) \end{gathered}$$

$$\begin{gathered} D^{T}P=\left[\begin{array}{cccc}{d}_{11}& d_{21}& \cdots & d_{n1}\\ d_{12}& d_{22}& \cdots & d_{n2}\\ ⋮& ⋮& \ddots & ⋮\\ d_{1l}& d_{2l}& \cdots & d_{nl}\end{array}\right]\left[\begin{array}{cccc}{\sum }_{j=1}^n{c}_{1j}& 0& \cdots & 0\\ 0& {\sum }_{j=1}^n{c}_{2j}& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & {\sum }_{j=1}^n{c}_{nj}\end{array}\right] \\ =\left[\begin{array}{cccc}{d}_{11}{\sum }_{j=1}^n{c}_{1j}& d_{21}{\sum }_{j=1}^n{c}_{2j}& \cdots & d_{n1}{\sum }_{j=1}^n{c}_{nj}\\ d_{12}{\sum }_{j=1}^n{c}_{1j}& d_{22}{\sum }_{j=1}^n{c}_{2j}& \cdots & d_{n2}{\sum }_{j=1}^n{c}_{nj}\\ ⋮& ⋮& \ddots & ⋮\\ d_{1l}{\sum }_{j=1}^n{c}_{1j}& d_{2l}{\sum }_{j=1}^n{c}_{2j}& \cdots & d_{nl}{\sum }_{j=1}^n{c}_{nj}\end{array}\right] \end{gathered}$$
$$\begin{gathered} tr(D^{T}PD)=tr(\left[\begin{array}{cccc}{d}_{11}{\sum }_{j=1}^n{c}_{1j}& d_{21}{\sum }_{j=1}^n{c}_{2j}& \cdots & d_{n1}{\sum }_{j=1}^n{c}_{nj}\\ d_{12}{\sum }_{j=1}^n{c}_{1j}& d_{22}{\sum }_{j=1}^n{c}_{2j}& \cdots & d_{n2}{\sum }_{j=1}^n{c}_{nj}\\ ⋮& ⋮& \ddots & ⋮\\ d_{1l}{\sum }_{j=1}^n{c}_{1j}& d_{2l}{\sum }_{j=1}^n{c}_{2j}& \cdots & d_{nl}{\sum }_{j=1}^n{c}_{nj}\end{array}\right]\left[\begin{array}{cccc}{d}_{11}& d_{12}& \cdots & d_{1l}\\ d_{21}& d_{22}& \cdots & d_{2l}\\ ⋮& ⋮& \ddots & ⋮\\ d_{n1}& d_{n2}& \cdots & d_{nl}\end{array}\right]) \\ =d_{11}^2\sum _{j=1}^n{c}_{1j}+d_{21}^2\sum _{j=1}^n{c}_{2j}+\cdots +d_{n1}^2\sum _{j=1}^n{c}_{nj} \\ +d_{12}^2\sum _{j=1}^n{c}_{1j}+d_{22}^2\sum _{j=1}^n{c}_{2j}+\cdots +d_{n2}^2\sum _{j=1}^n{c}_{nj} \\ +\dots \\ +d_{1l}^2\sum _{j=1}^n{c}_{1j}+d_{2l}^2\sum _{j=1}^n{c}_{2j}+\cdots +d_{nl}^2\sum _{j=1}^n{c}_{nj} \\ =\sum _{j=1}^n{c}_{1j}(d_{11}^2+d_{12}^2+\cdots +d_{1l}^2) \\ +\sum _{j=1}^n{c}_{2j}(d_{21}^2+d_{22}^2+\cdots +d_{2l}^2) \\ +\dots \\ +\sum _{j=1}^n{c}_{nj}(d_{n1}^2+d_{n2}^2+\cdots +d_{nl}^2) \end{gathered}$$

$$\begin{gathered} D^{T}C=\left[\begin{array}{cccc}{d}_{11}& d_{21}& \cdots & d_{n1}\\ d_{12}& d_{22}& \cdots & d_{n2}\\ ⋮& ⋮& \ddots & ⋮\\ d_{1l}& d_{2l}& \cdots & d_{nl}\end{array}\right]\left[\begin{array}{cccc}1& c_{12}& \cdots & c_{1n}\\ c_{21}& 1& \cdots & c_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ c_{n1}& c_{n2}& \cdots & 1\end{array}\right] \\ =\left[\begin{array}{cccc}{\sum }_{s=1}^n{d}_{s1}{c}_{s1}& {\sum }_{s=1}^n{d}_{s1}{c}_{s2}& \cdots & {\sum }_{s=1}^n{d}_{s1}{c}_{sn}\\ {\sum }_{s=1}^n{d}_{s2}{c}_{s1}& {\sum }_{s=1}^n{d}_{s2}{c}_{s2}& \cdots & {\sum }_{s=1}^n{d}_{s2}{c}_{sn}\\ ⋮& ⋮& \ddots & ⋮\\ {\sum }_{s=1}^n{d}_{sl}{c}_{s1}& {\sum }_{s=1}^n{d}_{sl}{c}_{s2}& \cdots & {\sum }_{s=1}^n{d}_{sl}{c}_{sn}\end{array}\right] \end{gathered}$$
$$\begin{gathered} tr(D^{T}CD)=tr(\left[\begin{array}{cccc}{\sum }_{s=1}^n{d}_{s1}{c}_{s1}& {\sum }_{s=1}^n{d}_{s1}{c}_{s2}& \cdots & {\sum }_{s=1}^n{d}_{s1}{c}_{sn}\\ {\sum }_{s=1}^n{d}_{s2}{c}_{s1}& {\sum }_{s=1}^n{d}_{s2}{c}_{s2}& \cdots & {\sum }_{s=1}^n{d}_{s2}{c}_{sn}\\ ⋮& ⋮& \ddots & ⋮\\ {\sum }_{s=1}^n{d}_{sl}{c}_{s1}& {\sum }_{s=1}^n{d}_{sl}{c}_{s2}& \cdots & {\sum }_{s=1}^n{d}_{sl}{c}_{sn}\end{array}\right]\left[\begin{array}{cccc}{d}_{11}& d_{12}& \cdots & d_{1l}\\ d_{21}& d_{22}& \cdots & d_{2l}\\ ⋮& ⋮& \ddots & ⋮\\ d_{n1}& d_{n2}& \cdots & d_{nl}\end{array}\right]) \\ =d_{11}\sum _{s=1}^n{d}_{s1}{c}_{s1}+d_{21}\sum _{s=1}^n{d}_{s1}{c}_{s2}+\cdots +d_{n1}\sum _{s=1}^n{d}_{s1}{c}_{sn} \\ +d_{12}\sum _{s=1}^n{d}_{s2}{c}_{s1}+d_{22}\sum _{s=1}^n{d}_{s2}{c}_{s2}+\cdots +d_{n2}\sum _{s=1}^n{d}_{s2}{c}_{sn} \\ +\dots \\ +d_{1l}\sum _{s=1}^n{d}_{sl}{c}_{s1}+d_{2l}\sum _{s=1}^n{d}_{sl}{c}_{s2}+\cdots +d_{nl}\sum _{s=1}^n{d}_{sl}{c}_{sn} \end{gathered}$$

则
$$\begin{gathered} tr[D^T(P-C)D] \\ =tr(D^{T}PD)-tr(D^{T}CD) \\ =\sum _{j=1}^n{c}_{1j}(d_{11}^2+d_{12}^2+\cdots +d_{1l}^2) \\ +\sum _{j=1}^n{c}_{2j}(d_{21}^2+d_{22}^2+\cdots +d_{2l}^2) \\ +\dots \\ +\sum _{j=1}^n{c}_{nj}(d_{n1}^2+d_{n2}^2+\cdots +d_{nl}^2) \\ -(d_{11}\sum _{s=1}^n{d}_{s1}{c}_{s1}+d_{21}\sum _{s=1}^n{d}_{s1}{c}_{s2}+\cdots +d_{n1}\sum _{s=1}^n{d}_{s1}{c}_{sn} \\ +d_{12}\sum _{s=1}^n{d}_{s2}{c}_{s1}+d_{22}\sum _{s=1}^n{d}_{s2}{c}_{s2}+\cdots +d_{n2}\sum _{s=1}^n{d}_{s2}{c}_{sn} \\ +\dots \\ +d_{1l}\sum _{s=1}^n{d}_{sl}{c}_{s1}+d_{2l}\sum _{s=1}^n{d}_{sl}{c}_{s2}+\cdots +d_{nl}\sum _{s=1}^n{d}_{sl}{c}_{sn}) \\ =\sum _{j=1}^n{c}_{1j}(d_{11}^2+d_{12}^2+\cdots +d_{1l}^2-\sum _{k=1}^l{d}_{jk}{d}_{1k}) \\ +\sum _{j=1}^n{c}_{2j}(d_{21}^2+d_{22}^2+\cdots +d_{2l}^2-\sum _{k=1}^l{d}_{jk}{d}_{2k}) \\ +\dots \\ +\sum _{j=1}^n{c}_{nj}(d_{n1}^2+d_{n2}^2+\cdots +d_{nl}^2-\sum _{k=1}^l{d}_{jk}{d}_{nk}) \\ =\sum _{i=1}^n\sum _{j=1}^n{c}_{ij}(d_{i1}^2+d_{i2}^2+\cdots +d_{il}^2-\sum _{k=1}^l{d}_{jk}{d}_{ik}) \\ =\sum _{i=1}^n\sum _{j=1}^n{c}_{ij}(\sum _{k=1}^l{d}_{ik}^2-\sum _{k=1}^l{d}_{jk}{d}_{ik}) \\ =\sum _{i=1}^n\sum _{j=1}^n{c}_{ij}\sum _{k=1}^l(d_{ik}^2-d_{jk}{d}_{ik}) \end{gathered}$$
由于矩阵C中 $C_{ij}=C_{ji}$，所以有：

$$\begin{gathered} 2tr[D^T(P-C)D] \\ =2\sum _{i=1}^n\sum _{j=1}^n{c}_{ij}\sum _{k=1}^l(d_{ik}^2-d_{jk}{d}_{ik}) \\ =\sum _{i=1}^n\sum _{j=1}^n{c}_{ij}\sum _{k=1}^l(d_{ik}^2+d_{jk}^2-2d_{ik}{d}_{jk}) \\ =\sum _{i=1}^n\sum _{j=1}^n{c}_{ij}\sum _{k=1}^l(d_{ik}-d_{jk}{)}^2 \\ =\sum _{i,j}{c}_{ij}\Vert d_i-d_j{\Vert }^2 \end{gathered}$$
所以结论得证，最终表达式应该是：

$$2tr[D^T(P-C)D]=\sum _{i,j}{c}_{ij}\Vert d_i-d_j{\Vert }_2^2\text{\hspace{1em}(3)}$$
注意的是：如果$d_i$ 是列向量，则是上式（转置项在前），如果是行向量，则是下式（转置项在后），实际上二者在最初的表达时，通过一个转置即可实现转换，所以结果有差异。

在形式表达式，参考下：

3.对于公式2的推导

由于公式2和公式1及公式3在意义上差不多，所以可以简单的认为公式2也是成立的。

4. 公式理解

• $C_{ij}$表示按类似皮尔逊方法获得的相关系数，取值范围[-1,1], 1为正相关，-1为负相关。
• 当$C_{ij}=1$时，表示正相关，这个时候要min化，则$d_i-d_j$应尽可能小，即前者越接近，输出应越接近。
• 当$C_{ij}=-1$时，表示负相关，这个时候要min化，则$d_i-d_j$应尽可能大，即前者越不像，输出也应越不像。
• $C_{ij}$是指相关系数，可以用皮尔逊相关系数进行计算，也可以用其他方式。
  
  或
  
  其中t是一个常数，通常设置为1，参：Multi-label feature selection via manifold regularization and dependence maximization[黄睿]

5. 问题

公式1的推导，是从结论反向推导的。那如果是拿到等式的一边，怎么能得出另一边呢？？？这个太难了，感觉，不知道各位是否有好的办法。欢迎留言区评论或私信讨论，谢谢！