协方差矩阵

For this data, we could calculate the variance $\sigma (x,x)$ in the x-direction and the variance $\sigma (y,y)$in the y-direction.
对于上面的数据，我们可以计算出数据沿x轴方向的方差 $\sigma (x,x)$，和数据沿y轴方向的方差 $\sigma (y,y)$

However, the horizontal spread and the vertical spread of the data does not explain the clear diagonal correlation.
然而，数据沿着平行于坐标轴方向扩展的方向并不能够解释数据在对角方向上的信息。

Figure 2 clearly shows that on average, if the x-value of a data point increases, then also the y-value increases, resulting in a positive correlation.
如上图所示，随着x增大，y值也增大，我们称二者是正相关的。

This correlation can be captured by extending the notion of variance to what is called the ‘covariance’ of the data:
为了进一步地描述数据之间的相关性，我们由方差引出了协方差。
$$\sigma (x,y)=\mathbb{E}[(x-\mathbb{E}(x))(y-\mathbb{E}(y))]$$
For 2D data, we thus obtain$\sigma (x,x)$, $\sigma (y,y)$, $\sigma (x,y)$ and $\sigma (y,x)$.
对于二维的数据，我们能够分别得到$\sigma (x,x)$, $\sigma (y,y)$, $\sigma (x,y)$ and $\sigma (y,x)$.

These four values can be summarized in a matrix, called the covariance matrix:
上面四个值，我们可以把它放在同一个矩阵里，叫作协方差矩阵。

$$\Sigma =\left[\begin{array}{cc}\sigma (x,x)& \sigma (x,y)\\ \sigma (y,x)& \sigma (y,y)\end{array}\right]$$
If x is positively correlated with y, y is also positively correlated with x. In other words, we can state that $\sigma (x,y)=\sigma (y,x)$.

Therefore, the covariance matrix is always a symmetric matrix with the variances on its diagonal and the covariances off-diagonal.

Two-dimensional normally distributed data is explained completely by its mean and its $2\times 2$ covariance matrix. Similarly, a $3\times 3$covariance matrix is used to capture the spread of three-dimensional data, and a $N\times N$ covariance matrix captures the spread of N-dimensional data.

Figure 3 illustrates how the overall shape of the data defines the covariance matrix:

参考

《A geometric interpretation of the covariance matrix》

https://www.visiondummy.com/2014/04/geometric-interpretation-covariance-matrix/