统计机器学习-多元高斯分布

多元高斯分布：

假设   $y^1,y^3,...,y^d$  分布独立，服从分布 $N(0,1)$
联合分布 $y=(y^1,y^2,...,y^d)$ 服从分布
$$g(y)=\prod _{j=1}^d\frac{1}{\sqrt{2\pi }}{e}^{-\frac{(y^j{)}^2}{2}}=\frac{1}{(2\pi {)}^{\frac{d}{2}}}{e}^{-\frac{1}{2}{y}^{T}y}$$
$$期望：E(y)=0；方差：Var(y)=1$$
做变化：$$x=Ty+\mu$$令：$$\sum =T{T}^T$$
$$f(x)=g(y)|det(T){|}^{-1}(det指行列式)$$
$$=\frac{1}{(2\pi {)}^{\frac{d}{2}}(det\sum {)}^{\frac{1}{2}}}{e}^{[-\frac{1}{2}(x-\mu {)}^T{\sum }^{-1}(x-\mu )]}$$
$$期望：E(x)=TE(y)+\mu =\mu$$$$方差：Var(x)=Var(Ty+\mu )=TV(y)T^T=\sum$$

条件高斯分布与边缘分布：

如果两组变量是联合高斯分布，那么以⼀组变量为条件，另⼀组变量同样是高斯分布。类似地，任何⼀个变量的边缘分布也是高斯分布。
$$x={(}_{x_b}^{x_a})$$$$\mu ={(}_{{\mu }_b}^{{\mu }_a})$$$$\sum ={(}_{{\sum }_{ba}{\sum }_{bb}}^{{\sum }_{aa}{\sum }_{ab}})$$$$\Lambda =(\sum {)}^{-1}$$$$\Lambda ={(}_{{\Lambda }_{ba}{\Lambda }_{bb}}^{{\Lambda }_{aa}{\Lambda }_{ab}})$$
条件分布是高斯分布：
$$p(x_a|x_b)=N(x_a|{\mu }_{a|b},{\Lambda }_{aa}^{-1})$$$${\mu }_{a|b}={\mu }_a-{\Lambda }_{aa}^{-1}{\Lambda }_{ab}(x_b-{\mu }_b)$$
边缘分布是高斯分布：
$$p(x_a)=N(x_a|{\mu }_a,(\sum {)}_{aa})$$

极大似然估计：
$$L=\frac{n}{2}logdet(\sum {)}^{-1}-\sum _{i=1}^n\frac{1}{2}(x_i-\mu {)}^T(\sum {)}^{-1}(x_i-\mu )$$

多元高斯分布可视化：

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import matplotlib as mpl

if __name__ == '__main__':
    mpl.rcParams['font.sans-serif'] = ['SimHei']
    mpl.rcParams['axes.unicode_minus'] = False

    d = np.random.randn(10000000, 2)
    N = 30
    density, edges = np.histogramdd(d, bins=[30, 30])
    print("样本总数: ", np.sum(density))
    density = density/density.max()
    x = y = np.arange(N)
    t = np.meshgrid(x,y)
    fig = plt.figure()
    ax = Axes3D(fig)
    ax.scatter(t[0], t[1], density, c='r', s=15*density, marker='o', depthshade=True)
    ax.plot_surface(t[0], t[1], density, cmap='rainbow', rstride=1, cstride=1, alpha=0.9, lw=1)
    ax.set_xlabel("x轴")
    ax.set_ylabel("y轴")
    ax.set_zlabel("z轴")
    plt.title("二元高斯分布")
    plt.tight_layout(0.1)
    plt.show()