正态(高斯)分布

一维正态分布

$$\begin{array}{r}f(x)=\frac{1}{\sigma \sqrt{2\pi }}\exp \left(-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right)\end{array}$$则称$X\sim N(\mu ,{\sigma }^2)$

$$\begin{array}{r}\begin{array}{rl}{\int }_{-\infty }^{+\infty }f(x)\mathrm{d}x& ={\int }_{-\infty }^{+\infty }\frac{1}{\sigma \sqrt{2\pi }}\exp \left(-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right)\mathrm{d}x\\ & \stackrel{t=\frac{x-\mu }{\sigma }}{=}\frac{1}{\sqrt{2\pi }}{\int }_{-\infty }^{+\infty }\exp \left(-\frac{t^2}{2}\right)\mathrm{d}t=1\end{array}\end{array}$$
$$\begin{array}{r}\begin{array}{rl}{\left({\int }_{-\infty }^{+\infty }\exp \left(-\frac{t^2}{2}\right)\mathrm{d}t\right)}^2& ={\int }_{-\infty }^{+\infty }{\int }_{-\infty }^{+\infty }\exp \left(-\frac{t^2+u^2}{2}\right)\mathrm{d}t\mathrm{d}u\\ & \stackrel{t=\rho cos\theta ,u=\rho sin\theta }{=}{\int }_0^{2\pi }\left({\int }_0^{+\infty }\exp \left(-\frac{{\rho }^2}{2}\right)\rho \mathrm{d}\rho \right)\mathrm{d}\theta ={\int }_0^{2\pi }\mathrm{d}\theta =2\pi \end{array}\end{array}$$

标准正态分布

$$\begin{array}{rl}\phi (x)& =\frac{1}{\sqrt{2\pi }}\exp \left(-\frac{x^2}{2}\right)\\ \Phi (x)& =\frac{1}{\sqrt{2\pi }}{\int }_{-\infty }^{+\infty }\exp \left(-\frac{x^2}{2}\right)dx\end{array}$$

性质：

• $\phi (-x)=\phi (x)$
• $\Phi (-x)=1-\Phi (x)$
  pf：
  $$\begin{array}{r}\Phi (-x)={\int }_{-\infty }^{+\infty }\phi (t)dt\stackrel{t=-u}{=}{\int }_x^{+\infty }\phi (u)du={\int }_{-\infty }^{+\infty }\phi (u)du-{\int }_{-\infty }^x\phi (u)du=1-\Phi (x)\end{array}$$

高维正态分布

先从各维度不相关的多元正态分布入手

d维数据$x={\left[\begin{array}{cccc}{x}_1& x_2& \cdots & x_d\end{array}\right]}^T$，各维度的均值为${\mu }_1,{\mu }_2,\cdots ,{\mu }_d$，标准差为${\sigma }_1,{\sigma }_2,\cdots ,{\sigma }_d$

高斯概率密度函数可以表示为
$$\begin{array}{rl}p(\mathbf{x})& =p(x_1)p(x_2)\cdots p(x_d)=\frac{1}{(\sqrt{2\pi }{)}^d{\sigma }_1{\sigma }_2\cdots {\sigma }_d}\exp \left(-\frac{1}{2}\left[(\frac{x_1-{\mu }_1}{{\sigma }_1}{)}^2+(\frac{x_2-{\mu }_2}{{\sigma }_2}{)}^2+\cdots +(\frac{x_d-{\mu }_d}{{\sigma }_d}{)}^2\right]\right)\\ d^2(x,\mu )& =(\frac{x_1-{\mu }_1}{{\sigma }_1}{)}^2+(\frac{x_2-{\mu }_2}{{\sigma }_2}{)}^2+\cdots +(\frac{x_d-{\mu }_d}{{\sigma }_d}{)}^2=(X-\mu {)}^T{\mathbf{\Sigma }}^{-1}(X-\mu )\\ & =\left[\begin{array}{cccc}{x}_1-{\mu }_1& x_2-{\mu }_2& \cdots & x_d-{\mu }_d\end{array}\right]\left[\begin{array}{cccc}\frac{1}{{\sigma }_1^2}& & \cdots & 0\\ 0& \frac{1}{{\sigma }_2^2}& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & \frac{1}{{\sigma }_d^2}\end{array}\right]\left[\begin{array}{c}{x}_1-{\mu }_1\\ x_2-{\mu }_2\\ ⋮\\ x_d-{\mu }_d\end{array}\right]\end{array}$$
得到高维正态分布：
$$\begin{array}{r}p(\mathbf{x})=\frac{1}{(\sqrt{2\pi }{)}^d{\sigma }_1{\sigma }_2\cdots {\sigma }_d}\exp \left(-\frac{1}{2}[d^2(x,\mu )]\right)=\frac{1}{(2\pi {)}^{\frac{d}{2}}|\mathbf{\Sigma }{|}^{\frac{1}{2}}}\exp \left(-\frac{1}{2}(X-\mu {)}^T{\mathbf{\Sigma }}^{-1}(X-\mu )\right)\end{array}$$

高斯分布的归一化积

设$p_1(x)=\mathcal{N}(x|{\mu }_1,{\sigma }_1),p_2(x)=\mathcal{N}(x|{\mu }_2,{\sigma }_2)$均是关于变量$x$的分布
$$\begin{array}{rl}{p}_1(x)p_2(x)& \sim \exp \left(-\frac{1}{2{\sigma }_1^2}(x-{\mu }_1{)}^2\right)\exp \left(-\frac{1}{2{\sigma }_2^2}(x-{\mu }_2{)}^2\right)\\ & =\exp \left(-\frac{1}{2}\frac{({\sigma }_1^2+{\sigma }_2^2)x^2-2({\mu }_1{\sigma }_2^2+{\mu }_2{\sigma }_1^2)x+\mathrm{constant}}{{\sigma }_1^2{\sigma }_2^2}\right)\\ & \sim \exp \left(-\frac{1}{2}\frac{{\sigma }_1^2+{\sigma }_2^2}{{\sigma }_1^2{\sigma }_2^2}{\left(x-\frac{{\mu }_1{\sigma }_2^2+{\mu }_2{\sigma }_1^2}{{\sigma }_1^2+{\sigma }_2^2}\right)}^2\right)\end{array}$$
得到两个高斯分布相乘仍为高斯分布
$$\begin{array}{l}\mu =\frac{{\mu }_1{\sigma }_2^2+{\mu }_2{\sigma }_1^2}{{\sigma }_1^2+{\sigma }_2^2}\\ \\ \sigma =\sqrt{\frac{{\sigma }_1^2{\sigma }_2^2}{{\sigma }_1^2+{\sigma }_2^2}}\end{array}⟹\begin{array}{l}\mu =\left(\frac{{\mu }_1}{{\sigma }_1^2}+\frac{{\mu }_2}{{\sigma }_2^2}\right){\sigma }^2\\ \\ \frac{1}{{\sigma }^2}=\frac{1}{{\sigma }_1^2}+\frac{1}{{\sigma }_2^2}\end{array}$$

高维高斯分布

$$\begin{array}{rl}{p}_1(\mathbf{x})p_2(\mathbf{x})\sim & \exp \left(-\frac{1}{2}(\mathbf{x}-{\mathbf{\mu }}_1{)}^T{\mathbf{\Sigma }}_1^{-1}(\mathbf{x}-{\mathbf{\mu }}_1)\right)\exp \left(-\frac{1}{2}(\mathbf{x}-{\mathbf{\mu }}_2{)}^T{\mathbf{\Sigma }}_2^{-1}(\mathbf{x}-{\mathbf{\mu }}_2)\right)\\ =& \exp \left(-\frac{1}{2}\left[{\mathbf{x}}^T({\mathbf{\Sigma }}_1^{-1}+{\mathbf{\Sigma }}_2^{-1})\mathbf{x}-2({{\mathbf{\mu }}_1}^T{\mathbf{\Sigma }}_1^{-1}+{{\mathbf{\mu }}_2}^T{\mathbf{\Sigma }}_2^{-1})\mathbf{x}+{{\mathbf{\mu }}_1}^T{\mathbf{\Sigma }}_1^{-1}{\mathbf{\mu }}_1+{{\mathbf{\mu }}_2}^T{\mathbf{\Sigma }}_2^{-1}{\mathbf{\mu }}_2\right]\right)\\ =& \exp \left(-\frac{{\left[\mathbf{x}-{\left({\mathbf{\Sigma }}_1^{-1}+{\mathbf{\Sigma }}_2^{-1}\right)}^{-1}\left({\mathbf{\Sigma }}_1^{-1}{\mathbf{\mu }}_1+{\mathbf{\Sigma }}_2^{-1}{\mathbf{\mu }}_2\right)\right]}^T\left({\mathbf{\Sigma }}_1^{-1}+{\mathbf{\Sigma }}_2^{-1}\right)\left[\mathbf{x}-{\left({\mathbf{\Sigma }}_1^{-1}+{\mathbf{\Sigma }}_2^{-1}\right)}^{-1}\left({\mathbf{\Sigma }}_1^{-1}{\mathbf{\mu }}_1+{\mathbf{\Sigma }}_2^{-1}{\mathbf{\mu }}_2\right)\right]}{2}+\mathrm{constant}\right)\end{array}$$

得到两个高斯分布相乘仍为高斯分布
$$\begin{array}{rl}\mathbf{\mu }& =\left({\mathbf{\Sigma }}_1^{-1}{\mathbf{\mu }}_1+{\mathbf{\Sigma }}_2^{-1}{\mathbf{\mu }}_2\right){\left({\mathbf{\Sigma }}_1^{-1}+{\mathbf{\Sigma }}_2^{-1}\right)}^{-1}\\ \mathbf{\Sigma }& ={\left({\mathbf{\Sigma }}_1^{-1}+{\mathbf{\Sigma }}_2^{-1}\right)}^{-1}\end{array}$$
即
$$\begin{array}{rl}{\mathbf{\Sigma }}^{-1}& =\left({\mathbf{\Sigma }}_1^{-1}+{\mathbf{\Sigma }}_2^{-1}\right)\\ \left({\mathbf{\Sigma }}_1^{-1}+{\mathbf{\Sigma }}_2^{-1}\right)\mathbf{\mu }& =\left({\mathbf{\Sigma }}_1^{-1}{\mathbf{\mu }}_1+{\mathbf{\Sigma }}_2^{-1}{\mathbf{\mu }}_2\right)\end{array}$$
进一步扩展到K个高斯分布的归一化积：
$$\begin{array}{rl}{\mathbf{\Sigma }}^{-1}& =\sum _{k=1}^K{\mathbf{\Sigma }}_k^{-1}\\ {\mathbf{\Sigma }}^{-1}\mathbf{\mu }& =\sum _{k=1}^K{\mathbf{\Sigma }}_k^{-1}{\mathbf{\mu }}_k\end{array}$$

高斯分布随机变量线性变换的归一化积

$$\begin{array}{rl}\exp \left(-\frac{1}{2}(\mathbf{x}-\mathbf{\mu }{)}^T{\mathbf{\Sigma }}^{-1}(\mathbf{x}-\mathbf{\mu })\right)\sim & \exp \left(-\frac{1}{2}({\mathbf{G}}_1\mathbf{x}-{\mathbf{\mu }}_1{)}^T{\mathbf{\Sigma }}_1^{-1}({\mathbf{G}}_1\mathbf{x}-{\mathbf{\mu }}_1)\right)\exp \left(-\frac{1}{2}({\mathbf{G}}_2\mathbf{x}-{\mathbf{\mu }}_2{)}^T{\mathbf{\Sigma }}_2^{-1}({\mathbf{G}}_2\mathbf{x}-{\mathbf{\mu }}_2)\right)\\ & =\exp \left(-\frac{1}{2}\left[{\mathbf{x}}^T({\mathbf{G}}_1^T{\mathbf{\Sigma }}_1^{-1}{\mathbf{G}}_1+{\mathbf{G}}_2^T{\mathbf{\Sigma }}_2^{-1}{\mathbf{G}}_2)\mathbf{x}-2({{\mathbf{\mu }}_1}^T{\mathbf{\Sigma }}_1^{-1}{\mathbf{G}}_1+{{\mathbf{\mu }}_2}^T{\mathbf{\Sigma }}_2^{-1}{\mathbf{G}}_2)\mathbf{x}+{{\mathbf{\mu }}_1}^T{\mathbf{\Sigma }}_1^{-1}{\mathbf{\mu }}_1+{{\mathbf{\mu }}_2}^T{\mathbf{\Sigma }}_2^{-1}{\mathbf{\mu }}_2\right]\right)\\ & =\exp \left(-\frac{{\left[\mathbf{x}-{\left({\mathbf{G}}_1^T{\mathbf{\Sigma }}_1^{-1}{\mathbf{G}}_1+{\mathbf{G}}_2^T{\mathbf{\Sigma }}_2^{-1}{\mathbf{G}}_2\right)}^{-1}\left({\mathbf{G}}_1^T{\mathbf{\Sigma }}_1^{-1}{\mathbf{\mu }}_1+{\mathbf{G}}_2^T{\mathbf{\Sigma }}_2^{-1}{\mathbf{\mu }}_2\right)\right]}^T\left({\mathbf{G}}_1^T{\mathbf{\Sigma }}_1^{-1}{\mathbf{G}}_1+{\mathbf{G}}_2^T{\mathbf{\Sigma }}_2^{-1}{\mathbf{G}}_2\right)\left[\mathbf{x}-{\left({\mathbf{G}}_1^T{\mathbf{\Sigma }}_1^{-1}{\mathbf{G}}_1+{\mathbf{G}}_2^T{\mathbf{\Sigma }}_2^{-1}{\mathbf{G}}_2\right)}^{-1}\left({\mathbf{G}}_1^T{\mathbf{\Sigma }}_1^{-1}{\mathbf{\mu }}_1+{\mathbf{G}}_2^T{\mathbf{\Sigma }}_2^{-1}{\mathbf{\mu }}_2\right)\right]}{2}+\mathrm{constant}\right)\end{array}$$
可得
$$\begin{array}{rl}{\mathbf{\Sigma }}^{-1}& =\left({\mathbf{G}}_1^T{\mathbf{\Sigma }}_1^{-1}{\mathbf{G}}_1+{\mathbf{G}}_2^T{\mathbf{\Sigma }}_2^{-1}{\mathbf{G}}_2\right)\\ \left({\mathbf{G}}_1^T{\mathbf{\Sigma }}_1^{-1}{\mathbf{G}}_1+{\mathbf{G}}_2^T{\mathbf{\Sigma }}_2^{-1}{\mathbf{G}}_2\right)\mathbf{\mu }={\mathbf{\Sigma }}^{-1}\mathbf{\mu }& =\left({\mathbf{G}}_1^T{\mathbf{\Sigma }}_1^{-1}{\mathbf{\mu }}_1+{\mathbf{G}}_2^T{\mathbf{\Sigma }}_2^{-1}{\mathbf{\mu }}_2\right)\end{array}$$
进一步扩展到K个高斯分布的归一化积：
$$\begin{array}{rl}{\mathbf{\Sigma }}^{-1}& =\sum _{k=1}^K{\mathbf{G}}_k^T{\mathbf{\Sigma }}_k^{-1}{\mathbf{G}}_k\\ {\mathbf{\Sigma }}^{-1}\mathbf{\mu }& =\sum _{k=1}^K{\mathbf{G}}_k^T{\mathbf{\Sigma }}_k^{-1}{\mathbf{\mu }}_k\end{array}$$

联合高斯概率密度函数，分解与推断

设有一对服从多元正态分布的变量$(\mathbf{x},\mathbf{y})$，联合概率为
$$p\left(\mathbf{x},\mathbf{y}\right)=\mathcal{N}\left(\left[\begin{array}{c}{\mathbf{\mu }}_x\\ {\mathbf{\mu }}_y\end{array}\right],\left[\begin{array}{cc}{\mathbf{\Sigma }}_{xx}& {\mathbf{\Sigma }}_{xy}\\ {\mathbf{\Sigma }}_{yx}& {\mathbf{\Sigma }}_{yy}\end{array}\right]\right)$$
注意${\mathbf{\Sigma }}_{yx}={\mathbf{\Sigma }}_{xy}^T$。
我们总是可以将联合概率密度分解成两个因子的乘积(条件概率乘以边缘概率)
$$p\left(\mathbf{x},\mathbf{y}\right)=p\left(\mathbf{x}|\mathbf{y}\right)p\left(\mathbf{y}\right)$$
对协方差矩阵进行相似对角化得：
$$\begin{array}{rl}\left[\begin{array}{cc}\mathbf{I}& -{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\\ 0& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}{\mathbf{\Sigma }}_{xx}& {\mathbf{\Sigma }}_{xy}\\ {\mathbf{\Sigma }}_{yx}& {\mathbf{\Sigma }}_{xy}\end{array}\right]& =\left[\begin{array}{cc}{\mathbf{\Sigma }}_{xx}-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}& 0\\ {\mathbf{\Sigma }}_{yx}& {\mathbf{\Sigma }}_{yy}\end{array}\right]\\ & \\ \left[\begin{array}{cc}\mathbf{I}& -{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\\ 0& 1\end{array}\right]\left[\begin{array}{cc}{\mathbf{\Sigma }}_{xx}& {\mathbf{\Sigma }}_{xy}\\ {\mathbf{\Sigma }}_{yx}& {\mathbf{\Sigma }}_{yy}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& 0\\ -{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}& \mathbf{I}\end{array}\right]& =\left[\begin{array}{cc}{\mathbf{\Sigma }}_{xx}-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}& 0\\ 0& {\mathbf{\Sigma }}_{yy}\end{array}\right]\\ & \\ \left[\begin{array}{cc}{\mathbf{\Sigma }}_{xx}& {\mathbf{\Sigma }}_{xy}\\ {\mathbf{\Sigma }}_{yx}& {\mathbf{\Sigma }}_{yy}\end{array}\right]& =\left[\begin{array}{cc}\mathbf{I}& {\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\\ 0& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}{\mathbf{\Sigma }}_{xx}-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}& 0\\ 0& {\mathbf{\Sigma }}_{yy}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& 0\\ {\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}& \mathbf{I}\end{array}\right]\end{array}$$

联合概率密度$p(\mathbf{x},\mathbf{y})$指数部分的二次项
$$\begin{gathered} {\left(\left[\begin{array}{c}\mathbf{x}\\ \mathbf{y}\end{array}\right]-\left[\begin{array}{c}{\mathbf{\mu }}_x\\ {\mathbf{\mu }}_y\end{array}\right]\right)}^T{\left(\begin{array}{cc}{\mathbf{\Sigma }}_{xx}& {\mathbf{\Sigma }}_{xy}\\ {\mathbf{\Sigma }}_{yx}& {\mathbf{\Sigma }}_{yy}\end{array}\right)}^{-1}\left(\left[\begin{array}{c}\mathbf{x}\\ \mathbf{y}\end{array}\right]-\left[\begin{array}{c}{\mathbf{\mu }}_x\\ {\mathbf{\mu }}_y\end{array}\right]\right) \\ ={\left(\left[\begin{array}{c}\mathbf{x}\\ \mathbf{y}\end{array}\right]-\left[\begin{array}{c}{\mathbf{\mu }}_x\\ {\mathbf{\mu }}_y\end{array}\right]\right)}^T\left[\begin{array}{cc}\mathbf{I}& 0\\ -{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}{\left({\mathbf{\Sigma }}_{xx}-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}\right)}^{-1}& 0\\ 0& {\mathbf{\Sigma }}_{yy}^{-1}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& -{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\\ 0& \mathbf{I}\end{array}\right]\left(\left[\begin{array}{c}\mathbf{x}\\ \mathbf{y}\end{array}\right]-\left[\begin{array}{c}{\mathbf{\mu }}_x\\ {\mathbf{\mu }}_y\end{array}\right]\right) \\ ={\left[\begin{array}{c}\left(\mathbf{x}-{\mathbf{\mu }}_x\right)-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\left(y-{\mathbf{\mu }}_y\right)\\ \mathbf{y}-{\mathbf{\mu }}_y\end{array}\right]}^T\left[\begin{array}{cc}{\left({\mathbf{\Sigma }}_{xx}-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}{\mathbf{\Sigma }}_{yx}\right)}^{-1}& 0\\ 0& {\mathbf{\Sigma }}_{yy}^{-1}\end{array}\right]\left[\begin{array}{c}\left(\mathbf{x}-{\mathbf{\mu }}_x\right)-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\left(\mathbf{y}-{\mathbf{\mu }}_y\right)\\ \mathbf{y}-{\mathbf{\mu }}_y\end{array}\right] \\ ={\left[\mathbf{x}-{\mathbf{\mu }}_x-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\left(\mathbf{y}-{\mathbf{\mu }}_y\right)\right]}^T{\left({\mathbf{\Sigma }}_{xx}-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}\right)}^{-1}\left[\mathbf{x}-{\mathbf{\mu }}_x-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\left(y-{\mathbf{\mu }}_y\right)\right]+{\left(\mathbf{y}-{\mathbf{\mu }}_y\right)}^T{\mathbf{\Sigma }}_{yy}^{-1}\left(\mathbf{y}-{\mathbf{\mu }}_y\right) \end{gathered}$$
这是两个二次项的和。由于幂运算中同底数幂相乘，底数不变、指数相加的性质，可以得到：
$$\begin{array}{l}p\left(\mathbf{x},\mathbf{y}\right)=p\left(\mathbf{x}|\mathbf{y}\right)p\left(\mathbf{y}\right)\\ p\left(\mathbf{x}|\mathbf{y}\right)=\mathcal{N}({\mathbf{\mu }}_x+{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\left(\mathbf{y}-{\mathbf{\mu }}_y\right),{\mathbf{\Sigma }}_{xx}-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx})\\ p\left(\mathbf{y}\right)=\mathcal{N}\left({\mathbf{\mu }}_y,{\mathbf{\Sigma }}_{yy}\right)\end{array}$$

高斯分布随机变量的非线性变换

研究高斯分布经过一个随机非线性变换之后的情况，即计算：
$$\begin{array}{r}p(\mathbf{y})={\int }_{-\infty }^{\infty }p(\mathbf{y}|\mathbf{x})p(\mathbf{x})\mathrm{d}\mathbf{x}\end{array}$$
其中
$$\begin{array}{rl}p(\mathbf{y}|\mathbf{x})& =\mathcal{N}(\mathbf{g}(\mathbf{x}),\mathbf{R})\\ p(\mathbf{x})& =\mathcal{N}({\mathbf{\mu }}_x,{\mathbf{\Sigma }}_{xx})\end{array}$$
这里$\mathbf{g}(\cdot )$表示$\mathbf{g}:\mathbf{x}\mapsto \mathbf{y}$，是一个非线性映射。它受零均值高斯噪声干扰，其协方差为$\mathbf{R}$。后文我们需要用到这类随机非线性映射对传感器进行建模。
对非线性变换进行线性化后，得到
$$\begin{array}{rl}\mathbf{g}\left(\mathbf{x}\right)& \approx {\mathbf{\mu }}_y+\mathbf{G}\left(\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}\right)\\ \mathbf{G}& =\frac{\partial \mathbf{g}\left(\mathbf{x}\right)}{\partial \mathbf{x}}{|}_{\mathbf{x}={\mathbf{\mu }}_{\mathbf{x}}}\\ {\mathbf{\mu }}_y& =\mathbf{g}\left({\mathbf{\mu }}_{\mathbf{x}}\right)\end{array}$$

$$\begin{array}{r}\begin{array}{rl}p\left(\mathbf{y}\right)& ={\int }_{-\infty }^{+\infty }p\left(\mathbf{y}|\mathbf{x}\right)p\left(\mathbf{x}\right)\mathrm{d}\mathbf{x}\\ & =\eta {\int }_{-\infty }^{+\infty }\exp \left\{-\frac{1}{2}{\left[\mathbf{y}-\left({\mathbf{\mu }}_y+\mathbf{G}\left(\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}\right)\right)\right]}^T{\mathbf{R}}^{-1}\left[\mathbf{y}-\left({\mathbf{\mu }}_y+\mathbf{G}\left(\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}\right)\right)\right]\right\}\\ & \times \exp \left[-\frac{1}{2}{\left(\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}\right)}^T{\mathbf{\Sigma }}_{xx}^{-1}\left(\mathbf{x}-{\mathbf{\mu }}_x\right)\right]\mathrm{d}\mathbf{x}\\ & =\eta {\int }_{-\infty }^{+\infty }\exp \left\{-\frac{1}{2}\left[{\left(\mathbf{y}-{\mathbf{\mu }}_y\right)}^T{\mathbf{R}}^{-1}\left(\mathbf{y}-{\mathbf{\mu }}_y\right)+{\left(\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}\right)}^T{\mathbf{G}}^T{\mathbf{R}}^{-1}\mathbf{G}\left(\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}\right)-2{\left(\mathbf{y}-{\mathbf{\mu }}_y\right)}^T{\mathbf{R}}^{-1}\mathbf{G}\left(\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}\right)+{\left(\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}\right)}^T{\Sigma }_{xx}^{-1}\left(\mathbf{x}-{\mathbf{\mu }}_x\right)\right]\right\}\mathrm{d}\mathbf{x}\\ & =\eta \exp \left[-\frac{1}{2}{\left(\mathbf{y}-{\mathbf{\mu }}_{\mathbf{y}}\right)}^T{\mathbf{R}}^{-1}\left(\mathbf{y}-{\mathbf{\mu }}_{\mathbf{y}}\right)\right]{\int }_{-\infty }^{+\infty }\exp \left\{-\frac{1}{2}\left[{\left(\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}\right)}^T\left({\mathbf{G}}^T{\mathbf{R}}^{-1}\mathbf{G}+{\Sigma }_{xx}^{-1}\right)\left(\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}\right)-2{\left(\mathbf{y}-{\mu }_y\right)}^T{\mathbf{R}}^{-1}\mathbf{G}\left(\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}\right)\right]\right\}\mathrm{d}\mathbf{x}\end{array}\end{array}$$
其中$\eta$是归一化常量，对积分号里的指数部分进行配方得到如下二次项
$${\left[\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}-\mathbf{F}\left(\mathbf{y}-{\mathbf{\mu }}_{\mathbf{y}}\right)\right]}^T\left({\mathbf{G}}^T{\mathbf{R}}^{-1}\mathbf{G}+{{\mathbf{\Sigma }}_{\mathbf{xx}}}^{-1}\right)\left[\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}-\mathbf{F}\left(\mathbf{y}-{\mathbf{\mu }}_y\right)\right]-{\left(\mathbf{y}-{\mathbf{\mu }}_y\right)}^T{\mathbf{F}}^T\left({\mathbf{G}}^T{\mathbf{R}}^{-1}\mathbf{G}+{{\mathbf{\Sigma }}_{\mathbf{xx}}}^{-1}\right)\mathbf{F}\left(\mathbf{y}-{\mathbf{\mu }}_{\mathbf{y}}\right)$$
其中第二个因子与$\mathbf{x}$无关，可以提取到积分外面，剩下的积分部分(第一个因子)就是$\mathbf{x}$的高斯分布，因此对$\mathbf{x}$积分可以得到一个常数，再与常数$\eta$合并。
$\mathbf{F}$ 满足如下等式：
$$\begin{array}{rl}{\left(\mathbf{y}-{\mathbf{\mu }}_{\mathbf{y}}\right)}^T{\mathbf{F}}^T\left({\mathbf{G}}^T{\mathbf{R}}^{-1}\mathbf{G}+{\mathbf{\Sigma }}_{xx}^{-1}\right)\left(\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}\right)& ={\left(\mathbf{y}-{\mathbf{\mu }}_{\mathbf{y}}\right)}^T{\mathbf{R}}^{-1}\mathbf{G}\left(\mathbf{x}-{\mathbf{\mu }}_{\mathbf{x}}\right)\\ {\mathbf{F}}^T\left({\mathbf{G}}^T{\mathbf{R}}^{-1}\mathbf{G}+{\mathbf{\Sigma }}_{xx}^{-1}\right)& ={\mathbf{R}}^{-1}\mathbf{G}\end{array}$$
于是：
$$\begin{array}{rl}p(\mathbf{y})& =\rho \exp \left\{-\frac{1}{2}{\left(\mathbf{y}-{\mathbf{\mu }}_{\mathbf{y}}\right)}^T\left[{\mathbf{R}}^{-1}-{\mathbf{F}}^T\left({\mathbf{G}}^T{\mathbf{R}}^{-1}\mathbf{G}+{\mathbf{\Sigma }}_{xx}^{-1}\right)\mathbf{F}\right]\left(\mathbf{y}-{\mathbf{\mu }}_{\mathbf{y}}\right)\right\}\\ & =\rho \exp \left\{-\frac{1}{2}{\left(\mathbf{y}-{\mathbf{\mu }}_{\mathbf{y}}\right)}^T\underset{由矩阵求逆引理(29)式得={\left(\mathbf{R}+\mathbf{G}{\mathbf{\Sigma }}_{xx}{\mathbf{G}}^{-1}\right)}^{-1}}{\underbrace{\left[{\mathbf{R}}^{-1}-{\mathbf{R}}^{-1}\mathbf{G}{\left({\mathbf{G}}^T{\mathbf{R}}^{-1}\mathbf{G}+{\mathbf{\Sigma }}_{xx}^{-1}\right)}^{-1}{\mathbf{G}}^T{\mathbf{R}}^{-1}\right]}}\left(\mathbf{y}-{\mathbf{\mu }}_{\mathbf{y}}\right)\right\}\\ & =\rho \exp \left[-\frac{1}{2}{\left(\mathbf{y}-{\mathbf{\mu }}_{\mathbf{y}}\right)}^T{\left(\mathbf{R}+\mathbf{G}{\mathbf{\Sigma }}_{xx}{\mathbf{G}}^{-1}\right)}^{-1}\left(\mathbf{y}-{\mathbf{\mu }}_{\mathbf{y}}\right)\right]\end{array}$$
其中$\rho$ 是一个新的归一化常量。该式即$\mathbf{x}$的高斯分布：
$$\mathbf{y}\sim \mathcal{N}({\mathbf{\mu }}_{\mathbf{y}},{\mathbf{\Sigma }}_{\mathbf{yy}})=\mathcal{N}(\underset{\mathbf{g}({\mathbf{\mu }}_{\mathbf{x}})}{\underbrace{{\mathbf{\mu }}_y}},\mathbf{R}+\mathbf{G}{\mathbf{\Sigma }}_{\mathbf{xx}}{\mathbf{G}}^T)$$
中间过程使用了矩阵求逆引理，见下文。

矩阵求逆引理

对于可逆矩阵，我们可以将它分解为一个下三角-对角-上三角(lower-diagnal-upper,LDU)形式或上三角-对角-下三角(upper-diagnal-lower,UDL)形式。

LDU

$$\begin{array}{r}\left[\begin{array}{cc}{\mathbf{A}}^{-1}& -\mathbf{B}\\ \mathbf{C}& \mathbf{D}\end{array}\right]=\left[\begin{array}{cc}\mathbf{I}& 0\\ \mathbf{C}\mathbf{A}& 1\end{array}\right]\left[\begin{array}{cc}{\mathbf{A}}^{-1}& 0\\ 0& \mathbf{D}+\mathbf{C}\mathbf{A}\mathbf{B}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& -\mathbf{A}\mathbf{B}\\ 0& \mathbf{I}\end{array}\right]\end{array}$$
对两侧求逆得：
$$\begin{array}{rl}{\left[\begin{array}{cc}{\mathbf{A}}^{-1}& -\mathbf{B}\\ \mathbf{C}& \mathbf{D}\end{array}\right]}^{-1}& =\left[\begin{array}{cc}\mathbf{I}& \mathbf{A}\mathbf{B}\\ 0& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}\mathbf{A}& 0\\ 0& {\left(\mathbf{D}+\mathbf{C}\mathbf{A}\mathbf{B}\right)}^{-1}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& 0\\ -\mathbf{C}\mathbf{A}& \mathbf{I}\end{array}\right]\\ & =\left[\begin{array}{cc}\mathbf{A}-\mathbf{A}\mathbf{B}{\left(\mathbf{D}+\mathbf{C}\mathbf{A}\mathbf{B}\right)}^{-1}\mathbf{C}\mathbf{A}& \mathbf{A}\mathbf{B}{\left(\mathbf{D}+\mathbf{C}\mathbf{A}\mathbf{B}\right)}^{-1}\\ -{\left(\mathbf{D}+\mathbf{C}\mathbf{A}\mathbf{B}\right)}^{-1}\mathbf{C}\mathbf{A}& {\left(\mathbf{D}+\mathbf{C}\mathbf{A}\mathbf{B}\right)}^{-1}\end{array}\right]\end{array}$$

UDL

$$\begin{array}{r}\left[\begin{array}{cc}{\mathbf{A}}^{-1}& -\mathbf{B}\\ \mathbf{C}& \mathbf{D}\end{array}\right]=\left[\begin{array}{cc}\mathbf{I}& -\mathbf{B}{\mathbf{D}}^{-1}\\ 0& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}{\mathbf{A}}^{-1}+\mathbf{B}{\mathbf{D}}^{-1}\mathbf{C}& 0\\ 0& \mathbf{D}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& 0\\ {\mathbf{D}}^{-1}\mathbf{C}& \mathbf{I}\end{array}\right]\end{array}$$
对两侧求逆得：
$$\begin{array}{rl}{\left[\begin{array}{cc}{\mathbf{A}}^{-1}& -\mathbf{B}\\ \mathbf{C}& \mathbf{D}\end{array}\right]}^{-1}& =\left[\begin{array}{cc}\mathbf{I}& 0\\ -{\mathbf{D}}^{-1}\mathbf{C}& \mathbf{I}\end{array}\right]\left[\begin{array}{cc}{\left({\mathbf{A}}^{-1}+\mathbf{B}{\mathbf{D}}^{-1}\mathbf{C}\right)}^{-1}& 0\\ 0& {\mathbf{D}}^{-1}\end{array}\right]\left[\begin{array}{cc}\mathbf{I}& \mathbf{B}{\mathbf{D}}^{-1}\\ 0& \mathbf{I}\end{array}\right]\\ & =\left[\begin{array}{cc}{\left({\mathbf{A}}^{-1}+\mathbf{B}{\mathbf{D}}^{-1}\mathbf{C}\right)}^{-1}& {\left({\mathbf{A}}^{-1}+\mathbf{B}{\mathbf{D}}^{-1}\mathbf{C}\right)}^1\mathbf{B}{\mathbf{D}}^{-1}\\ -{\mathbf{D}}^{-1}\mathbf{C}{\left({\mathbf{A}}^{-1}+\mathbf{B}{\mathbf{D}}^{-1}\mathbf{C}\right)}^{-1}& {\mathbf{D}}^{-1}-{\mathbf{D}}^{-1}\mathbf{C}{\left({\mathbf{A}}^{-1}+\mathbf{B}{\mathbf{D}}^{-1}\mathbf{C}\right)}^{-1}\mathbf{B}{\mathbf{D}}^{-1}\end{array}\right]\end{array}$$

SMW恒等式

比较得到如下恒等式：
$$\begin{array}{rlr}{\left({\mathbf{A}}^{-1}+\mathbf{B}{\mathbf{D}}^{-1}\mathbf{C}\right)}^{-1}& \equiv \mathbf{A}-\mathbf{A}\mathbf{B}{\left(\mathbf{D}+\mathbf{C}\mathbf{A}\mathbf{B}\right)}^{-1}\mathbf{C}\mathbf{A}& (1)\\ {\left(\mathbf{D}+\mathbf{C}\mathbf{A}\mathbf{B}\right)}^{-1}& \equiv {\mathbf{D}}^{-1}-{\mathbf{D}}^{-1}\mathbf{C}{\left({\mathbf{A}}^{-1}+\mathbf{B}{\mathbf{D}}^{-1}\mathbf{C}\right)}^{-1}\mathbf{B}{\mathbf{D}}^{-1}& (2)\\ \mathbf{A}\mathbf{B}{\left(\mathbf{D}+\mathbf{C}\mathbf{A}\mathbf{B}\right)}^{-1}& \equiv {\left({\mathbf{A}}^{-1}+\mathbf{B}{\mathbf{D}}^{-1}\mathbf{C}\right)}^{-1}\mathbf{B}{\mathbf{D}}^{-1}& (3)\\ {\left(\mathbf{D}+\mathbf{C}\mathbf{A}\mathbf{B}\right)}^{-1}\mathbf{C}\mathbf{A}& \equiv {\mathbf{D}}^{-1}\mathbf{C}{\left({\mathbf{A}}^{-1}+\mathbf{B}{\mathbf{D}}^{-1}\mathbf{C}\right)}^{-1}& (4)\end{array}$$