807补充（四）（高斯积分篇）

807补充（四）（高斯积分篇）

一.Jacobian矩阵（雅可比矩阵）

Jacobian矩阵可被视为是一种组织梯度向量的方法。在前三篇中，我们给出了梯度的定义与计算公式，借助梯度可以简单得出雅可比矩阵的定义。
$${\text{D}}_{\mathbf{x}}\mathbf{f}\stackrel{\text{def}}{=}({\nabla }_{\mathbf{x}}\mathbf{f}{)}^T$$
易看出当雅可比矩阵为方阵时$\mathbf{f}(\mathbf{x})与$$\mathbf{x}$是同维度向量，若雅可比矩阵不为方阵，则从$\mathbf{f}(\mathbf{x})$到$\mathbf{x}$的映射是降维映射或升维映射。

在微分几何中雅可比矩阵可以衡量两个函数之间的变换是否光滑。

二.Jacobian行列式

当雅可比矩阵为方阵时，雅可比矩阵的行列式可用于重积分换元，在二维情况下，有以下二重积分换元成立
$$\begin{array}{rl}{\iint }_{\text{D1}}f(x,y)\text{dxdy}& ={\iint }_{\text{D2}}f(g(u,v),h(u,v))|J|\text{dudv}\\ x& =g(u,v)\\ y& =h(u,v)\end{array}$$
其中
$$\begin{array}{rl}J& =|\frac{\partial x\partial y}{\partial u\partial v}|\\ & =\det (\left[\begin{array}{cc}\frac{\partial x}{\partial u}& \frac{\partial x}{\partial v}\\ \frac{\partial y}{\partial u}& \frac{\partial y}{\partial v}\end{array}\right])\end{array}$$
即雅可比矩阵行列式,$\text{D1,D2}$是变换前后的区域。

推广至n维空间也成立。

具体证明过程可参考

三.高斯积分

“物理学家只学会了求解高斯积分”

1. $I={\int }_{-\infty }^{\infty }{e}^{-\frac{x^2}{2}}\text{dx}$

$$\begin{array}{rl}I& =\sqrt{I^2}\\ & =\sqrt{{\int }_{-\infty }^{\infty }{e}^{-\frac{x^2}{2}}\text{dx}{\int }_{-\infty }^{\infty }{e}^{-\frac{y^2}{2}}\text{dy}}\\ & =\sqrt{{\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }{e}^{-1/2\ast (x^2+y^2)}\text{dxdy}}\\ & =\sqrt{{\int }_0^{2\pi }\text{d}\theta {\int }_0^{\infty }{e}^{-1/2\ast r^2}r\text{dr}}\\ & =\sqrt{2\pi }\end{array}$$

2.$I={\int }_{-\infty }^{\infty }{e}^{-\frac{a{x}^2}{2}+jx}\text{dx}$
$$\begin{array}{rl}I& ={\int }_{-\infty }^{\infty }{e}^{-\frac{a{x}^2}{2}+jx}\text{dx}\\ & ={\int }_{-\infty }^{\infty }{e}^{-\frac{1}{2}a(x^2-\frac{2j}{a}x+\frac{j^2}{a^2})+\frac{j^2}{2a^2}}\text{dx}\\ & =e^{\frac{j^2}{2a^2}}{\int }_{-\infty }^{\infty }{e}^{-\frac{1}{2}a(x-\frac{j}{a}{)}^2}\text{d}x\\ & =e^{\frac{j^2}{2a^2}}\sqrt{\frac{2\pi }{a}}\end{array}$$

​ 3.n重高斯积分:$I={\int }_{-\infty }^{\infty }{e}^{-\frac{1}{2}{\sum }_{i=1}^n{x}_i^2}\text{d}{x}_1\text{d}{x}_2\cdots \text{d}{x}_n={\int }_{-\infty }^{\infty }{e}^{-\frac{1}{2}{\mathbf{x}}^T\mathbf{x}}\text{d}\mathbf{x}$

​ 由1易知$I=(2\pi {)}^{\frac{n}{2}}$

4.二次型任意：$I={\int }_{-\infty }^{\infty }{e}^{-\frac{1}{2}{\mathbf{x}}^T\mathbf{K}\mathbf{x}}\text{d}\mathbf{x}$,$\mathbf{K}$为正定矩阵。

​ 做分解$\mathbf{K}={\mathbf{S}}^T\mathbf{S}$,使得$-\frac{1}{2}{\mathbf{x}}^T\mathbf{K}\mathbf{x}=-\frac{1}{2}(\mathbf{S}\mathbf{x}{)}^T(\mathbf{S}\mathbf{x})$,令$\mathbf{y}=\mathbf{S}\mathbf{x}$
$$\begin{array}{rl}I& ={\int }_{-\infty }^{\infty }{e}^{-\frac{1}{2}{\mathbf{x}}^T\mathbf{K}\mathbf{x}}\text{d}\mathbf{x}\\ & ={\int }_{-\infty }^{\infty }{e}^{-\frac{1}{2}(\mathbf{S}\mathbf{x}{)}^T(\mathbf{S}\mathbf{x})}\text{d}\mathbf{x}\\ & ={\int }_{-\infty }^{\infty }{e}^{-\frac{1}{2}{\mathbf{y}}^T\mathbf{y}}|\frac{\partial \mathbf{x}}{\partial \mathbf{y}}|\text{d}\mathbf{y}\\ & =|\frac{\partial \mathbf{x}}{\partial \mathbf{y}}|\ast (2\pi {)}^{\frac{n}{2}}\\ & =|\mathbf{S}{|}^{-1}\ast (2\pi {)}^{\frac{n}{2}}\\ & =|\mathbf{K}{|}^{-1/2}\ast (2\pi {)}^{\frac{n}{2}}\\ & =\sqrt{\frac{(2\pi {)}^n}{\det \mathbf{K}}}\end{array}$$
5.$I={\int }_{-\infty }^{\infty }{e}^{-\frac{1}{2}{\mathbf{x}}^T\mathbf{K}\mathbf{x}+{\mathbf{b}}^T\mathbf{x}}\text{d}\mathbf{x}$
$$-\frac{1}{2}{\mathbf{x}}^T\mathbf{K}\mathbf{x}+{\mathbf{b}}^T\mathbf{x}=-\frac{1}{2}{\left(\mathbf{x}-{\mathbf{K}}^{-1}\mathbf{b}\right)}^T\mathbf{K}\left(\mathbf{x}-{\mathbf{K}}^{-1}\mathbf{b}\right)+\frac{1}{2}{\mathbf{b}}^T{\mathbf{K}}^{-1}\mathbf{b}$$

​ 可知$I=\sqrt{\frac{(2\pi {)}^n}{\det \mathbf{K}}}{e}^{\frac{1}{2}{\mathbf{b}}^T{\mathbf{K}}^{-1}\mathbf{b}}$

四.例

（1）n维高斯分布的表达式为$p(\mathbf{x})=\frac{1}{(2\pi {)}^{\frac{n}{2}}}\frac{1}{\sqrt{\det |\Sigma |}}{e}^{-\frac{1}{2}(\mathbf{x}-\mu {)}^T{\Sigma }^{-1}(\mathbf{x}-\mu )}$
$$\begin{array}{rl}{\int }_{-\infty }^{\infty }p(\mathbf{x})\text{d}\mathbf{x}& =\frac{1}{(2\pi {)}^{\frac{n}{2}}}\frac{1}{\sqrt{\det |\Sigma |}}{\int }_{-\infty }^{\infty }{e}^{-\frac{1}{2}(\mathbf{x}-\mu {)}^T{\Sigma }^{-1}(\mathbf{x}-\mu )}\text{d}\mathbf{x}\\ & =\frac{1}{(2\pi {)}^{\frac{n}{2}}}\frac{1}{\sqrt{\det |\Sigma |}}\ast {\int }_{-\infty }^{\infty }{e}^{-\frac{1}{2}{\mathbf{x}}^T{\Sigma }^{-1}\mathbf{x}}\text{d}\mathbf{x}\\ & =1\end{array}$$
​ (2)巴氏距离（Bhattacharyya distance),其定义为
$$BD(p(\mathbf{x}),q(\mathbf{x}))=-\log \int \sqrt{p(\mathbf{x}),q(\mathbf{x})}\text{d}\mathbf{x}$$
​ 对于两个正态分布来说，它们的巴氏距离是以下积分的负对数
$$\begin{array}{rl}\int \sqrt{p(\mathbf{x})q(\mathbf{x})}d\mathbf{x}& =\frac{1}{\sqrt[4]{(2\pi {)}^{2n}\det \left({\mathbf{\Sigma }}_p{\mathbf{\Sigma }}_q\right)}}\times \int \exp \left\{-\frac{1}{4}{\left(\mathbf{x}-{\mathbf{\mu }}_p\right)}^{\top }{\mathbf{\Sigma }}_p^{-1}\left(\mathbf{x}-{\mathbf{\mu }}_p\right)-\frac{1}{4}{\left(\mathbf{x}-{\mathbf{\mu }}_q\right)}^{\top }{\mathbf{\Sigma }}_q^{-1}\left(\mathbf{x}-{\mathbf{\mu }}_q\right)\right\}d\mathbf{x}\end{array}$$
记$\mathbf{y}=\mathbf{x}-{\mathbf{\mu }}_p,\mathbf{\Delta }={\mathbf{\mu }}_p-{\mathbf{\mu }}_q$,换元可得
$$\begin{array}{rl}& \int \exp \left\{-\frac{1}{4}{\mathbf{y}}^{\top }{\mathbf{\Sigma }}_p^{-1}\mathbf{y}-\frac{1}{4}(\mathbf{y}+\mathbf{\Delta }{)}^{\top }{\mathbf{\Sigma }}_q^{-1}(\mathbf{y}+\mathbf{\Delta })\right\}d\mathbf{y}\\ =& \int \exp \left\{-\frac{1}{4}{\mathbf{y}}^{\top }\left({\mathbf{\Sigma }}_p^{-1}+{\mathbf{\Sigma }}_q^{-1}\right)\mathbf{y}-\frac{1}{2}{\mathbf{\Delta }}^{\top }{\mathbf{\Sigma }}_q^{-1}\mathbf{y}-\frac{1}{4}{\mathbf{\Delta }}^{\top }{\mathbf{\Sigma }}_q^{-1}\mathbf{\Delta }\right\}d\mathbf{y}\\ =& \int \exp \left\{-\frac{1}{2}{\mathbf{y}}^{\top }\left({\mathbf{\Sigma }}_p^{-1}\mathbf{\Sigma }{\mathbf{\Sigma }}_q^{-1}\right)\mathbf{y}-\frac{1}{2}{\mathbf{\Delta }}^{\top }{\mathbf{\Sigma }}_q^{-1}\mathbf{y}-\frac{1}{4}{\mathbf{\Delta }}^{\top }{\mathbf{\Sigma }}_q^{-1}\mathbf{\Delta }\right\}d\mathbf{y}\end{array}$$
其中$\mathbf{\Sigma }=\frac{1}{2}\left({\mathbf{\Sigma }}_p+{\mathbf{\Sigma }}_q\right)$

最后积分的结果为$\sqrt{(2\pi {)}^n\det \left({\mathbf{\Sigma }}_q{\mathbf{\Sigma }}^{-1}{\mathbf{\Sigma }}_p\right)}\exp \left\{-\frac{1}{8}{\mathbf{\Delta }}^{\top }{\mathbf{\Sigma }}^{-1}\mathbf{\Delta }\right\}$

也可直接利用如下公式计算
$$\begin{array}{rl}& -\frac{1}{2}{\left(\mathbf{x}-{\mathbf{m}}_1\right)}^T{\mathbf{\Sigma }}_1^{-1}\left(\mathbf{x}-{\mathbf{m}}_1\right)-\frac{1}{2}{\left(\mathbf{x}-{\mathbf{m}}_2\right)}^T{\mathbf{\Sigma }}_2^{-1}\left(\mathbf{x}-{\mathbf{m}}_2\right)\\ & =-\frac{1}{2}{\left(\mathbf{x}-{\mathbf{m}}_c\right)}^T{\mathbf{\Sigma }}_c^{-1}\left(\mathbf{x}-{\mathbf{m}}_c\right)+C\\ & {\mathbf{\Sigma }}_c^{-1}={\mathbf{\Sigma }}_1^{-1}+{\mathbf{\Sigma }}_2^{-1}\\ & {\mathbf{m}}_c={\left({\mathbf{\Sigma }}_1^{-1}+{\mathbf{\Sigma }}_2^{-1}\right)}^{-1}\left({\mathbf{\Sigma }}_1^{-1}{\mathbf{m}}_1+{\mathbf{\Sigma }}_2^{-1}{\mathbf{m}}_2\right)\\ & C=\frac{1}{2}\left({\mathbf{m}}_1^T{\mathbf{\Sigma }}_1^{-1}+{\mathbf{m}}_2^T{\mathbf{\Sigma }}_2^{-1}\right){\left({\mathbf{\Sigma }}_1^{-1}+{\mathbf{\Sigma }}_2^{-1}\right)}^{-1}\left({\mathbf{\Sigma }}_1^{-1}{\mathbf{m}}_1+{\mathbf{\Sigma }}_2^{-1}{\mathbf{m}}_2\right)\\ & -\frac{1}{2}\left({\mathbf{m}}_1^T{\mathbf{\Sigma }}_1^{-1}{\mathbf{m}}_1+{\mathbf{m}}_2^T{\mathbf{\Sigma }}_2^{-1}{\mathbf{m}}_2\right)\\ & \end{array}$$

所以最终
$$\begin{array}{rl}BD(p(\mathbf{x}),q(\mathbf{x}))& =-\log \frac{\sqrt{(2\pi {)}^n\det \left({\mathbf{\Sigma }}_q{\mathbf{\Sigma }}^{-1}{\mathbf{\Sigma }}_p\right)}}{\sqrt[4]{(2\pi {)}^{2n}\det \left({\mathbf{\Sigma }}_p{\mathbf{\Sigma }}_q\right)}}\exp \left\{-\frac{1}{8}{\mathbf{\Delta }}^{\top }{\mathbf{\Sigma }}^{-1}\mathbf{\Delta }\right\}\\ & =-\log \frac{\sqrt[4]{\det \left({\mathbf{\Sigma }}_p{\mathbf{\Sigma }}_q\right)}}{\sqrt{\det (\mathbf{\Sigma })}}\exp \left\{-\frac{1}{8}{\mathbf{\Delta }}^{\top }{\mathbf{\Sigma }}^{-1}\mathbf{\Delta }\right\}\\ & =\frac{1}{2}\log \frac{\det (\mathbf{\Sigma })}{\sqrt{\det \left({\mathbf{\Sigma }}_p{\mathbf{\Sigma }}_q\right)}}+\frac{1}{8}{\left({\mathbf{\mu }}_p-{\mathbf{\mu }}_q\right)}^{\top }{\mathbf{\Sigma }}^{-1}\left({\mathbf{\mu }}_p-{\mathbf{\mu }}_q\right)\end{array}$$