一维/二维高斯分布的负对数似然推导

参考wikipedia Multivariate normal distribution 及 Normal distribution

基础公式

多元高斯分布公式
$$f(X)=f(x_1,x_2,...,x_k)=\frac{1}{\sqrt{{(2\pi )}^k|\sum |}}{e}^{-\frac{1}{2}(X-\mu {)}^T{\sum }^{-1}(X-\mu )}$$
其中，$\mu =({\mu }_{x_1},{\mu }_{x_2},...,{\mu }_{x_k})$
那么一维高斯分布公式
$$f(x)=\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{(x-{\mu }_x{)}^2}{2{\sigma }^2}}$$
二维高斯分布公式
$$f(x,y)=\frac{1}{2\pi \sqrt{|\sum |}}{e}^{-\frac{1}{2}\left(\begin{array}{cc}x-{\mu }_x& y-{\mu }_y\end{array}\right){\sum }^{-1}\left(\begin{array}{c}x-{\mu }_x\\ y-{\mu }_y\end{array}\right)}$$

为简化推导，后文将 $d_x=x-{\mu }_x$ 记为x

一维高斯分布的负对数似然

$$\begin{gathered} NLL(x,\mu |{\sigma }^2)=-log(f(x))=-log(\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-\frac{x^2}{2{\sigma }^2}}) \\ =-(-log(\sqrt{2\pi {\sigma }^2})-\frac{x^2}{2{\sigma }^2}) \\ =\frac{log(2\pi )}{2}+log(\sigma )+\frac{x^2}{2{\sigma }^2} \end{gathered}$$

二维高斯分布的负对数似然

二维高斯分布公式展开如下
$$\begin{array}{rl}f(x,y)& =\frac{1}{2\pi \sqrt{|\sum |}}{e}^{-\frac{1}{2}\left(\begin{array}{cc}x& y\end{array}\right){\sum }^{-1}\left(\begin{array}{c}x\\ y\end{array}\right)}\\ & =\frac{1}{2\pi \sqrt{|\sum |}}{e}^{-\frac{1}{2}\frac{x^2{\sigma }_y^2-2\rho xy{\sigma }_x{\sigma }_y+y^2{\sigma }_x^2}{det(|\sum |)}}\\ & =\frac{1}{2\pi \sqrt{|\sum |}}{e}^{-\frac{1}{2}\frac{x^2{\sigma }_y^2-2\rho xy{\sigma }_x{\sigma }_y+y^2{\sigma }_x^2}{(1-{\rho }^2){\sigma }_x^2{\sigma }_y^2}}\\ & =\frac{1}{2\pi {\sigma }_x{\sigma }_y\sqrt{1-{\rho }^2}}{e}^{-\frac{1}{2(1-{\rho }^2)}(\frac{x^2}{{\sigma }_x^2}-\frac{2\rho xy}{{\sigma }_x{\sigma }_y}+\frac{y^2}{{\sigma }_y^2})}\end{array}$$
其中，
$$\sum =\left(\begin{array}{cc}{{\sigma }_x}^2& {\sigma }_{xy}\\ {\sigma }_{xy}& {{\sigma }_y}^2\end{array}\right)=\left(\begin{array}{cc}{{\sigma }_x}^2& \rho {\sigma }_x{\sigma }_y\\ \rho {\sigma }_x{\sigma }_y& {{\sigma }_y}^2\end{array}\right)$$
$${\sum }^{-1}=\frac{1}{|\sum |}\left(\begin{array}{cc}{\sigma }_y^2& -\rho {\sigma }_x{\sigma }_y\\ -\rho {\sigma }_x{\sigma }_y& {\sigma }_x^2\end{array}\right)$$
那么负对数似然为
$$\begin{array}{rl}-log(f(x,y))& =-log(\frac{1}{2\pi {\sigma }_x{\sigma }_y\sqrt{1-{\rho }^2}}{e}^{-\frac{1}{2(1-{\rho }^2)}(\frac{x^2}{{\sigma }_x^2}-\frac{2\rho xy}{{\sigma }_x{\sigma }_y}+\frac{y^2}{{\sigma }_y^2})})\\ & =-(-log(2\pi {\sigma }_x{\sigma }_y)-\frac{1}{2}log(1-{\rho }^2)+(-\frac{1}{2(1-{\rho }^2)}(\frac{x^2}{{\sigma }_x^2}-\frac{2\rho xy}{{\sigma }_x{\sigma }_y}+\frac{y^2}{{\sigma }_y^2})))\\ & =log(2\pi {\sigma }_x{\sigma }_y)+\frac{1}{2}log(1-{\rho }^2)+\frac{1}{2(1-{\rho }^2)}(\frac{x^2}{{\sigma }_x^2}-\frac{2\rho xy}{{\sigma }_x{\sigma }_y}+\frac{y^2}{{\sigma }_y^2})\end{array}$$