【高斯分布】01-极大似然估计

高斯分布

输入数据:$$X=(x_1,x_2,...,x_n{)}^T=\left(\begin{array}{c}{x}_1^T\\ x_2^T\\ ...\\ x_n^T\end{array}\right)$$
$x_i\in R^p,x_i{\sim }^{iid}N(\mu ,\Sigma ),\theta =(\mu ,\Sigma )$

iid指独立同分布
回顾：
1.数学期望是对随机变量中心位置的一种度量，是每次实验中可能的结果乘以其结果的总和
$E(x)=xf(x)$
方差就是这种风险的度量，即随机变量的变异性
$E(x)=(x-\mu )f(x)$
2.独立：一个事件的发生不依赖于另外一个事件，两个事件同时发生的概率为P(AB) = P(A)·P(B)
独立同分布：各事件相互独立，但满足同一个概率分布

$MLE:{\theta }_{MLE}=argma{x}_{(\theta )}P(X|\theta )$
$令p=1,\theta =(\mu ,{\sigma }^2)$
$$\begin{array}{lcr}P(x)& =& \frac{1}{\sqrt{2\pi }\sigma }exp(-\frac{(x-\mu {)}^2}{2{\sigma }^2})\\ P(x)& =& \frac{1}{\sqrt{2\pi }\frac{p}{2}|\sum {|}^{\frac{1}{2}}}exp(-\frac{1}{2}(x-\mu {)}^T{\sum }^{-1}(x-\mu )))\\ & & \\ logP(x|\theta )& =& log{\prod }_{i=1}^{N}p(x_i|\theta )\\ & =& {\sum }_{i=1}^{N}logp(x_i|\theta )\\ & =& {\sum }_{i=1}^{N}log\frac{1}{\sqrt{2\pi }\sigma }exp(-\frac{(x-\mu {)}^2}{2{\sigma }^2})\\ & =& {\sum }_{i=1}^N[log\frac{1}{\sqrt{2\pi }}+log\frac{1}{\sigma }-\frac{(x_i-\mu {)}^2}{2{\sigma }^2}]\\ & & \\ {\mu }_{MLE}& =& argma{x}_{\mu }logP(x|\theta )\\ & =& argmax{\sum }_{i=1}^N-\frac{x_i-\mu }{2{\sigma }^2}\\ & =& argma{x}_{u}min{\sum }_{i=1}N-\frac{x_i-\mu }{2{\sigma }^2}\\ & & \\ \frac{\partial }{\partial \mu }\sum (x_i-\mu {)}^2& =& {\sum }_{i=1}^{N}2(x_i-\mu )(-1)=0\\ {\sum }_{i=1}^N(x_i-\mu )& =& 0\end{array}$$

$$\begin{array}{lcr}{\mu }_{MLE}& =& \frac{1}{N}{\sum }_{i=1}^N{x}_i(无偏估计)\\ E[{\mu }_{MLE}]& =& \frac{1}{N}{\sum }_{i=1}^{N}E[x_i]\\ & =& \frac{1}{N}{\sum }_{i=1}^N\mu \\ & =& \frac{1}{N}N\mu \\ & =& \mu \end{array}$$

$\begin{array}{lcrcc}{\sigma }_{MLE}^2& =& argma{x}_{\sigma }P(x|\sigma )& & \\ & =& argmax\sum (-log\sigma -\frac{1}{2{\sigma }^2})& & \\ & & & & \\ \frac{\partial \wp }{\partial \sigma }& =& {\sum }_{i=1}^N[-\frac{1}{\sigma }+\frac{1}{2}(x_i-\mu {)}^2(+2){\sigma }^{-3}]& =& 0\\ {\sum }_{i=1}^N[-\frac{1}{\sigma }+(x_i-\mu {)}^2{\sigma }^{-3}]& =& 0& & \\ -{\sum }_{i=1}^N{\sigma }^2+{\sum }_{i=1}^N(x_i-\mu {)}^2& =& 0& & \\ {\sum }_{i=1}^N{\sigma }^2& =& {\sum }_{i=1}^N(x_i-\mu {)}^2& & \\ {\sigma }_{MLE}^2& =& \frac{1}{N}{\sum }_{i=1}N(x_i-\mu {)}^2(有偏估计)& & \\ & & & & \\ E[{\sigma }_{MLE}^2]& =& \frac{N-1}{N}{\sigma }^2& & \\ \hat{\sigma }& =& \frac{1}{N-1}{\sum }_{i=1}^N(x_i-{\mu }_{MLE})(无偏估计)& & \end{array}$

参考资料

1.板书 高斯分布1-极大似然估计