# 参数估计方法和非参数估计方法

ML 专栏收录该内容
76 篇文章 0 订阅

https://wenku.baidu.com/view/1cf9639efab069dc502201fe.html

1、最大似然估计MLE

$p(\theta|X) = \frac{p(X|\theta) \cdot p(\theta)}{p(X)}$

$posterior = \frac{likelihood \cdot prior}{evidence}$

$L(\theta | X) = p(X | \theta) = \prod_{x \in X}{p(X = x | \theta)}$

$\hat{\theta}_{ML} = argmax_\theta L(\theta | X) = argmax_\theta \sum_{x \in X}\log p(x|\theta)$

\begin{aligned} L &= \log\prod_{i=1}^Np(C=c_i|p)=\sum_{i=1}^N\log p(C=c_i|p) \\ &= n^{(1)}\log p(C = 1|p) + n^{(0)}\log p(C = 0|p)\\ &= n^{(1)}\log p + n^{(0)}\log (1-p) \end{aligned}

$\frac{\partial{L}} {\partial{p}} = \frac{n^{(1)}}{p} - \frac{n^{(0)}}{1-p} = 0$

$\hat{p}_{ML} = \frac{n^{(1)}}{n^{(1)} + n^{(0)}} = \frac{n^{(1)}}{N}$

2、最大后验估计MAP

\begin{aligned} \hat{\theta}_{MAP} &= argmax_\theta \frac{p(X | \theta) p(\theta)}{p(X)}\\ &= argmax_\theta p(X | \theta)p(\theta) \\ &= argmax_\theta \{L(\theta|X) + \log p(\theta)\}\\ &= argmax_\theta \{\sum_{x \in X} \log p(x | \theta) + \log p(\theta)\} \end{aligned}

$p(\theta)= p(\theta|\alpha)$

$p(\tilde{x}|X) = \int_{\theta \in \Theta}p(\tilde{x}|\hat{\theta}_{MAP}) p(\theta | X) d\theta = p(\tilde{x}|\hat{\theta}_{MAP})$

$p(p|\alpha, \beta) = \frac{1}{B(\alpha, \beta)}p^{\alpha - 1}(1-p)^{\beta - 1} \stackrel{\triangle}{=}Beta(p|\alpha, \beta)$

$B(\alpha, \beta) = \frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha + \beta)}$

Beta分布的随机变量范围是[0,1],所以可以生成normalised probability values。下图给出了不同参数情况下的Beta分布的概率密度函数

$\frac{\partial \hat\theta_{MAP}}{\partial p} = \frac{n^{(1)}}{p}-\frac{n^{(0)}}{1-p}+\frac{\alpha - 1}{p}-\frac{\beta - 1}{1 - p} = 0$

$\hat{p}_{MAP} = \frac{n^{(1)} + \alpha - 1}{n^{(1)} + n^{(0)} + \alpha + \beta - 2} = \frac{n^{(1)} + 4}{n^{(1)} + n^{(0)} + 8}$

3 贝叶斯估计

$p(\theta|X) = \frac{p(X|\theta) \cdot p(\theta)}{p(X)}$

$p(X) = \int_{\theta \in \Theta}p(X|\theta)p(\theta)d\theta$

$p(\hat{x}|X) = \int_{\theta \in \Theta} p(\hat{x} | \theta)p(\theta|X)d\theta=\int_{\theta \in \Theta}p(\hat{x}|\theta)\frac{p(X|\theta)p(\theta)}{p(X)}d\theta$

$\int_p\prod_{t=1}^{|T|}P_t^{\alpha_t - 1} = B(\alpha)$

Gregor Heinrich, Parameter estimation for test analysis, technical report

Wikipedia Beta分布词条 ,  http://en.wikipedia.org/wiki/Beta_distribution

• 1
点赞
• 0
评论
• 7
收藏
• 扫一扫，分享海报

04-11
09-03 1万+

04-03 6056
08-16 2929
08-06 2万+
05-06 3万+
02-24 1765
04-25 3万+
05-23 1万+
08-07 3万+
11-07 1412
04-06 1万+
03-17 2306