HongYiLee Classification Notes

HongYiLee Classification Notes

标签： Notes MechineLearning DeepLearning Classification

---

Classification分类问题

Generative Model 生成模型

$P(x)=P(x|C_1)P(C1)+P(x)P(x|C2)$通过先验概率$P(C_1)$和$P(C_2)$生成$P(x)$

Prior 先验概率$P(C_1)$和$P(C_2)$

Assume the points are sampled from a Gaussion Distribution高斯分布。
Given the points how to find the Gaussion Distribution.

But the Gaussion Distribution with any mean $mu$ and covariance matrix $\Sigma$ can generative these points. So we use the Likelihood Function.似然函数。

We want to find the Maximum Likelihood on this Likelihood Function.
$\mu^*,\Sigma^*=argmax_{\mu,\Sigma}(\mu,\Sigma)$
which: $\mu^*=\frac{1}{n}\sum_{i=1}^{n}{x^i}$
and: $\Sigma^*=\frac{1}{n}\sum_{i=1}^{n}{(x^i-x^*)(x^i-x^*)^T}$

So Class1 is a Gaussion distribution and Class2 is a Gaussion distribution, once we get each $\mu,\Sigma$ of the distribution, we can caclate the problem.

How to Modifying the Model

Different Distribution can share the same $\Sigma$. 应该要share怎么样的一个$\Sigma$比较合适呢，可以将两个Class的$\Sigma$加权平均。变成两个类共同share的一个协方差矩阵。
When We share the same covariance, the boundary become linear and the perfomance become better.当共用一个协方差矩阵的时候，边界变成了线性的并且表现变得更好。

Posterior Probabllity 后验概率(精彩推导，马上开始)

$$P(C_1|x)=\frac{P(x|C_1)P(C_1)}{P(x|C_1)P(C_1)+P(x|2)P(C_2)} =\frac{1}{1+\frac{P(x|C_2)P(C_2)}{P(x|C_1)P(C_1)}} =\frac{1}{1+e^{-z}}$$
Here we let $z=ln\frac{P(x|C_1)P(C_1)}{P(x|C_2)P(C_2)}$.看起来很眼熟吗？没错这就是 sigmod函数啦！
$$z=w^Tx+b$$
$$P(C_1|x)=\frac{1}{1+e^{-z}}=\sigma(z)=\sigma(w^Tx+b)$$

So we got Logistic Regression.