离散变量贝叶斯决策简介

• 贝叶斯决策
  最小风险：
  $$\min R\left({\alpha }_i\mid \mathrm{x}\right)=\sum _{j=1}^c\lambda \left({\alpha }_i\mid {\omega }_j\right)P\left({\omega }_j\mid \mathrm{x}\right)$$
  最小错误率（最大后验概率）：$\max P\left({\omega }_i\mid \mathbf{x}\right)$
• 独立二值特征（binary features）
  – 独立
  $$p(\mathbf{x})=p\left(x_1{x}_2\cdots x_d\right)=\prod _{i=1}^{d}p\left(x_i\right)$$
  – 二值，这边假设只有两类，样本是$d$维的
  $$\begin{array}{rl}& p_i=\mathrm{Prob}\left(x_i=1\mid {\omega }_1\right)i=1,\dots ,d\\ & q_i=\mathrm{Prob}\left(x_i=1\mid {\omega }_2\right)i=1,\dots ,d\end{array}$$
  – 类条件概率密度
  $$\begin{array}{rl}& P\left(\mathbf{x}\mid {\omega }_1\right)=\prod _{i=1}^d{p}_i^{x_i}{\left(1-p_i\right)}^{1-x_i}\\ & P\left(\mathbf{x}\mid {\omega }_2\right)=\prod _{i=1}^d{q}_i^{x_i}{\left(1-q_i\right)}^{1-x_i}\end{array}$$
  $x_i$是样本的第$i$维特征，取值$0,1$。如果$x_i=0$，则$p_i^{x_i}{\left(1-p_i\right)}^{1-x_i}=1-p_i$；如果$x_i=1$，则$p_i^{x_i}{\left(1-p_i\right)}^{1-x_i}=p_i$
  – 似然比
  $$\frac{P\left(\mathrm{x}\mid {\omega }_1\right)}{P\left(\mathrm{x}\mid {\omega }_2\right)}=\prod _{i=1}^d{\left(\frac{p_i}{q_i}\right)}^{x_i}{\left(\frac{1-p_i}{1-q_i}\right)}^{1-x_i}$$
  – 判别函数
  $$g(\mathbf{x})=\log \frac{p\left(\mathbf{x}\mid {\omega }_1\right)P\left({\omega }_1\right)}{p\left(\mathbf{x}\mid {\omega }_2\right)P\left({\omega }_2\right)}=\sum _{i=1}^d\left[x_i\ln \frac{p_i}{q_i}+\left(1-x_i\right)\ln \frac{1-p_i}{1-q_i}\right]+\ln \frac{P\left({\omega }_1\right)}{P\left({\omega }_2\right)}$$
  上式为线性判别函数
  $$g(\mathbf{x})=\sum _{i=1}^d{w}_i{x}_i+w_0$$
  $$\begin{array}{rl}& w_i=\ln \frac{p_i\left(1-q_i\right)}{q_i\left(1-p_i\right)}i=1,\dots ,d\\ & w_0=\sum _{i=1}^d\ln \frac{1-p_i}{1-q_i}+\ln \frac{P\left({\omega }_1\right)}{P\left({\omega }_2\right)}\end{array}$$

【例一】： 若$P\left({\omega }_1\right)=0.5,P\left({\omega }_2\right)=0.5$，$p_i=0.8,q_i=0.5,i=1,2,3$
$$\begin{array}{rl}& g(\mathbf{x})=\sum _{i=1}^d{w}_i{x}_i+w_0\\ & w_i=\ln \frac{.8(1-.5)}{.5(1-.8)}=1.3863\\ & w_0=\sum _{i=1}^3\ln \frac{1-.8}{1-.5}+\ln \frac{.5}{.5}=-2.7489\end{array}$$

【例二】： 若$P\left({\omega }_1\right)=0.5,P\left({\omega }_2\right)=0.5$，$p_1=p_2=0.8,p_3=0.5;q_i=0.5,i=1,2,3$
$$\begin{array}{rl}& w_1=w_2=\ln \frac{.8(1-.5)}{.5(1-.8)}=1.3863\\ & w_3=0\\ & {\mathrm{w}}_0=2\ln \frac{1-0.8}{1-0.5}=-1.8326\end{array}$$