条件概率与连式法则

条件概率与连式法则

条件概率公式:
\[ p(y|x) = \frac{p(x,y)}{p(x)} \tag{1} \]
链式法则:
\[ \begin{align} p(x_1,x_2,...,x_n) &= p(x_n)\prod_{i=1}^{n-1}{{p(x_i)}{p(x_i|x_{i+1},x_{i+2},...,x_{n})}} \nonumber\\ &= p(x_1)\prod_{i=2}^{n}{p(x_i|x_{1},x_{2}...,x_{i-1})} \tag{2} \end{align} \]
其实, 链式法则就是根据条件概率公式推导得来, 例如:
\[ \begin{align} p(a,b,c,d) &= p(a|b,c,d)p(b,c,d) \nonumber\\ &= p(a|b,c,d)p(b|c,d)p(c,d) \nonumber \\ &= p(a|b,c,d)p(b|c,d)p(c|d)p(d) \tag{3} \end{align} \]

\[ \begin{align} p(a,b,c,d) &= p(d|a,b,c)p(a,b,c) \nonumber \\ &= p(d|a,b,c)p(c|a,b)p(a,b) \nonumber \\ &= p(d|a,b,c)p(c|a,b)p(b|a)p(a) \tag{4} \end{align} \]

例子, 假设有样本\({(x_1, y_1),...,(x_n, y_n)}\), \(x_i\)和\(y_i\)分别表示第\(i\)个样本的特征和类别, \(1 \le i \le n​\), \(x_i\) 与 \(x_j\) 相互独立, \(x_i\) 与 \(y_j\) 相互独立, 那么有
\[ \begin{align} p(x_1,x_2,...,x_n,y_1,y_2,...,y_n) &= p(y_1,y_2,...,y_n)p(x_1,x_2,...,x_n|y_1,y_2,...,y_n) \nonumber \\ &= p(y_1,y_2,...,y_n) \cdot \prod_{i=1}^{n}{p(x_i|y_i)} \tag{5} \end{align} \]
又有假设后面的两本只与前面两个样本的取值有关(NLP等领域常用), 则
\[ p(y_1,y_2,...,y_n) = \prod_{i=1}^{n}p(y_i|y_{i-1},y_{i-2}) \tag{6} \]
其中\(y_{0} = y_{-1} = *​\)
\[ p(x_1,x_2,...,x_n|y_1,y_2,...,y_n) = \prod_{i=1}^{n}{p(x_i|x_{i-1},...,x_1,y_1,...y_n)} \tag{7} \]
\(p(x_i|x_{i-1},...,x_1,y_1,...y_n)\)表示已知\(x_{i-1},...,x_1,y_1,...y_n\)的条件下, 取值为\(x_i\)的概率.

转载于:https://www.cnblogs.com/ouyxy/p/10625972.html