NLP
语言模型
最大似然估计
\(p(w_{i} | w_{i-1}) = \frac{c(w_{i-1}w_{i})}{\sum \limits_{w_{i}} c(w_{i-1}w_{i})}\)
句子的概率计算公式
\[ \begin{align*} s & = w_{1}w_{2}...w_{l}, (l 是句子中词的个数) \\ p(s) & = p(w_{1}|<BOS>)p(w_{2}|w_{1})p(w_{3} | w_{2}w_{1})...p(w_{l}|w_{1}...w_{l-1}) \\ & = \prod\limits_{i=1}^{l}p(w_{i}|w_{1}...w_{i-1}) \end{align*} \]
困惑度
\(PP_{T}(T) = 2^{H_{P}(T)}\)
交叉熵
\(H_{p}(T) = -\frac{1}{W_{T}}log_{2} p(T)\)