NLP笔记（一）

来源：Cousera哥大自然语言处理课程

一阶马尔科夫过程

$$\begin{equation}\begin{split} &P(X_1=x_1,X_2=x_2,...,X_n=x_n)\\ &=P(X_1=x_1)\sum_{i=2}^{n}P(X_{i}|X_{1},X_{2},...,X_{i-1})\\ &=P(X_1=x_1)\sum_{i=2}^{n}P(X_{i}|X_{i-1}) \end{split}\end{equation}$$

二阶马尔科夫过程

$$\begin{equation}\begin{split} &P(X_1=x_1,X_2=x_2,...,X_n=x_n)\\ &=P(X_1=x_1)P(X_2=x_2|X_1=x_1)\sum_{i=3}^{n}P(X_{i}|X_{i-2}=x_{i-2},X_{i-1}=x_{i-1})\\ &=\sum_{i=1}^{n}P(X_{i}|X_{i-2}=x_{i-2},X_{i-1}=x_{i-1}) \end{split}\end{equation}$$
设$X_{-1}=X_{0}=*$，表示开始标志

---

三元语言模型

The Trigram Estimation Problem

混淆度(Perplexity)

Perplexity is a measure of effective “branching factor”

偏差&方差权衡

Why

• Unigram & Bigram converge quickly to its true underlying value.
• Trigram has low bias, but need large datasets to get an accurate estimate to avoid “ZERO”.

Linear Interpolation

Estimate The λ Values