CS224n week4
N-grams model
针对OOV两种方式
稀疏问题:
1.分子为0
使用Smoothing(discounting)
- Laplace smoothing(add-1 smoothing):
discount dc
-
Add-k smoothing:
2.分母为0 -
Backoff and Interpolation:
we only “back off” to a lower-order n-gram if we have zero evidence for a higher-order n-gram
held-out 、discount
-
Katz backoff
-
Kneser-Ney Smoothing
PERPLEXITY’S RELATION TO ENTROPY
- Entropy
- Entory rate