- 文本表示:从one-hot到word2vec。
1.1 词袋模型:离散、高维、稀疏。
1.2 分布式表示:连续、低维、稠密。word2vec词向量原理并实践,用来表示文本。
1. Word vectors
把词变成计算机看的懂的语言就是词向量。
看这里:Understanding word vectors
1.1 传统词向量
1.1.1 one-hot
最简单的词向量之一叫one-hot encoded vectors。
譬如,有一组词{magic, dragon, king, queen},用one-hot表示:
For high-cardinality variables — those with many unique categories — the dimensionality of the transformed vector becomes unmanageable.
The mapping is completely uninformed: “similar” categories are not placed closer to each other in embedding space.