NLP笔记

word Embedding

• cs224n课程很不错，强烈推荐
• word2vec因为最后一层是softmax，所以向量本身其实意义不大，是向量间的相对距离编码了信息。如，gensim.most_similar实现其实考虑的是夹角。

>>> un = False
>>> v = model.word_vec("king", use_norm=un) + model.word_vec("woman", use_norm=un) - model.word_vec("man", use_norm=un)
>>> un = True
>>> v2 = model.word_vec("king", use_norm=un) + model.word_vec("woman", use_norm=un) - model.word_vec("man", use_norm=un)
>>> model.most_similar(positive=[v], topn=6)
[('king', 0.8449392318725586), ('queen', 0.7300517559051514), ('monarch', 0.6454660892486572), ('princess', 0.6156251430511475), ('crown_prince', 0.5818676948547363), ('prince', 0.5777117609977722)]
>>> model.most_similar(positive=[v2], topn=6)
[('king', 0.7992597222328186), ('queen', 0.7118192911148071), ('monarch', 0.6189674139022827), ('princess', 0.5902431011199951), ('crown_prince', 0.5499460697174072), ('prince', 0.5377321243286133)]
>>> model.most_similar(positive=["king", "woman"], negative=["man"], topn=6)
[('queen', 0.7118192911148071), ('monarch', 0.6189674139022827), ('princess', 0.5902431011199951), ('crown_prince', 0.5499460697174072), ('prince', 0.5377321243286133), ('kings', 0.5236844420433044)]

• 如何从字向量得到句向量呢，有两篇相似的实现A Simple but Tough-to-Beat Baseline for Sentence Embeddings和Unsupervised RandomWalk Sentence Embeddings: A Strong but Simple Baseline。对字向量进行加权平均，然后再投影到去掉最大主成分的子空间中去。第二篇文章说他们的效果要好一些。