gensim训练词向量word2vec

1. gensim的word2vec简单使用

Code Example:

from gensim.models import word2vec

sents = [
'I am a good student'.split(),
'Good good study day day up'.split()
]
model = word2vec.Word2Vec(sents, size=100, window=5, min_count=2, workers=10)
# 打印单词'good'的词向量
print(model.wv.word_vec('good'))
# 打印和'good'相似的前2个单词
print(model.wv.most_similar('good', topn=2))
# 保存模型到文件
model.save('w2v.model')

word2vec.Word2Vec参数解释：

• size：词向量输出维度
• window：上下文窗口
• min_count：忽略词频小于此阈值的单词
• workers：使用的线程数

2. 增量训练词向量

Code Example:

def retrain(data_file, old_model_file, new_model_file):
    sents = XXX
    model = word2vec.Word2Vec.load(old_model_file)
    model.build_vocab(sents, update=True)
    model.train(sents, total_examples=model.corpus_count, epochs=model.iter)
    model.save(new_model_file)