1. gensim的word2vec简单使用
Code Example:
from gensim.models import word2vec
sents = [
'I am a good student'.split(),
'Good good study day day up'.split()
]
model = word2vec.Word2Vec(sents, size=100, window=5, min_count=2, workers=10)
# 打印单词'good'的词向量
print(model.wv.word_vec('good'))
# 打印和'good'相似的前2个单词
print(model.wv.most_similar('good', topn=2))
# 保存模型到文件
model.save('w2v.model')
word2vec.Word2Vec参数解释:
size
:词向量输出维度window
:上下文窗口min_count
:忽略词频小于此阈值的单词workers
:使用的线程数
2. 增量训练词向量
Code Example:
def retrain(data_file, old_model_file, new_model_file):
sents = XXX
model = word2vec.Word2Vec.load(old_model_file)
model.build_vocab(sents, update=True)
model.train(sents, total_examples=model.corpus_count, epochs=model.iter)
model.save(new_model_file)