gensim.word2vec 训练词向量

一、训练词向量

# 导入必要的包
import jieba
import numpy as np
import pandas as pd
from gensim.models.word2vec import Word2Vec
import joblib
from sklearn.svm import SVC

# 读取两个类别的语料
pos = pd.read_csv('weather_pos.txt', encoding='UTF-8', header=None)
neg = pd.read_csv('weather_neg.txt', encoding='UTF-8', header=None)

# 进行分词处理
pos['words'] = pos[0].apply(lambda x: jieba.lcut(x))
neg['words'] = neg[0].apply(lambda x: jieba.lcut(x))

# 将正负语料进行合并成训练语料然后并打上标签，正语料打上标签1，负语料打上标签0
x = np.concatenate((pos['words'], neg['words']))
y = np.concatenate((np.ones(len(pos)), np.zeros(len(neg))))

# 训练词向量128维
word2vec = Word2Vec(x, size=128, window=3, min_count=5, sg=1, hs=1, iter=10, workers=25)
word2vec.save('word2vec.model')

二、使用词向量

1. 加载词向量

word2vec_model = Word2Vec.load('word2vec.model')

2. 将所有词的词向量保存

word2vec_model .wv.save_word2vec_format('word2vec.txt', binary=False)

3. 获取某个词的词向量

word2vec_model .wv['温度']

4. 获取某个词的相似词

word2vec_model .most_similar('天气')