背景:
想做一个基于Word2Vec的分析标题与标题之间相关性的模型,训练完之后保存模型。再加载发生了如下错误:
在这里插入代码片
Traceback (most recent call last):
File "D:\AIcourse_design\code\main.py", line 37, in <module>
load_model()
File "D:\AIcourse_design\code\main.py", line 25, in load_model
model = gensim.models.Word2Vec.load('m')
File "D:\anaconda3\lib\site-packages\gensim\models\word2vec.py", line 1930, in load
model = super(Word2Vec, cls).load(*args, **kwargs)
File "D:\anaconda3\lib\site-packages\gensim\utils.py", line 485, in load
obj = unpickle(fname)
File "D:\anaconda3\lib\site-packages\gensim\utils.py", line 1460, in unpickle
return _pickle.load(f, encoding='latin1') # needed because loading from S3 doesn't support readline()
_pickle.UnpicklingError: unpickling stack underflow
进程已结束,退出代码1
一开始,保存模型的语句是:
from gensim.models import Word2Vec
model = Word2Vec(res, sg=1, vector_size=100, window=5, min_count=1, negative=3, sample=0.001, hs=1,
workers=4)
model.wv.save_word2vec_format('model.txt',binary=False)
加载模型的语句是:
import gensim
model = gensim.models.Word2Vec.load('model.txt')
尝试了很多网上的教程,均无功而返,直到看到了这篇文章。结合报错语句的最后一行,猜测应该是文件格式的问题。遂采用.pkl来保存
https://cloud.tencent.com/developer/article/1734787#:~:text=%E4%BF%9D%E5%AD%98%E6%A8%A1%E5%9E%8B%EF%BC%8C%E9%80%9A%E5%B8%B8%E9%87%87%E7%94%A8pkl%E5%BD%A2%E5%BC%8F%E4%BF%9D%E5%AD%98%EF%BC%8C%E4%BB%A5%E4%BE%BF%E4%B8%8B%E6%AC%A1%E7%9B%B4%E6%8E%A5%E5%8A%A0%E8%BD%BD%E5%8D%B3%E5%8F%AF%20%23%20%E5%8A%A0%E8%BD%BD%E6%A8%A1%E5%9E%8B,model%20%3D%20Word2Vec.load%28model_path%29
保存模型语句改为:
model = Word2Vec(res, sg=1, vector_size=100, window=5, min_count=1, negative=3, sample=0.001, hs=1, workers=4)
model.save('model.pkl')
加载模型语句改为:
model = Word2Vec.load('model.pkl')
最后解决!