尝试运行huggingface上的Kosmos,https://huggingface.co/ydshieh/kosmos-2-patch14-224失败,报错:
nameerror: name 'kosmos2tokenizer' is not defined
查看报错代码:
vi /root/.cache/huggingface/modules/transformers_modules/ydshieh/kosmos-2-patch14-224/48e3edebaeb02dc9fe105f40e85a43a3b440dc72/tokenization_kosmos2_fast.py
看到以下代码:
print('available: ', is_sentencepiece_available())
if is_sentencepiece_available():
from .tokenization_kosmos2 import Kosmos2Tokenizer
else:
Kosmos2TokenizerFast = None
说明由于is_sentencepiece_available()
判断为False,导致没有导入Kosmos2Tokenizer,因此要安装该依赖。
深入可知,该依赖是sentencepiece,所以执行以下命令:
pip install sentencepiece
之后就可以执行demo了。