1.问题描述:
在使用Transformer或者SentenceTransformer时有时会遇到如下警告,甚是恼火!!!
2.错误:
WARNING:transformers.tokenization_utils_base:Truncation was not explicitely activated but max_length
is provided a specific value, please use truncation=True
to explicitely truncate examples to max length. Defaulting to ‘longest_first’ truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to truncation
.
3.解决方法:
import logging
logging.basicConfig(level=logging.ERROR)
4.特殊情况
当使用的是jupter notebook,添加完代码需要重启服务才可以哦!
如果恰好解决了你的问题的话,麻烦点个赞呗O(∩_∩)O。