模型训练突然报以下两个错误,记录一下解决办法:
错误1:module ‘triton.compiler’ has no attribute ‘OutOfResources’
解决方案来自于git1:
pip3 install -U --index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/Triton-Nightly/pypi/simple/ triton-nightly
错误2:Warning:huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks…
解决方案来自于2:
在脚本最上方加入以下语句—
import os
os.environ["TOKENIZERS_PARALLELISM"] = "false"