部署llama3.1时出现ValueError: `rope_scaling` must be a dictionary with two fields的解决方案

在部署最新模型llama3.1时出现了ValueError: rope_scaling must be a dictionary with two fields, type and factor, got {‘factor’: 8.0, ‘low_freq_factor’: 1.0, ‘high_freq_factor’: 4.0, ‘original_max_position_embeddings’: 8192, ‘rope_type’: ‘llama3’}

完整报错

[2024-07-25 01:37:53,226] [INFO] [real_accelerator.py:133:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Traceback (most recent call last):
  File "/chenhui/zhangwuhan/stage2/BMLU-0.2.0/src/English_chat/Llama.py", line 90, in <module>
    main(model_path=args.model_path,max_length=args.max_length)
  File "/chenhui/zhangwuhan/stage2/BMLU-0.2.0/src/English_chat/Llama.py", line 71, in main
    pipeline = transformers.pipeline(
  File "/usr/local/lib/python3.10/dist-packages/transformers/pipelines/__init__.py", line 816, in pipeline
    config = AutoConfig.from_pretrained(
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py", line 952, in from_pretrained
    return config_class.from_dict(config_dict, **unused_kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 761, in from_dict
    config = cls(**config_dict)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/configuration_llama.py", line 161, in __init__
    self._rope_scaling_validation()
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/configuration_llama.py", line 181, in _rope_scaling_validation
    raise ValueError(
ValueError: `rope_scaling` must be a dictionary with two fields, `type` and `factor`, got {'factor': 8.0, 'low_freq_factor': 1.0, 'high_freq_factor': 4.0, 'original_max_position_embeddings': 8192, 'rope_type': 'llama3'}

解决方法

更新使用最新版本的transformers：

pip install --upgrade transformers