ImportError: cannot import name ‘LlamaFlashAttention2‘ from ‘transformers.models.llama.modeling_llam

• 操作系统：ubuntu22.04

错误描述

使用llama-factory训练时，

`python3 my_run_sft.py train ./config.yaml `

有时会出现以下错误：

Traceback (most recent call last):
  File "/media/dingxin/data/study/GPT/llama/LLaMA-Factory/my_run_sft.py", line 71, in <module>
    from transformers.models.llama.modeling_llama import (
ImportError: cannot import name 'LlamaFlashAttention2' from 'transformers.models.llama.modeling_llama' (/media/dingxin/data/anaconda3/envs/llama_factory/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py)

解决办法

安装指定的transformers版本4.47.1

pip install transformers==4.47.1

顺利解决