flash_attn_2_cuda 报错：undefined symbol

...
    from .hf_engine import HuggingfaceEngine
  File "/home/wangguisen/miniconda3/lib/python3.10/site-packages/llmtuner/chat/hf_engine.py", line 12, in <module>
    from ..model import load_model, load_tokenizer
  File "/home/wangguisen/miniconda3/lib/python3.10/site-packages/llmtuner/model/__init__.py", line 1, in <module>
    from .loader import load_config, load_model, load_tokenizer
  File "/home/wangguisen/miniconda3/lib/python3.10/site-packages/llmtuner/model/loader.py", line 9, in <module>
    from .patcher import patch_config, patch_model, patch_tokenizer, patch_valuehead_model
  File "/home/wangguisen/miniconda3/lib/python3.10/site-packages/llmtuner/model/patcher.py", line 16, in <module>
    from .utils.longlora import configure_longlora
  File "/home/wangguisen/miniconda3/lib/python3.10/site-packages/llmtuner/model/utils/longlora.py", line 6, in <module>
    from transformers.models.llama.modeling_llama import (
  File "/home/wangguisen/miniconda3/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 55, in <module>
    from flash_attn import flash_attn_func, flash_attn_varlen_func
  File "/home/wangguisen/miniconda3/lib/python3.10/site-packages/flash_attn/__init__.py", line 3, in <module>
    from flash_attn.flash_attn_interface import (
  File "/home/wangguisen/miniconda3/lib/python3.10/site-packages/flash_attn/flash_attn_interface.py", line 10, in <module>
    import flash_attn_2_cuda as flash_attn_cuda
ImportError: /home/wangguisen/miniconda3/lib/python3.10/site-packages/flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c104cuda9SetDeviceEi

升级下版本即可，装个2.5.8：

pip install flash_attn-2.5.8+cu118torch2.3cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

24.7.26 又遇到这个问题，这次认真记录下

我的torch版本如下：

proxychains4 conda install pytorch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 pytorch-cuda=11.8 -c pytorch -c nvidia

最开始下的版本是：

proxychains4 pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.5.9.post1/flash_attn-2.5.9.post1+cu118torch2.3cxx11abiFALSE-cp311-cp311-linux_x86_64.whl

import 的时候报错：

>>> import flash_attn_2_cuda
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: libc10.so: cannot open shared object file: No such file or directory
>>> 

这个其实是要先导入 torch：

>>> import torch
>>> import flash_attn_2_cuda

ref: https://blog.csdn.net/beneficial/article/details/127675796

在实际跑的时候报错：

    from flash_attn.flash_attn_interface import (
  File "/home/wangguisen/miniconda3/envs/smartRetrieval/lib/python3.11/site-packages/flash_attn/flash_attn_interface.py", line 10, in <module>
    import flash_attn_2_cuda as flash_attn_cuda
ImportError: /home/wangguisen/miniconda3/envs/smartRetrieval/lib/python3.11/site-packages/flash_attn_2_cuda.cpython-311-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC2ENS_14SourceLocationENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

下载新的 C++11 ABI 编译的库即可：

proxychains4 pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.5.9.post1/flash_attn-2.5.9.post1+cu118torch2.3cxx11abiTRUE-cp311-cp311-linux_x86_64.whl

注意：如果你用的是 LLaMA-Factory，那么你应该用的是 vllm-flash-attn

pip install https://github.com/vllm-project/vllm/releases/download/v0.4.3/vllm-0.4.3+cu118-cp311-cp311-manylinux1_x86_64.whl

还有一个问题，也是版本的事

见我提的 issue（已解决）：https://github.com/hiyouga/LLaMA-Factory/issues/5082