解决Unsloth遇到NotImplementedError: No operator found for memory_efficient_attention_forward with inputs

1. 问题

在V100显卡调用unsloth微调时报错：NotImplementedError: No operator found for memory_efficient_attention_forward with inputs。

• 我的环境：
  pytorch=2.3, cuda=11.8。
  torch 2.3.1+cu118 pypi_0 pypi
  torchaudio 2.3.1+cu118 pypi_0 pypi
  torchvision 0.18.1+cu118 pypi_0 pypi

• 我的报错

NotImplementedError: No operator found for memory_efficient_attention_forward with inputs:
query : shape=(2, 267, 8, 4, 128) (torch.float16)
key : shape=(2, 267, 8, 4, 128) (torch.float16)
value : shape=(2, 267, 8, 4, 128) (torch.float16)
attn_bias : <class ‘xformers.ops.fmha.attn_bias.LowerTriangularMask’>
p : 0.0
flshattF@0.0.0 is not supported because:
- xFormers wasn’t build with CUDA support
- requires device with capability > (8, 0) but your GPU has capability (7, 0) (too old)
- operator wasn’t built - see python -m xformers.info for more info
cutlassF is not supported because:
- xFormers wasn’t build with CUDA support
- operator wasn’t built - see python -m xformers.info for more info
smallkF is not supported because:
- max(query.shape[-1] != value.shape[-1]) > 32
- xFormers wasn’t build with CUDA support
- dtype=torch.float16 (supported: {torch.float32})
- attn_bias type is <class ‘xformers.ops.fmha.attn_bias.LowerTriangularMask’>
- operator wasn’t built - see python -m xformers.info for more info
- operator does not support BMGHK format
- unsupported embed per head: 128

2. 解决方案

去PyTorch官网重新下载安装pytorch=2.2。并且看起来pytorch=2.1和pytorch=2.3均不太行。
参考自：Fine-tuning on a V100 GPU #496

pip install torch==2.2.2 torchvision==0.17.2 torchaudio==2.2.2 --index-url https://download.pytorch.org/whl/cu118