最简单的方法:
https://blog.csdn.net/Qanyh/article/details/138812171?spm=1001.2014.3001.5501
按照他这个运行即可。
但是我现在装的的cuda 12.x, causal_conv1d 1.1.0 不能安装。
因此采用以下方法:
selective_scan_interface.py 中的 causal_conv1d 输入输出进行微调即可:
正向:
conv1d_out = causal_conv1d_cuda.causal_conv1d_fwd(x, conv1d_weight, conv1d_bias, None, True)
改为:
conv1d_out = causal_conv1d_cuda.causal_conv1d_fwd(x, conv1d_weight, conv1d_bias,None,None, None, True)
反向:
dx, dconv1d_weight, dconv1d_bias= causal_conv1d_cuda.causal_conv1d_bwd(
x, conv1d_weight, conv1d_bias, dconv1d_out, None, dx, True
)
改为:
dx, dconv1d_weight, dconv1d_bias, *_ = causal_conv1d_cuda.causal_conv1d_bwd(
x, conv1d_weight, conv1d_bias, dconv1d_out, None, None,None, dx, False, True
)
复现效果是否有变化暂时不知道。