xingyi ai集群transformer engine安装

AI集群服务

星光k8s目前只有AI集群服务能使用RDMA，因此通过AI集群服务中的pytorch镜像开启环境。

安装步骤

export http_proxy="http://99.72.0.200:3138"
export https_proxy="http://99.72.0.200:3138"
export CUDNN_PATH=/app/nvhpc/cudnn/8.9
wget https://github.com/NVIDIA/TransformerEngine/archive/refs/tags/v1.8.zip
unzip v1.8.zip
cd TransformerEngine-1.8/3rdparty/cudnn-frontend
wget https://github.com/NVIDIA/cudnn-frontend/archive/98ca4e1941fe3263f128f74f10063a3ea35c7019.zip
unzip cudnn-frontend-98ca4e1941fe3263f128f74f10063a3ea35c7019.zip
mv cudnn-frontend-98ca4e1941fe3263f128f74f10063a3ea35c7019/* ./
rm cudnn-frontend-98ca4e1941fe3263f128f74f10063a3ea35c7019 cudnn-frontend-98ca4e1941fe3263f128f74f10063a3ea35c7019.zip -rf
//https://github.com/NVIDIA/cudnn-frontend/archive/refs/tags/v1.5.2.zip
wget https://github.com/google/googletest/archive/f8d7d77c06936315286eb55f8de22cd23c188571.zip
unzip f8d7d77c06936315286eb55f8de22cd23c188571.zip
mv googletest-f8d7d77c06936315286eb55f8de22cd23c188571/* ./
rm googletest-f8d7d77c06936315286eb55f8de22cd23c188571/ -rf
rm f8d7d77c06936315286eb55f8de22cd23c188571.zip
export CUDACXX=/app/nvhpc/Linux_x86_64/24.3/cuda/12.3/bin/nvcc
cd ..
pip install .

报错

1. AssertionError: Could not find libtransformer_engine.so

export LD_LIBRARY_PATH=/GLOBALFS/nscc-gz_qmo2/software/TransformerEngine-1.8/build/lib.linux-x86_64-cpython-310/transformer_
engine：$LD_LIBRARY_PATH

2. TypeError: Descriptors cannot not be created directly.
   If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.

pip install protobuf==3.19.0