新来了A100*8的服务器,需要部署一个轩辕2.0大模型,途中踩了很多坑,现回忆并记录一下。
一、配置环境
首先是配置环境,算力是8.9,需要配置cuda11.8及以上版本。
cuda和cudnn的安装网上教程比较多,就不写了。
安装好cuda和cudnn以后,torch.cuda.is_available()一直是False,此时不明确原因,强烈建议下载cuda-samples,方便查找原因。
进入文件夹cuda-samples-master/Samples/1_Utilities/bandwidthTest
make
./bandwidthTest
此时在make或者运行时都会有报错原因,去查找相应的解决方案就行了,我这里是缺两个基础包
>>> torch.cuda.is_available()
/home/anaconda3/envs/py10/lib/python3.10/site-packages/torch/cuda/__init__.py:107: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 802: system not yet initialized (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.)
./bandwidthTest
[CUDA Bandwidth Test] - Starting...
Running on...
cudaGetDeviceProperties returned 802
-> system not yet initialized
CUDA error at bandwidthTest.cu:256 code=802(cudaErrorSystemNotReady) "cudaSetDevice(currentDevice)"
如果是CUDA error at bandwidthTest.cu:256 code=802(cudaErrorSy