K8S集群 - cuda-卸载再安装[11.7]

cuda卸载
https://blog.csdn.net/u014090429/article/details/109674648

显卡驱动卸载

卸载命令：

/usr/bin/nvidia-uninstall

详情：
https://blog.csdn.net/m0_46825740/article/details/125448274

查看显卡驱动命令

nvidia-smi

查看还有哪些进程仍在使用nvidia

sudo lsof /dev/nvidia*

记住内核

安装命令

./NVIDIA-Linux-x86_64-515.105.01.run --kernel-source-path=/usr/src/kernels/3.10.0-1160.83.1.el7.x86_64 -k $(uname -r)

详细解释：

CentOS 7 安装 NVIDIA 显卡驱动和 CUDA Toolkit_centos先安装显卡驱动还是先安装cudatoolkit-CSDN博客

禁用默认驱动

centos7篇---禁用默认的驱动nouveau，安装cuda和cudnn_centos禁用nouveau驱动-CSDN博客

lsmod | grep nouveau

vi /etc/modprobe.d/blacklist-nouveau.conf

vim /usr/lib/modprobe.d/dist-blacklist.conf

blacklist nouveau
options nouveau modeset=0

WARNING: nvidia-installer was forced to guess the X library path '/usr/lib64' and X module path '/usr/lib64/xorg/modules'; these paths were not queryable from the system. If X fails to find the NVIDIA X
driver module, please install the `pkg-config` utility and the X.Org SDK/development package for your distribution and reinstall the driver.

ERROR: Unable to load the 'nvidia-drm' kernel module.

ERROR: Installation has failed. Please see the file '/var/log/nvidia-installer.log' for details. You may find suggestions on fixing installation problems in the README available on the Linux driver
download page at www.nvidia.com.

安装日志：

vim /var/log/nvidia-installer.log

安装cuda日志

Ubuntu18.04安装cuda+ [ERROR]: Install of driver component failed.的解决-CSDN博客

不要选择驱动

centos7篇---禁用默认的驱动nouveau，安装cuda和cudnn_centos禁用nouveau驱动-CSDN博客

如果已经安装好了显卡驱动，安装cuda的时候一定不要再选择Driver 

配置环境变量

vim ~/.bashrc

export CUDA_HOME=/usr/local/cuda
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
export CUDA_ROOT=/usr/local/cuda

source 

执行GPU报错：

[root@k8s-worker-02 v0.0.1]# docker run --name xxx-gpu-v0.0.1 --gpus all --shm-size 4G -itd xxx-gpu:v0.0.1 /usr/sbin/init xxxxxx docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

离线安装

nvidia-container-runtime-3.9.0-1.noarch.rpm

cd  nvidia-container-runtime

rpm -Uvh --force --nodeps *.rpm

重启docker

systemctl restart docker