install deepspeed with sparse attention dep

最新推荐文章于 2024-09-13 21:35:42 发布

chzenable

最新推荐文章于 2024-09-13 21:35:42 发布

阅读量706

点赞数

分类专栏：屁话

本文链接：https://blog.csdn.net/zhuanju6759/article/details/109226707

版权

本文详细介绍了如何在您的环境中安装DeepSpeed，这是一个用于加速深度学习训练的库，特别是针对使用稀疏注意力机制的模型。通过遵循步骤，您可以成功配置并开始利用DeepSpeed提升模型训练效率。

摘要由CSDN通过智能技术生成

docker pull ubuntu:18.04
# run the image to create a container based on it
docker run -it ubuntu:18.04 bash
# update the apt software source index
   19  apt update
   20  apt install git
    8  apt install clang-10
    5  apt install llvm-9-dev
    6  apt install cmake

# clone the deepspeed, which is based on the version of torch 1.2 ( with cuda 10.0 )
# we need to install the torch 1.6.0 with minimium cuda version of 10.2
   21  git clone https://github.com/microsoft/DeepSpeed.git
   22  cd DeepSpeed/
   26  apt install python3 pip3
   27  apt install python3 python-pip
   31  apt install wget
   32  wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
   33  bash Miniconda3-latest-Linux-x86_64.sh 
   36  eval "$(/root/miniconda3/bin/conda shell.bash hook)" 
   37  conda init
   38  conda create -n tr cudatoolkit=10.2 pytorch=1.5
   39  conda activate tr
   42  apt-get install llvm-9-dev software-properties-common
   43  pip install triton

# install cuda=10.2
   45  wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
   73  sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600
   46  wget http://developer.download.nvidia.com/compute/cuda/10.2/Prod/local_installers/cuda-repo-ubuntu1804-10-2-local-10.2.89-440.33.01_1.0-1_amd64.deb
   76  sudo dpkg -i cuda-repo-ubuntu1804-10-2-local-10.2.89-440.33.01_1.0-1_amd64.deb
   77  sudo apt-key add /var/cuda-repo-10-2-local-10.2.89-440.33.01/7fa2af80.pub
   78  sudo apt-get update
   79  sudo apt-get -y install cuda

# use the key bind of ctrl + d to exit the container 
# use `docker ps -a` to query the info of docker container and find the exit

最低0.47元/天解锁文章