Docker实操6——配置好的强化学习Dockerfile

最新推荐文章于 2023-09-03 13:29:59 发布

Nemo555

最新推荐文章于 2023-09-03 13:29:59 发布

阅读量2k

点赞数 1

分类专栏：环境配置文章标签： docker 强化学习

本文链接：https://blog.csdn.net/weixin_40056577/article/details/122268258

版权

环境配置专栏收录该内容

7 篇文章 3 订阅

订阅专栏

自定义专属的强化环境

环境概述
一、文件内容
终：完整的Dockerfile
小坑

环境概述

基本信息	具体版本
系统	Ubuntu 18.04 LTS
用户:密码	pamirl:qwerty
pytorch	1.10.1
python	3.8
mujoco	2.1.0
cuda	10.0
pytorch—cudatoolkit	10.2
强化环境库	dm_control, mujoco,pybullet
强化算法库	stable-baselines3
jupyter lab	password, ip, remote_access等配置

可在dockerfile中修改对应的版本，但修改前须了解它们彼此的依赖关系，可阅读setup.py

执行时的状态：

操作： sh build_image.sh ; sh start_container.sh

可能大家会问一个问题：cuda和cudatoolkit有什么关系？

这里的cuda:10.0是系统层面的关于CUDA的完整工具包，包括Nvidia驱动、编译程序、工具包、IDE、调试器等一系列程序
cudatoolkit是为了使用pytorch框架而“事先被编译好的”动态链接库
因此如果只为了使用pytorch，只需关心cudatoolkit就好了（如果要写pytorch以外的cuda程序，就需要关心cuda完整工具包的版本）

一、文件内容

1.1 sh文件

# build_image.sh
## 使用当前路径.下的Dockerfile，创建名为stable_rl:launch的镜像
docker build -t stable_rl:launch .

# start_container.sh
## mount_dir:挂载主机上的mount_dir路径到容器内的/home/pamirl/rl_dev
## 利用image_name创建容器
jupyter_port=8888
ssh_port=8889
container_nick_name="stable_rl"
image_name="stable_rl:launch"
mount_dir="/home/pami/rl_dev"

nvidia-docker run -itd -p ${jupyter_port}:8888 -p ${ssh_port}:22 -u pamirl -v ${mount_dir}:/home/pamirl/rl_dev --name ${container_nick_name} ${image_name} bash

自行修改端口、名字、挂载路径、容器内的用户名等

1.2 换源的txt文件

# change_apt.txt
deb https://mirrors.ustc.edu.cn/ubuntu/ bionic main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-security main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-security main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse

deb http://mirrors.aliyun.com/ubuntu/ xenial main
deb-src http://mirrors.aliyun.com/ubuntu/ xenial main
deb http://mirrors.aliyun.com/ubuntu/ xenial-updates main
deb-src http://mirrors.aliyun.com/ubuntu/ xenial-updates main
deb http://mirrors.aliyun.com/ubuntu/ xenial universe
deb-src http://mirrors.aliyun.com/ubuntu/ xenial universe
deb http://mirrors.aliyun.com/ubuntu/ xenial-updates universe
deb-src http://mirrors.aliyun.com/ubuntu/ xenial-updates universe
deb http://mirrors.aliyun.com/ubuntu/ xenial-security main
deb-src http://mirrors.aliyun.com/ubuntu/ xenial-security main
deb http://mirrors.aliyun.com/ubuntu/ xenial-security universe
deb-src http://mirrors.aliyun.com/ubuntu/ xenial-security universe

#change_conda.txt
channels:
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/
  - defaults
show_channel_urls: true
custom_channels:
  conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  msys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  bioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  menpo: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  pytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  pytorch-lts: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  simpleitk: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud

pip的源直接在dockerfile中指定了pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
从github下载的均使用了它的镜像网站github.cnpmjs.org
所以如果不熟悉docker挂https_proxy，或者懒的话，国内就根据自己需要换掉apt , pip , conda, github的源

1.3 测试环境的py文件

# test_cuda.py
import torch
print(torch.__version__)
print(torch.cuda.is_available())  # cuda是否可用
print(torch.cuda.device_count())  # 返回GPU的数量
print(torch.cuda.get_device_name(0))  # 返回gpu名字，设备索引默认从0开始

# test_mujoco.py
import mujoco_py
import os
mj_path = mujoco_py.utils.discover_mujoco()
xml_path = os.path.join(mj_path, 'model', 'humanoid.xml')
model = mujoco_py.load_model_from_path(xml_path)
sim = mujoco_py.MjSim(model)

print(sim.data.qpos)
sim.step()
print(sim.data.qpos)

# test_pybullet.py
import os
import gym
import pybullet_envs

from stable_baselines3.common.vec_env import DummyVecEnv, VecNormalize
from stable_baselines3 import PPO

env = DummyVecEnv([lambda: gym.make("HalfCheetahBulletEnv-v0")])
# Automatically normalize the input features and reward
env = VecNormalize(env, norm_obs=True, norm_reward=True,
                   clip_obs=10.)

model = PPO('MlpPolicy', env)
model.learn(total_timesteps=2000)
print("finish learning")
# Don't forget to save the VecNormalize statistics when saving the agent
log_dir = "/tmp/"
model.save(log_dir + "ppo_halfcheetah")
print("saving model at {0}".format(log_dir))
stats_path = os.path.join(log_dir, "vec_normalize.pkl")
env.save(stats_path)
print("saving env at {}".format(stats_path))
# To demonstrate loading
del model, env

# Load the saved statistics
env = DummyVecEnv([lambda: gym.make("HalfCheetahBulletEnv-v0")])
print("loading env and model")
env = VecNormalize.load(stats_path, env)
#  do not update them at test time
env.training = False
# reward normalization is not needed at test time
env.norm_reward = False

# Load the agent
model = PPO.load(log_dir + "ppo_halfcheetah", env=env)

# test_stable_baselines.py
import gym
from stable_baselines3 import PPO

env = gym.make("Ant-v3")
model = PPO("MlpPolicy", env, verbose=1)
model.learn(total_timesteps=1000)

obs = env.reset()
for i in range(1000):
    action, _states = model.predict(obs, deterministic=True)
    obs, reward, done, info = env.step(action)
    #env.render()
    if done:
      obs = env.reset()

env.close()

这些基本是从各官网copy下来的例子，进行测试。

终：完整的Dockerfile

FROM nvidia/cuda:10.0-base

COPY ./change_apt.txt /
RUN cp /etc/apt/sources.list /etc/apt/sources_init.list \
	&& cat /change_apt.txt > /etc/apt/sources.list \
	&& rm /change_apt.txt \
	&& apt update

RUN apt update && DEBIAN_FRONTEND=noninteractive apt install -y --allow-unauthenticated --no-install-recommends \
    build-essential apt-utils cmake git curl vim ca-certificates sudo tmux jq unrar\
    libjpeg-dev libpng-dev \
    libgtk3.0 libsm6 cmake ffmpeg pkg-config \
    qtbase5-dev libqt5opengl5-dev libassimp-dev \
    libboost-python-dev libtinyxml-dev bash \
    wget unzip libosmesa6-dev software-properties-common \
    libopenmpi-dev libglew-dev openssh-server \
    libosmesa6-dev libgl1-mesa-glx libgl1-mesa-dev patchelf libglfw3 \
    && rm -rf /var/lib/apt/lists/*

RUN useradd --create-home --shell /bin/bash pamirl \
	&& adduser pamirl sudo \
	&& echo 'pamirl:qwerty' | chpasswd \
	&& su pamirl

USER pamirl
WORKDIR /home/pamirl
# miniconda
RUN wget -q https://mirrors.tuna.tsinghua.edu.cn/anaconda/miniconda/Miniconda3-latest-Linux-x86_64.sh && \
    bash Miniconda3-latest-Linux-x86_64.sh -b -p miniconda3 && \
    rm Miniconda3-latest-Linux-x86_64.sh
ENV PATH /home/pamirl/miniconda3/bin:$PATH
# mujoco
RUN wget https://github.com.cnpmjs.org/deepmind/mujoco/releases/download/2.1.0/mujoco210-linux-x86_64.tar.gz \
		&& mkdir -p .mujoco \
		&& whoami \
		&& pwd \
		&& tar -zxf mujoco210-linux-x86_64.tar.gz -C "$HOME/.mujoco" \
		&& rm mujoco210-linux-x86_64.tar.gz
ENV LD_LIBRARY_PATH /home/pamirl/.mujoco/mujoco210/bin:${LD_LIBRARY_PATH}

# conda env
COPY ./change_conda.txt ./
RUN conda config --describe \
	&& cat ./change_conda.txt > .condarc \
	&& rm change_conda.txt \
	&& conda create -y -n deep_rl python=3.8

SHELL ["/bin/bash", "--login", "-c"]

# basis pytorch
RUN pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple \
	&& source activate deep_rl \
	&& cat .condarc \
	&& pip install -q dm_control gym \
	&& pip install -U 'mujoco-py<2.2,>=2.1' \
	&& conda install -y pytorch=1.10.1 torchvision torchaudio cudatoolkit=10.2 -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch \
	&& pip install glfw Cython imageio lockfile

# stable-baselines3
RUN pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple \
	&& source activate deep_rl \
	&& pip install -U stable-baselines3[extra] pyglet pybullet

# atari
#RUN source activate deep_rl \
#	&& wget -q http://www.atarimania.com/roms/Roms.rar \
#	&& mkdir ~/atari_roms
#	&& unrar e Roms.rar ~/atari_roms \
#	&& python -m atari_py.import_roms ~/atari_roms \
#	&& rm Roms.rar \
#	&& rm -rf ~/atari_roms

# install jupyerlab
RUN source activate deep_rl \
	&& conda install jupyterlab ipykernel \
	&& python -m ipykernel install --user --name deep_rl --display-name deep_rl:stable-baselines3

# configure jupyter lab & vscode
RUN source activate deep_rl \
	&& jupyter lab --generate-config \
	&& cd .jupyter \
	#&& echo c.ServerApp.password=$(cat jupyter_server_config.json | jq .ServerApp.password) >> jupyter_lab_config.py \
	&& echo "c.ServerApp.password = \"$(python -c "from jupyter_server.auth import passwd;print(passwd('qwerty'))")\"" >> jupyter_lab_config.py \
	&& echo "c.ServerApp.ip = '*'" >> jupyter_lab_config.py \
	&& echo "c.ServerApp.allow_remote_access=True" >> jupyter_lab_config.py

# Add test scripts
ADD test*.py ./
# Add conda dependency of your code
RUN source activate deep_rl \
	&& conda install -y scikit-learn tensorboardX \
	&& conda init

系统用户、密码、jupyterlab的配置、端口都可根据自行需要修改

小坑

GPU是RTX 3070或3090，就需要指定cudatoolkit=11.3，否则会报错误：GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation
如果使用cuda:11.0-base基础镜像，会出现apt无法安装的问题，因为11.0的操作系统是Ubuntu 20.04 LTS，需要对apt的源进行额外更换
权限的问题，如果希望容器内用户的程序和文件权限与host主机身份权限一致，需要在Dockerfile中指定pamirl用户的$UID，对应部分修改如下：

# 容器内程序的权限与host当前用户($UID)一致
ARG UID
# 在build image的时候传入当前主机用户的UID
RUN useradd -u $UID --create-home --shell /bin/bash pamirl \
		# 如无必要，可删除用户的sudo权限
        && adduser pamirl sudo \
        && echo 'pamirl:qwerty' | chpasswd \
        && su pamirl
#####################
# 容器内文件权限与host当前用户权限一致
ADD --chown=$UID test*.py ./

build_image.sh制作镜像命令修改如下：( 比如build时希望每个密码不一样，都可以通过定义ARG，在制作镜像时传入，其余同理）
docker build --build-arg UID=$UID -t stable_rl:origin .