基础要求
- 操作系统: Ubuntu 18.04.3 LTS
配置apt源
- 配置清华源
$ sudo vi /etc/apt/sources.list
# 默认注释了源码镜像以提高 apt update 速度,如有需要可自行取消注释
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-security main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-security main restricted universe multiverse
# 预发布软件源,不建议启用
# deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
- 配置smplayer源
$ cat /etc/apt/sources.list.d/rvm-ubuntu-smplayer-bionic.list
deb http://ppa.launchpad.net/rvm/smplayer/ubuntu bionic main
# deb-src http://ppa.launchpad.net/rvm/smplayer/ubuntu bionic main
- 配置本地源
$ cat /etc/apt/sources.list.d/nccl-2.5.6-ga-cuda10.1.list
deb file:///var/nccl-repo-2.5.6-ga-cuda10.1 /
- 配置ppa源
$ sudo vi /etc/apt/sources.list.d/graphics-drivers-ubuntu-ppa-bionic.list
deb http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic main
# deb-src http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic main
# deb-src http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic main
- 配置nvidia-docker源
$ sudo vi /etc/apt/sources.list.d/nvidia-docker.list
deb https://nvidia.github.io/libnvidia-container/stable/ubuntu18.04/$(ARCH) /
#deb https://nvidia.github.io/libnvidia-container/experimental/ubuntu18.04/$(ARCH) /
deb https://nvidia.github.io/nvidia-container-runtime/stable/ubuntu18.04/$(ARCH) /
#deb https://nvidia.github.io/nvidia-container-runtime/experimental/ubuntu18.04/$(ARCH) /
deb https://nvidia.github.io/nvidia-docker/ubuntu18.04/$(ARCH) /
- 配置上海交大和阿里docker源
$ cat /etc/apt/sources.list
deb http://ftp.sjtu.edu.cn/ubuntu/ bionic main restricted
deb http://ftp.sjtu.edu.cn/ubuntu/ bionic-updates main restricted
deb http://ftp.sjtu.edu.cn/ubuntu/ bionic universe
deb http://ftp.sjtu.edu.cn/ubuntu/ bionic-updates universe
deb http://ftp.sjtu.edu.cn/ubuntu/ bionic multiverse
deb http://ftp.sjtu.edu.cn/ubuntu/ bionic-updates multiverse
deb http://ftp.sjtu.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
deb http://ftp.sjtu.edu.cn/ubuntu bionic-security main restricted
deb http://ftp.sjtu.edu.cn/ubuntu bionic-security universe
deb http://ftp.sjtu.edu.cn/ubuntu bionic-security multiverse
deb [arch=amd64] http://mirrors.aliyun.com/docker-ce/linux/ubuntu bionic stable
# deb-src [arch=amd64] http://mirrors.aliyun.com/docker-ce/linux/ubuntu bionic stable
deb http://security.ubuntu.com/ubuntu trusty-security main
- 制作离线的本地源
- 拷贝能联网的机器的deb安装包到不能联网的服务器,然后制作本地源进行安装.
- ubuntu14以上的版本,需要制作release签名。
- 缺点:操作繁琐易出错
- 优点: 定制化程度高,可以安装自己需要的包,不需要安装多余的包。
7.1 联网机器下载包和制作离线deb源
- 安装gpg软件
$ sudo apt-get install -y gnupg rng-tools
密钥创建过程中,需要使用到足够的随机数(random),可先行安装rng-tools, 该工具可以常驻后台的方式, 生成随机数,避免gpg密钥创建过程中的长时间等待问题
$ sudo rngd -r /dev/urandom
$ sudo gpg --gen-key
public and secret key created and signed
查看key
$ sudo gpg --list-key
导出公钥和私钥
私钥,供Server端,对release文件签名使用
gpg -a --export-secret-keys username > Ubuntu_Local_Archive_Automatic_Signing_Key_2017.sec
公钥,需在Ubuntu client 导入,供apt-get使用
gpg -a --export username> username.pub
准备安装包源
$ sudo rm -rf /var/cache/apt/archives/* # 清空缓存目录
$ sudo apt-get -d install <包名> # -d 仅下载安装包,不安装
$ mkdir /var/debpackages # 新建deb包存放目录
$ cp -r /var/cache/apt/archives/*.deb /var/debpackages/
在debs这个目录创建Packages.gz,注意生成的路径带debpackages
$ sudo apt-ftparchive packages debpackages > debpackages/Packages
$ cd debpackages/ && gzip -c Packages > Packages.gz
在debpackages这个目录下创建release file
$ apt-ftparchive release ./ > Release
ubuntu apt-get 对软件包索引,首先要求InRelease文件,其次才去找Release、Release.gpg文件; 这情况下, 其实只需要创建InRelease文件(包含Release文件和明文签名)即可:
$ gpg --clearsign -o InRelease Release
$ gpg -abs -o Release.gpg Release
将生成的目录和username.pub拷贝到另外一台ubuntu18.04的服务器的/data目录下
7.2 离线服务器添加离线源和安装包
导入公钥
$ sudo apt-key add username.pub
备份源文件:
$ sudo mv /etc/apt/source.list /etc/apt/source.list_bak
添加本地源
$ sudo vi /etc/apt/source.list
deb file:///data/debpackages /
更新索引和安装包
$ sudo apt-get update
$ sudo apt-get -y install <包名>
配置Anaconda的Python环境
- 下载
wget -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/Anaconda3-5.2.0-Linux-x86_64.sh -O ./Anaconda3-2020.02-Linux-x86_64.sh
- 静默安装
$ bash Anaconda3-2020.02-Linux-x86_64.sh -b -p /data/anaconda3
设置环境变量
$ echo -e "export PATH=$PATH:/data/anaconda3/bin" >> ${HOME}/.bashrc
$ source ${HOME}/.bashrc
$ mkdir ${HOME}/.pip
$ echo -e "[global]\nindex-url = https://mirrors.aliyun.com/pypi/simple/" >> ${HOME}/.pip/pip.conf
$ echo -e "[install]\ntrusted-host=mirrors.aliyun.com" >> ${HOME}/.pip/pip.conf
- 配置utsc的conda源
命令执行:
conda config --add channels https://mirrors.ustc.edu.cn/anaconda/pkgs/free/
conda config --add channels https://mirrors.ustc.edu.cn/anaconda/pkgs/main/
conda config --set show_channel_urls yes
# 附加库
conda config --add channels https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge/
conda config --add channels https://mirrors.ustc.edu.cn/anaconda/cloud/msys2/
conda config --add channels https://mirrors.ustc.edu.cn/anaconda/cloud/bioconda/
conda config --add channels https://mirrors.ustc.edu.cn/anaconda/cloud/menpo/
配置文件内写入:
$ sudo vi ~/.condarc
channels:
- https://mirrors.ustc.edu.cn/anaconda/cloud/menpo/
- https://mirrors.ustc.edu.cn/anaconda/cloud/bioconda/
- https://mirrors.ustc.edu.cn/anaconda/cloud/msys2/
- https://mirrors.ustc.edu.cn/anaconda/cloud/conda-forge/
- https://mirrors.ustc.edu.cn/anaconda/pkgs/main/
- https://mirrors.ustc.edu.cn/anaconda/pkgs/free/
- defaults
show_channel_urls: true
- 配置清华的conda源
$ sudo vi ~/.condarc
channels:
- defaults
show_channel_urls: true
channel_alias: https://mirrors.tuna.tsinghua.edu.cn/anaconda
default_channels:
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/pro
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
- https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/
custom_channels:
conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
msys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
bioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
menpo: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
pytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
simpleitk: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
auto_activate_base: false
- conda基础操作
Conda 是著名的包管理器和虚拟环境管理器。
* 只更新conda
$ conda update -n base conda
* 设置base虚拟环境为false
$ conda config --set auto_activate_base false
Conda环境迁移
第一种迁移虚拟环境方法
- 联网情况下
- 导出当前环境
conda env export > daozhang.yaml
- 在新机器上导入环境
conda env create -f daozhang.yaml
- conda删除daozhang的虚拟环境
conda remove -n daozhang --all
- 离线情况下
Conda-pack
是一个命令行工具,用于打包 conda 环境,其中包括该环境中安装的软件包的所有二进制文件。 当您想在有限或没有网络访问的系统中重现环境时,此功能很有用。
注意,conda-pack 指定平台和操作系统,目标计算机必须具有与源计算机相同的平台和操作系统。
要安装 conda-pack,请确保您位于 root 或 base 环境中,Conda-pack 可通过 conda-forge 或者 PyPI 安装。
conda-forge:
conda install -c conda-forge conda-pack
PyPI:
pip install conda-pack
- 打包一个环境
# Pack environment my_env into my_env.tar.gz
conda pack -n my_env
# Pack environment my_env into out_name.tar.gz
conda pack -n my_env -o out_name.tar.gz
# Pack environment located at an explicit path into my_env.tar.gz
conda pack -p /explicit/path/to/my_env
示例
$ conda activate base
$ pip install conda-pack
$ conda pack -n daozhang -o pose_daozhang.tar.gz
- 重现一个环境
# Unpack environment into directory `my_env`
mkdir -p my_env
tar -xzf my_env.tar.gz -C my_env
示例
$ mkdir /data/anaconda3/envs/{daozhang,huadie}
$ tar -zxf pose_daozhang.tar.gz -C /data/anaconda3/envs/daozhang/
$ tar -zxvf face_huadie.tar.gz -C /data/anaconda3/envs/huadie/
激活和退出虚拟环境
$ conda activate daozhang # 激活
$ conda deactivate daozhang # 退出
第二种复制虚拟环境方法
如果需要在具有 相同操作系统 的计算机之间复制环境,则可以生成spec list
。
生成spec list
文件:
conda list --explicit > spec-list.txt
重现环境:
conda create --name python-course --file spec-list.txt
博客首发于
- 知乎: https://www.zhihu.com/people/liyuanjieblog