Python on HPC(High Performance Computing)

HPC 上设置python
(详见https://www.stat.wisc.edu/services/hpc-cluster1/users-guide-software-paths）

简单操作

(第二次进入就只需要加粗的地方)

1. module avail看哪些软件模块可用

2. module load python/python3.6.4加载模块

3. which python确认路径(/workspace/software/python-3.6.4/bin/python)
   (Python比较特殊，最好在自己的workspace中建立环境后用anaconda或者miniconda：)

4. module load python/anaconda
   （Anaconda includes all python packages. There are many, so using anaconda will be large and have lots of extra packages you won’t need, however, it will have all the common things you’re looking for such as numpy, scipy, scikit-learn, etc. Miniconda is essentially an empty environment and you tell it which packages to include.）

5. which python确认路径(/workspace/software/anaconda3/bin/python)

6. conda create -p /workspace/NETID/myEnvironment anaconda

7. source activate /workspace/NETID/myEnvironment (进入环境)

8. python (直接运行)

9. conda deactivate / source deactivate（退出环境）

python的sh文件内容

(详见http://www.stat.wisc.edu/services/hpc-cluster1/users-guide-tutorials的python部分)

#!/bin/bash                                                                       
#SBATCH --mail-user=NETID@stat.wisc.edu                                       
#SBATCH --mail-type=ALL                                                           
#SBATCH -p long                                                                   
#SBATCH -J compare                                                                
#SBATCH -t 70:00:00                                                               
#SBATCH --cpus-per-task=40                                                        
#SBATCH -n 1                                                                      
#SBATCH --mem-per-cpu=2500M     
                                                  
module load python/anaconda
source activate /workspace/NETID/myEnvironment
python compare.py