所里的超算配置环境问题汇总

pip指向不对

使用pip出现/bin/python: bad interpreter: No such file or directory：

复制环境后使用pip出现了bad interpreter: No such file or directory：提示，

按照下面可以解决：
which pip
/home/jin016/.conda/envs/py35/bin/pip

which python
/home/jin016/.conda/envs/py35/bin/python

vim /home/jin016/.conda/envs/py35/bin/pip
把#！后面的改成python的路径就好了

 

Python on HPC(High Performance Computing)

简单操作

(第二次进入就只需要加粗的地方)

1. module avail看哪些软件模块可用

2. module load python/python3.6.4加载模块

3. which python确认路径(/workspace/software/python-3.6.4/bin/python)
   (Python比较特殊，最好在自己的workspace中建立环境后用anaconda或者miniconda：)

4. module load python/anaconda
   （Anaconda includes all python packages. There are many, so using anaconda will be large and have lots of extra packages you won’t need, however, it will have all the common things you’re looking for such as numpy, scipy, scikit-learn, etc. Miniconda is essentially an empty environment and you tell it which packages to include.）

5. which python确认路径(/workspace/software/anaconda3/bin/python)

6. conda create -p /workspace/NETID/myEnvironment anaconda

7. source activate /workspace/NETID/myEnvironment (进入环境)

8. python (直接运行)

9. conda deactivate / source deactivate（退出环境）

python的sh文件内容

#!/bin/bash
#SBATCH --job-name=TUMOR
#SBATCH --time=128:00:00
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=128gb
#SBATCH --gres=gpu:4
#SBATCH --nodes=1

module load openmpi
module load cuda/10.1.168
module load cudnn/v7.6.4-cuda101
module load python3/3.6.1
module load miniconda3
module load graphviz
source activate /scratch1/jin016/conda_env/dl_env
cd /scratch1/jin016/xxxxx/driver/
python xxxxxx.py

 

The question is when I run python XXXScript.py, it uses the interpreter in this location  .local/lib/python3.6

It looks like your miniconda python interpreter is finding modules installed to your home directory and causing some sort of conflict

 You can set the environment variable PYTHONUSERSITE to "1" which will cause python to ignore them:

https://docs.python.org/3/using/cmdline.html#envvar-PYTHONNOUSERSITE

 

How I set Python's USER_SITE; do I need to?

https://www.jb51.net/article/149625.htm  

 have a Python installation on OS X (10.10; maintained simply with pip) with my site packages in

/Library/Python/2.7/site-packages

Apple's packages in

/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python

and the standard Apple Python installation in

/System/Library/Frameworks/Python.framework/Versions/2.7

I did nothing special to set this up, so I assume it is pretty standard. I install my packages in the site packages directory, and the only thing I have done to "customize" is to prune the Apple packages I don't need or that duplicate ones I maintain in my site packages directory.

Everything works fine, and all my paths seem to be just what I'd expect; however I get a confusing result when I

python -m site

While this shows a sys.path that makes sense to me

sys.path = [
    '/Users/Rax',
    '/Users/Rax/Documents/Projects/Coding/Python', # From PYTHONPATH
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
    '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload',
    '/Library/Python/2.7/site-packages',
]

I also get

USER_BASE: '/Users/Rax/Library/Python/2.7' (doesn't exist)
USER_SITE: '/Users/Rax/Library/Python/2.7/lib/python/site-packages' (doesn't exist)
ENABLE_USER_SITE: True