Miniconda3+JupyterHub+Nginx实践

添加 jupytherhub 用户

[root@node1 bin]# adduser jupyterhub
[root@node1 bin]# passwd jupyterhub

安装Miniconda3

$ pwd
/home/jupyterhub
$ chmod +x ./Miniconda3-latest-Linux-x86_64.sh
$ ./Miniconda3-latest-Linux-x86_64.sh

安装 jupyterhub

• 可以使用清华的conda源加速下载

$ pwd
/home/jupyterhub/miniconda3/bin
$ ./conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
$ ./conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
$ ./conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
$ ./conda config --set show_channel_urls yes

创建 jupyterhub_env 虚拟环境

$ ./conda create -n jupyterhub_env python=3.8
$ ./conda env list

为虚拟环境安装 jupyterhub

$ ./conda install -n jupyterhub_env jupyterhub

激活虚拟环境

[jupyterhub@node1 bin]$ source ./activate jupyterhub_env
(jupyterhub_env) [jupyterhub@node1 bin]$

安装nodejs

# yum install -y nodejs

安装 configurable-http-proxy

# npm install -g configurable-http-proxy

生成配置文件

$ jupyterhub --generate-config -f /home/jupyterhub/jupyterhub_config.py

Q： ModuleNotFoundError: No module named ‘importlib_resources’ 。执行：
A：$ ./conda install -n jupyterhub_env importlib_resources

修改配置文件 jupyterhub_config.py

c.JupyterHub.bind_url = 'http://0.0.0.0:19030'
c.PAMAuthenticator.allowed_users= {'jupyterhub'}
c.LocalAuthenticator.create_system_users = True
c.Authenticator.admin_users = {'jupyterhub'}
c.PAMAuthenticator.open_sessions = False
c.Authenticator.delete_invalid_users = True
#想访问整个文件系统，但仍然默认用户访问他们的主目录
#c.Spawner.notebook_dir = '/'
#c.Spawner.default_url = '/tree/%U' # %U will be replaced with the username

前台启动 jupyterhub

$ jupyterhub -f /home/jupyterhub/jupyterhub_config.py

Q：ModuleNotFoundError: No module named ‘notebook’。执行：
A：$ ./conda install -n jupyterhub_env notebook

Q：Exception in Future <Task finished name=‘Task-34’ coro=<BaseHandler.spawn_single_user..finish_user_spawn() done, defined at /home/jupyterhub/miniconda3/envs/jupyterhub_env/lib/python3.8/site-packages/jupyterhub/handlers/base.py:900> exception=TimeoutError(“Server at http://127.0.0.1:39009/user/jupyterhub/ didn’t respond in 30 seconds”
A：启动jupyterhub的用户没有 c.Spawner.notebook_dir 目录的权限

常见问题列表：https://jupyterhub.readthedocs.io/en/stable/troubleshooting.html

集成ldap

安装 jupyterhub-ldapauthenticator

$ ./conda install -n jupyterhub_env jupyterhub-ldapauthenticator

修改配置文件

c.JupyterHub.bind_url = 'http://0.0.0.0:19030'
c.PAMAuthenticator.allowed_users= {'jupyterhub'}
c.LocalAuthenticator.create_system_users = True
#指定JupyterHub认证类型
c.JupyterHub.authenticator_class = 'ldapauthenticator.LDAPAuthenticator'
#OpenLDAP服务地址
c.LDAPAuthenticator.server_address = 'ldap://node3'
#访问用户和组的DN
c.LDAPAuthenticator.bind_dn_template = [
    "uid={username},ou=People,dc=xxxx,dc=com"
]

配置 nginx 反向代理

    location / {
        #proxy_set_header Origin "";
        proxy_pass http://10.168.1.3:19030;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $http_host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        # websocket headers
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_set_header X-Scheme $scheme;

        proxy_buffering off;
    }

Q：Blocking Cross Origin
A：解决跨域问题nginx配置：proxy_set_header Origin “”;

Q：下载文件失败。Blocking Cross Origin request for /user/jupyterhub/files/xx.pptx. Referer: http://node1:8088, Host: node1
A：将nginx 代理proxy_set_header Host $host; 修改为 proxy_set_header Host $http_host; 即可。官网原文 If nginx is not running on port 443, substitute $http_host for $host on the lines setting the Host header.

虚拟环境迁移

一般用于离线环境，可以将虚拟环境直接压缩传输到迁移的目标服务器，保证Miniconda3安装路径一致的情况下可以直接解压虚拟环境使用。

注意迁移的目标服务器需要离线安装node，configurable-http-proxy

其他命令

对虚拟环境中安装额外的包：

conda install -n your_env_name [package]

删除虚拟环境：

conda remove -n your_env_name --all

删除环境钟的某个包：

conda remove --name $your_env_name  $package_name

退出（base）环境：

conda deactivate

pip下载python包以及相关依赖：

pip3 download seaborn -d /data/seaborn_pkg/
pip3 download openpyxl -d /data/openpyxl_pkg/

pip离线安装依赖：

pip3 install --no-index --find-links=/data/seaborn_pkg/ seaborn
pip3 install --no-index --find-links=/data/openpyxl_pkg/ openpyxl