为您的jupyter和pycharm设置ssh隧道

Increasing the productivity of your working-from-home environment

提高在家工作环境的生产率

TL;DR, here’s my situation:

TL; DR，这是我的情况：

1. I am working with python and I need to access a GPU server in my lab.
   我正在使用python，需要在实验室中访问GPU服务器。
2. Since this pandemic started, I’ve been working from home.
   自大流行开始以来，我一直在家工作。
3. The server is located behind a private network that you need to ssh to a proxy server before another ssh to the GPU server. That requires two times authentications.
   该服务器位于专用网络后面，您需要先将该服务器SSH到代理服务器，然后再将其SSH到GPU服务器。 这需要两次身份验证。
4. I want to develop the codes locally with my PyCharm, deploy the codes directly to the server and use a browser to open a Jupyter notebook to debug, visualise and experiment with my codes.
   我想使用PyCharm在本地开发代码，将代码直接部署到服务器，并使用浏览器打开Jupyter笔记本来调试，可视化和试验我的代码。

Here’s the recipe that I setup a ssh tunnelling, to enable me to write, deploy and debug my codes without entering passwords.

这是我设置ssh隧道的方法，使我无需输入密码即可编写，部署和调试代码。

SSH配置 (SSH Config)

Usually I need to run this command to connect to my gpu-server through a proxy-server:

通常我需要运行以下命令以通过proxy-server连接到我的gpu-server proxy-server ：

$ ssh -L 8888:localhost:8888 -J username@proxy-server.mycompany.com username@gpu-server.mycompany.com

The -J option enables the jump proxy argument. The -L option creates a tunnel for port 8888 in my local computer to port 8888 to the gpu-server. This port is used by Jupyter so that I can open my notebook in the gpu-server from my local computer. However, this command will prompt two times password authentications, one for the proxy-server and the other one for the gpu-server.

-J选项启用跳转代理参数。 -L选项为本地计算机中的端口8888创建一个隧道，以将端口8888连接到gpu-server 。 Jupyter使用此端口，以便可以从本地计算机在gpu-server打开笔记本。 但是，此命令将提示两次密码认证，一次用于proxy-server ，另一次用于gpu-server 。

And it is horrible to remember that long command line. Alternatively, you can create a nice configuration setting.

记住这么长的命令行太可怕了。 或者，您可以创建一个不错的配置设置。

If you don’t have SSH config file yet, you can create an empty one:

如果您还没有SSH配置文件，则可以创建一个空文件：

$ touch .ssh/config

Edit the config file and create the following section:

编辑配置文件并创建以下部分：

Host gpu-server
   Hostname gpu-server.mycompany.com
   User username
   ProxyJump username@proxy-server.mycompany.com
   ServerAliveInterval 30
   ServerAliveCountMax 3
   LocalForward 8888 localhost:8888

I included ServerAliveInterval and ServerAliveCountMax to keep the SSH connection alive when there is no activity while I’m coding.

我编写了ServerAliveInterval和ServerAliveCountMax以在没有活动时使SSH连接保持活动状态。

With this configuration, I can just call the following command:

使用此配置，我可以仅调用以下命令：

$ ssh gpu-server

but you still need to enter your password twice.

但是您仍然需要两次输入密码。

SSH密钥 (SSH Keys)

To bypass authentication, you need to store SSH key pair both in the proxy and gpu servers. We will use ssh-keygen tool to create the key pair.

要绕过身份验证，您需要将SSH密钥对存储在代理服务器和gpu服务器中。 我们将使用ssh-keygen工具创建密钥对。

— Warning — : The ssh-keygen will create or overwrite the~/.ssh/id_rsa file. So if you have created a key pair file before, calling this command will destroy your previous key pairs.

— 警告—： ssh-keygen将创建或覆盖~/.ssh/id_rsa文件。 因此，如果您以前创建过密钥对文件，则调用此命令将破坏您以前的密钥对。

On your local terminal, run:

在本地终端上，运行：

$ ssh-keygen t rsa

It will ask paraphrase if you want to create an encrypted password, but this is optional. I did not use paraphrase.

它会询问您是否要创建加密的密码，但这是可选的。 我没有使用释义。

Now what you need to do is to transfer your public key to both proxy-server and gpu-server. We will do it differently, as the proxy server can be reached directly, but not the gpu server.

现在您需要做的是将您的公钥转移到proxy-server和gpu-server 。 我们将以不同的方式进行操作，因为可以直接访问代理服务器，而不能直接访问gpu服务器。

将公钥复制到proxy-server (Copying public key to the proxy-server)

Use ssh-copy-id to the proxy server. From your local terminal:

对代理服务器使用ssh-copy-id 。 从本地终端：

$ ssh-copy-id username@proxy-server.mycompany.com

This will copy and transfer your ~/.ssh/id_rsa.pub to proxy-server and save it as ~/.ssh/authorized_keys.

这会将您的~/.ssh/id_rsa.pub复制并传输到proxy-server ，并将其另存为~/.ssh/authorized_keys 。

Now you should be able to ssh to the proxy server without password. Try:

现在，您应该可以不用密码即可SSH到代理服务器。 尝试：

$ ssh username@proxy-server.mycompany.com

将公钥复制到gpu服务器 (Copying public key to the gpu-server)

We do it again for the gpu-server, but differently. From your local terminal:

我们再次对gpu-server ，但有所不同。 从本地终端：

$ cat ~/.ssh/id_rsa.pub | ssh -J username@proxy-server.mycompany.com username@gpu-server.mycompany.com "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >>  ~/.ssh/authorized_keys"

Now you can try SSH to your gpu-server using config file:

现在，您可以使用配置文件尝试SSH到您的gpu服务器：

$ ssh gpu-server

Voila!! No passwords.

瞧！ 没有密码。

启动Jupyter / Jupyter-Lab服务器 (Starting Jupyter / Jupyter-Lab server)

Here’s what you need to install on your gpu-server:

这是您需要在gpu服务器上安装的内容：

1. Create a python environment, either using pip or anaconda.

   使用pip或anaconda创建一个python环境。

2. Within your environment, install jupyter or jupyterlab, and other python packages you need for your work.

   在您的环境中，安装jupyter或jupyterlab以及您工作所需的其他python软件包。

Assuming you have done the above steps, you can try run jupyter to test whether you can reach it from your local computer.

假设您已完成上述步骤，则可以尝试运行jupyter来测试是否可以从本地计算机访问它。

Login to the gpu-server and activate your python environment. Then call:

登录到gpu-server并激活您的python环境。 然后致电：

[gpu-server] $ jupyter notebook --no-browser --ip=0.0.0.0 --port=8888

Ignore the link that the jupyter outputs. Open your local browser and go to http://127.0.0.1:8888/.

忽略jupyter输出的链接。 打开本地浏览器，然后转到http://127.0.0.1:8888/ 。

Voila!! Your localhost on port 8888 has been forwarded to the gpu-server using your ssh tunnel.

瞧！ 端口8888上的本地主机已使用ssh隧道转发到gpu-server 。

To run jupyter in the background, use the nohup command:

要在后台运行jupyter，请使用nohup命令：

[gpu-server] $ nohup jupyter --no-browser --ip=0.0.0.0 --port=8888 &> /tmp/jupyter.out &

You can see info and other debug messages from jupyter in /tmp/jupyter.out file in the gpu-server.

您可以在gpu-server /tmp/jupyter.out文件中查看来自jupyter的信息和其他调试消息。

IMPORTANT NOTE: The ssh tunnel will only open and active if you ssh to the gpu-server using your config file. As soon as you logout, the tunnel disappears. You may want to use some other tool to run ssh tunnel in the background.

重要说明 ：仅当使用配置文件ssh到gpu-server ，ssh隧道才会打开并处于活动状态。 注销后，隧道将消失。 您可能需要使用其他工具在后台运行ssh隧道。

设置PyCharm以自动部署 (Setting up PyCharm to auto-deploy)

One nice feature of PyCharm is the auto deployment to a remote server whenever you save a code. You can also sync a python interpreter with the remote server, but I prefer to use local python interpreter to reduce traffic during indexing and searching of some functions.

PyCharm的一个不错的功能是每当您保存代码时都会自动部署到远程服务器。 您还可以将python解释器与远程服务器同步，但是我更喜欢使用本地python解释器来减少索引和搜索某些功能期间的流量。

1. From a project, open PyCharm preferences dialog and go to Deployment under Build, Execution, Deployment menu on the left sidebar.

   从一个项目，开放PyCharm Preferences对话框，去Deployment下Build, Execution, Deployment菜单上的左侧边栏。

2. Create a new connection, give any name you want.
   创建一个新的连接，提供所需的任何名称。
3. Use SFTP type
   使用SFTP类型
4. In the SSH configuration, click the triple dots (…). This will pop up SSH Configurations dialog box.
   在SSH配置中，单击三点(…)。 这将弹出“ SSH配置”对话框。

5. Add a new configuration and set host (gpu-server), port (22), username, and set authentication type as “OpenSSH config and authentication agent”.

   添加新配置并设置主机( gpu-server )，端口(22)，用户名，并将身份验证类型设置为“ OpenSSH config and authentication agent”。

6. Test connection
   测试连接
7. Back to the new deployment connection window, click on Mappings tab to determine the deployment path. If your connection has been setup properly, then you can browser remote server path.
   返回新的部署连接窗口，单击“映射”选项卡以确定部署路径。 如果您的连接已正确设置，则可以浏览器远程服务器路径。

You can now switch on Automatic Upload under Tools → Deployment menu. Every time you save, the file will be deployed automatically to the gpu-server.

现在，您可以在工具→部署菜单下打开自动上传。 每次保存时，该文件将自动部署到gpu-server 。

Happy coding!!

编码愉快！！

翻译自: https://medium.com/@avan.sp/setting-up-ssh-tunnelling-for-your-jupyter-and-pycharm-2f607909f074