redis生产环境持久化_在SageMaker上安装持久性Julia环境

redis生产环境持久化

SageMaker is a great environment for data scientists to explore new languages and methods without having to worry too much about the underlying infrastructure. As long as you don’t need to customize the existing environments, everything looks great and just works. However, as soon as you want to venture off the beaten path you will run into some unexpected challenges.

SageMaker为数据科学家提供了一个探索新语言和新方法的绝佳环境，而无需过多地担心基础架构。 只要您不需要自定义现有的环境，一切看起来就很好并且可以正常工作。 但是，一旦您想走人迹罕至的道路，就会遇到一些意想不到的挑战。

Our data scientists are generally cost conscious and happily stop their SageMaker instances when they are not using them. Unfortunately, SageMaker doesn’t persist any data written outside of the ~/SageMaker directory. This means that changes to the Anaconda environments don’t persist across restarts of the notebook. This is very frustrating. To fix this we need to create a persistent Anaconda environment in the ~/SageMaker folder and tell Julia to place its packages there as well. Let’s get started!

我们的数据科学家通常精打细算，并且在不使用SageMaker实例时会高兴地停止它们。 不幸的是，SageMaker不会保留~/SageMaker目录之外写入的任何数据。 这意味着对Anaconda环境的更改不会在笔记本计算机重新启动后持续存在。 这非常令人沮丧。 为了解决这个问题，我们需要在~/SageMaker文件夹中创建一个持久的Anaconda环境，并告诉Julia也将其软件包放置在此处。 让我们开始吧！

Log on to your SageMaker environment and open a new Terminal session. Let’s create a new, empty Anaconda environment, located in the ~/SageMaker directory.

登录到您的SageMaker环境并打开一个新的终端会话。 让我们在~/SageMaker目录中创建一个新的空Anaconda环境。

conda create --yes --prefix ~/SageMaker/envs/julia

Download and extract the latest version of Julia

下载并提取最新版本的Julia

curl --silent https://julialang-s3.julialang.org/bin/linux/x64/1.5/julia-1.5.0-linux-x86_64.tar.gz | tar xzf -
cp -R julia-1.5.0/* ~/SageMaker/envs/julia/

Before we start installing packages for Julia we need to make sure that Julia is loading its packages from the right directory. This also ensures that packages you install via a notebook or from the Terminal will also be stored in the persistent space of the SageMaker instance.

在开始为Julia安装软件包之前，我们需要确保Julia从正确的目录加载其软件包。 这还可以确保您通过笔记本电脑或从终端安装的软件包也将存储在SageMaker实例的永久空间中。

mkdir -p ~/SageMaker/envs/julia/etc/conda/activate.decho 'export JULIA_DEPOT_PATH=~/SageMaker/envs/julia/depot' >> ~/SageMaker/envs/julia/etc/conda/activate.d/env.shecho -e 'empty!(DEPOT_PATH)\npush!(DEPOT_PATH,raw"/home/ec2-user/SageMaker/envs/julia/depot")' >> ~/SageMaker/envs/julia/etc/julia/startup.jl

We can now activate the environment and start installing our dependencies including IJulia. Launch the Julia REPL, install and activate IJulia.

现在，我们可以激活环境并开始安装我们的依赖项，包括IJulia。 启动Julia REPL，安装并激活IJulia。

juliausing Pkg
Pkg.add("IJulia")
using IJulia

Exit the REPL (Ctrl + D) and open Jupyter or JupyterLabs. The new Julia 1.5.0 Kernel should now be visible and you are all set.

退出REPL(Ctrl + D)并打开Jupyter或JupyterLabs。 现在应该可以看到新的Julia 1.5.0内核，并且一切就绪。

After restarting the SageMaker instance you will notice that the Julia kernel has vanished. To restore the kernel, simply execute

重新启动SageMaker实例后，您会注意到Julia内核已消失。 要还原内核，只需执行

conda run --prefix ~/SageMaker/envs/julia/ julia --eval 'using IJulia; IJulia.installkernel("Julia")'

You can get fancy and register this script as a Notebook Lifecycle Configuration which is executed automatically whenever the SageMaker instance is launched. You can find more details on how to set that up at https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html.

您可以将这个脚本注册为Notebook Lifecycle Configuration，并在启动SageMaker实例时自动执行。 您可以在https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html上找到有关如何进行设置的更多详细信息。

翻译自: https://towardsdatascience.com/installing-a-persistent-julia-environment-on-sagemaker-c67acdde9d4b

redis生产环境持久化