将dockerized flask应用程序部署到Google Cloud Platform

云部署 (CLOUD DEPLOYMENT)

By: Edward Krueger and Douglas Franklin.

作者： 爱德华·克鲁格 ( Edward Krueger)和道格拉斯·富兰克林 ( Douglas Franklin) 。

In this article, we’ll cover how to deploy an app with a Pipfile.lock to the cloud and connect the app to a cloud database. For more information on virtual environments or getting started with the environment and package manager Pipenv, check out this article!

在本文中，我们将介绍如何将具有Pipfile.lock的应用程序部署到云上以及如何将该应用程序连接到云数据库。 有关虚拟环境或环境和程序包管理器Pipenv入门的更多信息，请查看 本文 ！

部署问题 (Deployment Issues)

Newer developers often install everything at the system level due to a lack of understanding of, or experience with, virtual environments. Python packages installed with pip are placed at the system level. Retrieving requirements this way for every project creates an unmanageable global Python environment on your machine. Virtual environments allow you to compartmentalize your software while keeping an inventory of dependencies.

由于缺乏对虚拟环境的了解或经验，较新的开发人员通常在系统级别安装所有内容。 与pip一起安装的Python软件包位于系统级别。 以这种方式为每个项目检索需求会在您的计算机上创建一个难以管理的全局Python环境。 虚拟环境使您可以划分软件，同时保留依赖性清单。

Pipenv, a tool for virtual environment and package management, allows developers to create isolated software products that are much easier to deploy, build upon, and modify.

Pipenv是用于虚拟环境和程序包管理的工具，它使开发人员可以创建隔离的软件产品，这些产品更易于部署，构建和修改。

什么是Pipenv？ (What is Pipenv?)

Pipenv combines package management and virtual environment control into one tool for installing, removing, tracking, and documenting your dependencies; and to create, use, and manage your virtual environments. Pipenv is essentially pip and virtualenv wrapped together into a single product.

Pipenv将软件包管理和虚拟环境控制结合到一个工具中，用于安装，删除，跟踪和记录依赖项。 以及创建，使用和管理您的虚拟环境。 Pipenv本质上是将pip和virtualenv打包到一个产品中。

For app deployment, GCP can build environments from a Pipfile. Pipenv will automatically update our Pipfile as we add and remove dependencies.

对于应用程序部署，GCP可以从Pipfile构建环境。 当我们添加和删除依赖项时，Pipenv将自动更新我们的Pipfile。

Docker和Dockerfiles (Docker and Dockerfiles)

Photo by Cameron Venti on Unsplash

卡梅隆·文蒂(Cameron Venti)摄于Unsplash

Docker is the best way to put apps into production. Docker uses a Dockerfile to build a container. The built container is stored in Google Container Registry were it can be deployed. Docker containers can be built locally and will run on any system running Docker.

Docker是将应用程序投入生产的最佳方式。 Docker使用Dockerfile来构建容器。 构建的容器可以存储在Google Container Registry中。 Docker容器可以在本地构建，并且可以在运行Docker的任何系统上运行。

GCP Cloud Build allows you to build containers remotely using the instructions contained in Dockerfiles. Remote builds are easy to integrate into CI/CD pipelines. They also save local computational time and energy as Docker uses lots of RAM.

GCP Cloud Build允许您使用Dockerfiles中包含的说明远程构建容器。 远程构建易于集成到CI / CD管道中。 由于Docker使用大量RAM，它们还节省了本地计算时间和精力。

Here is the Dockerfile we used for this project:

这是我们用于该项目的Dockerfile：

FROM python:3.7-slim
ENV APP_HOME /app
WORKDIR $APP_HOME
COPY . ./
RUN pip install pipenv
RUN pipenv install --deploy --system
CMD exec gunicorn --bind :$PORT --workers 1 --worker-class uvicorn.workers.UvicornWorker  --threads 8 app.main:app

The first line of every Dockerfile begins with FROM. This is where we import our OS or programming language. The next line, starting with ENV, sets our environment variable ENV to APP_HOME / app.

每个Dockerfile的第一行都以FROM.开头FROM. 这是我们导入操作系统或编程语言的地方。 下一行从ENV开始，将我们的环境变量ENV设置为APP_HOME / app.

These lines are part of the Python cloud platform structure and you can read more about them in the documentation.

这些行是Python云平台结构的一部分，您可以在文档中阅读有关它们的更多信息。

The WORKDIR line sets our working directory to /app. Then, the Copy line makes local files available in the docker container.

WORKDIR行将我们的工作目录设置为/app 。 然后，复制行使本地文件在Docker容器中可用。

The next three lines involve setting up the environment and executing it on the server. The RUN command can be followed with any bash code you would like executed. We use RUN to install pipenv. Then use pipenv to install our dependencies. Finally, the CMDline executes our HTTP server gunicorn, binds our container to $PORT, assigns the port a worker, specifies the number of threads to use at that port and finally states the path to the app asapp.main:app.

接下来的三行内容涉及设置环境并在服务器上执行环境。 您可以在RUN命令后跟随您要执行的任何bash代码。 我们使用RUN来安装pipenv。 然后使用pipenv安装我们的依赖项。 最后， CMD行执行我们的HTTP服务器gunicorn，将我们的容器绑定到$PORT ，为该端口分配一个工作线程，指定在该端口上使用的线程数，最后将应用程序的路径声明为app.main:app 。

You can add a .dockerignore file to exclude files from your container image. The .dockerignore is used to keep files out of your container. For example, you likely do not want to include your test suite in your container.

您可以添加.dockerignore文件以从容器映像中排除文件。 .dockerignore用于将文件保留在容器之外。 例如，您可能不想在容器中包含测试套件。

To exclude files from being uploaded to Cloud Build, add a.gcloudignore file. Since Cloud Build copies your files to the cloud, you may want to omit images or data to cut down on storage costs.

要排除文件无法上传到Cloud Build，请添加.gcloudignore文件。 由于Cloud Build将文件复制到云中，因此您可能希望省略图像或数据以降低存储成本。

If you would like to use these, be sure to check out the documentation for .dockerignore and .gcloudignorefiles, however, know that the pattern is the same as a.gitignore !

如果您想使用它们，请务必查看.dockerignore和.gcloudignore文件的文档，但是，请知道该模式与.gitignore相同！

应用部署 (App Deployment)

We need to make some final changes to our project files in preparation for deployment.

我们需要对我们的项目文件进行一些最终更改，以准备进行部署。

We need to add gunicorn and Pymysql to our Pipfile with the following.

我们需要使用以下命令将gunicorn和Pymysql添加到我们的Pipfile中。

pipenv install gunicorn pymysql

Git add the Pipfile, Pipfile.lock, and the Dockerfile you made earlier to your repository.

Git将之前创建的Pipfile，Pipfile.lock和Dockerfile添加到存储库中。

Docker Images和Google Container Registry (Docker Images and Google Container Registry)

Now, once we have our Dockerfile ready, build your container image using Cloud Build by running the following command from the directory containing the Dockerfile:

现在，一旦我们准备好Dockerfile，请通过包含Dockerfile的目录中运行以下命令，使用Cloud Build构建您的容器映像：

gcloud builds submit --tag gcr.io/PROJECT-ID/container-name

Note: Replace PROJECT-ID with your GCP project ID and container-name with your container name. You can view your project ID by running the command gcloud config get-value project.

注意：将PROJECT-ID替换为GCP项目ID，并将container-name替换为容器名称。 您可以通过运行命令gcloud config get-value project来查看您的项目ID。

This Docker image now accessible at the GCP container registry or GCR and can be accessed via URL with Cloud Run.

现在，可以在GCP容器注册表或GCR上访问此Docker映像，并且可以通过Cloud Run通过URL访问。

使用CLI部署容器映像 (Deploy the container image using the CLI)

1. Deploy using the following command:
   使用以下命令进行部署：

gcloud run deploy --image gcr.io/PROJECT-ID/container-name --platform managed

Note: Replace PROJECT-ID with your GCP project ID and container-name with your containers’ name. You can view your project ID by running the command gcloud config get-value project.

注意：将PROJECT-ID替换为GCP项目ID，并将container-name替换为容器的名称。 您可以通过运行命令gcloud config get-value project来查看您的项目ID。

2. You will be prompted for service name and region: select the service name and region of your choice.

2.系统将提示您输入服务名称和区域：选择所需的服务名称和区域。

3. You will be prompted to allow unauthenticated invocations: respond y if you want public access, and n to limit IP access to resources in the same google project.

3.系统将提示您允许未经授权的调用 ：响应y如果你想公共访问， n限制IP访问的资源在同谷歌项目。

4. Wait a few moments until the deployment is complete. On success, the command line displays the service URL.

4.等待片刻，直到完成部署。 成功后，命令行将显示服务URL。

5. Visit your deployed container by opening the service URL in a web browser.

5.通过在Web浏览器中打开服务URL，访问已部署的容器。

使用GUI部署容器映像 (Deploy the container image using the GUI)

Now that we have a container image stored in GCR, we are ready to deploy our application. Visit GCP cloud run and click create service, be sure to set up billing as required.

现在我们已经在GCR中存储了一个容器映像，现在可以部署我们的应用程序了。 访问GCP云运行并点击创建服务，请确保根据需要设置结算信息。

Select the region you would like to serve and specify a unique service name. Then choose between public or private access to your application by choosing unauthenticated or authenticated, respectively.

选择您要提供服务的区域并指定唯一的服务名称。 然后，分别通过选择未认证或已认证来选择对应用程序的公共或私人访问。

Now we use our GCR container image URL from above. Paste the URL into the space or click select and find it using a dropdown list. Check out the advanced settings to specify server hardware, container port and additional commands, maximum requests and scaling behaviors.

现在，我们从上方使用GCR容器图片网址。 将URL粘贴到空格中，或单击“选择”并使用下拉列表找到它。 检查高级设置以指定服务器硬件，容器端口和其他命令，最大请求数和扩展行为。

Click create when you’re ready to build and deploy!

准备好构建和部署时，请单击创建！

Selecting a Container image from GCR

从GCR选择容器图像

You’ll be brought to the GCP Cloud Run service details page where you can manage the service and view metrics and build logs.

您将被带到GCP Cloud Run服务详细信息页面，您可以在其中管理服务并查看指标和构建日志。

Services details

服务详情

Click the URL to view your deployed application!

单击URL查看已部署的应用程序！

Woohoo!

hoo！

Congratulations! You have just deployed an application packaged in a container image to Cloud Run. Cloud Run automatically and horizontally scales your container image to handle the received requests, then scales down when demand decreases. You only pay for the CPU, memory, and networking consumed during request handling.

恭喜你！ 您刚刚将打包在容器映像中的应用程序部署到了Cloud Run。 Cloud Run自动并水平缩放您的容器映像以处理收到的请求，然后在需求减少时缩小。 您只需为请求处理期间消耗的CPU，内存和网络付费。

That being said, be sure to shut down your services when you do not want to pay for them!

话虽如此，当您不想为服务付费时，请务必将其关闭！

GCP数据库设置和部署 (GCP Database Set up and Deployment)

Go to the cloud console and set up billing if you haven’t already. Now you can create an SQL instance.

转到云控制台并设置帐单(如果尚未设置)。 现在您可以创建一个SQL实例。

Select the SQL dialect you would like to use, we are using MySQL.

选择您要使用SQL方言，我们正在使用MySQL。

Set an instance ID, password, and location.

设置实例ID，密码和位置。

将MySQL实例连接到您的Cloud Run Service (Connecting the MySQL Instance to your Cloud Run Service)

Setting a new Cloud SQL connection, like any configuration change, leads to the creation of a new Cloud Run revision. To connect your cloud service to your cloud database instance:

与任何配置更改一样，设置新的Cloud SQL连接会导致创建新的Cloud Run修订版。 要将云服务连接到云数据库实例：

1. Go to Cloud Run

   前往Cloud Ru n

2. Configure the service:
   配置服务：

If you are adding a Cloud SQL connection to a new service:

如果要将Cloud SQL连接添加到新服务 ：

• You need to have your service containerized and uploaded to the Container Registry.
  您需要将服务容器化并上传到容器注册表。

• Click CREATE SERVICE.

  点击创建服务 。

If you are adding Cloud SQL connections to an existing service:

如果要将Cloud SQL连接添加到现有服务 ：

• Click on the service name.
  单击服务名称。

• Click DEPLOY NEW REVISION.

  单击“ 部署新版本” 。

3. Enable connecting to a Cloud SQL:

3.启用连接到Cloud SQL：

• Click SHOW OPTIONAL SETTINGS:

  点击显示可选设置 ：

• If you are adding a connection to a Cloud SQL instance in your project, select the desired Cloud SQL instance from the dropdown menu after clicking add connection.

  如果您要向项目中的Cloud SQL实例添加连接 ，请点击添加连接 ，然后从下拉菜单中选择所需的Cloud SQL实例。

• If you are using a Cloud SQL instance from another project, select connection string in the dropdown and then enter the full instance connection name in the format PROJECT-ID:REGION:INSTANCE-ID.

  如果您正在使用另一个项目中的Cloud SQL实例，请在下拉列表中选择连接字符串 ，然后以PROJECT-ID：REGION：INSTANCE-ID格式输入完整的实例连接名称。

4. Click Create or Deploy.

4.单击创建或部署 。

In either case, we’ll want our connection string to look like the one below for now.

无论哪种情况，我们现在都希望连接字符串看起来像下面的那样。

mysql://ael7qci22z1qwer:nn9keetiyertrwdf@c584asdfgjnm02sk.cbetxkdfhwsb.us-east-1.rds.gcp.com:3306/fq14casdf1rb3y3n

We’ll need to make a change to the DB connection string so that it uses the Pymysql driver.

我们需要更改数据库连接字符串，以便它使用Pymysql驱动程序。

In a text editor, remove the mysql and add in its place mysql+pymysql and then save the updated string as your SQL connection.

在文本编辑器中，删除mysql并在其位置添加mysql+pymysql ，然后将更新后的字符串保存为SQL连接。

mysql+pymysql

Note that you do not have to use GCP’s SQL. If you are using a third-party database, you can add the connection string as a VAR instead of Cloud SQL and input your connection string.

请注意，您不必使用GCPSQL。 如果您使用的是第三方数据库，则可以将连接字符串添加为VAR而不是Cloud SQL，并输入您的连接字符串。

用.env隐藏连接字符串 (Hiding connection strings with .env)

Locally, create a new file called .env and add the connection string for your cloud database as DB_CONN,shown below.

在本地，创建一个名为.env的新文件，并将云数据库的连接字符串添加为DB_CONN,如下所示。

DB_CONN=”mysql+pymysql://root:PASSWORD@HOSTNAME:3306/records_db”

Note: Running pipenv shell gives us access to these hidden environmental variables. Similarly, we can access the hidden variables in Python with os.

注意：运行pipenv shell使我们可以访问这些隐藏的环境变量。 同样，我们可以使用os访问Python中的隐藏变量。

MySQL_DB_CONN = os.getenv(“DB_CONN”)

Be sure to add the above line to your database.py file so that it ready to connect to the cloud!

确保将以上行添加到您的database.py文件中，以便它可以连接到云！

This .env file now contains sensitive information and should be added to your .gitignore so that it doesn't end up somewhere publically visible.

现在，此.env文件包含敏感信息，应将其添加到您的.gitignore ，以使它不会在公开可见的地方结束。

Now that we have our app and database in the cloud let’s ensure our system is working correctly.

现在，我们已将应用程序和数据库存储在云中，接下来，确保我们的系统正常运行。

加载数据库 (Loading the Database)

Once you can see the database listed on GCP, you are ready to load the database with a load script. The following gist includes our load.py script.

一旦您可以看到GCP上列出的数据库，就可以使用加载脚本加载数据库了。 以下要点包括我们的load.py脚本。

import csv
import datetime


from app import models
from app.database import SessionLocal, engine


db = SessionLocal()


models.Base.metadata.create_all(bind=engine)


with open("sars_2003_complete_dataset_clean.csv", "r") as f:
    csv_reader = csv.DictReader(f)


    for row in csv_reader:
        db_record = models.Record(
            date=datetime.datetime.strptime(row["date"], "%Y-%m-%d"),
            country=row["country"],
            cases=row["cases"],
            deaths=row["deaths"],
            recoveries=row["recoveries"],
        )
        db.add(db_record)


    db.commit()


db.close()

Let’s run this load script to see if we can post to our DB.

让我们运行此加载脚本，看看是否可以发布到数据库中。

First, run the following line to enter your virtual environment.

首先，运行以下行进入您的虚拟环境。

pipenv shell

Then run your load.py script.

然后运行您的load.py脚本。

python load.py

Visit the remote app address and see if your data has been added to the cloud database. Be sure to check your build logs to find tracebacks if you run into any issues!

访问远程应用程序地址，查看您的数据是否已添加到云数据库中。 如果遇到任何问题，请务必检查您的构建日志以查找回溯！

For more clarification on this loading process or setting up your app in a modular way, visit our Medium guide to building a data API! That article explains the code above in detail.

要详细了解此加载过程或以模块化方式设置应用，请访问我们的中型指南以构建数据API ！ 该文章详细解释了上面的代码。

结论 (Conclusion)

In this article, we learned a little about environment management with pipenv and how to Dockerize apps. Then we covered how to store a Docker container in Google Container Registry and deploy the container with the Cloud Build CLI and GUI. Next, we set up a cloud SQL database and connected it to our app. Lastly, we discussed one way to load the database, running load.py locally. Note that if your app collects data itself, you only need to deploy the app and database, then the deployed app will populate the database as it collects data.

在本文中，我们了解了一些有关使用pipenv进行环境管理以及如何对应用程序进行Dockerize的知识。 然后，我们介绍了如何在Google Container Registry中存储Docker容器以及如何使用Cloud Build CLI和GUI部署该容器。 接下来，我们建立一个云SQL数据库并将其连接到我们的应用程序。 最后，我们讨论了一种加载数据库的方法，即在本地运行load.py。 请注意，如果您的应用程序本身收集数据，则只需部署应用程序和数据库，然后部署的应用程序将在收集数据时填充数据库。

Here is a link to the GitHub repository with our code for this project. Be sure to check out this code to see how we set up the whole codebase!

这是到GitHub存储库的链接，其中包含我们针对该项目的代码。 请务必查看此代码，以了解我们如何设置整个代码库！

翻译自: https://towardsdatascience.com/deploy-a-dockerized-flask-app-to-google-cloud-platform-71d91b39b25e