python cloud_我在Google Cloud App Engine Python 3中的3年经验python 3

python cloud

介绍 (Introduction)

Google Cloud has always believed in the vision of serverless by debuting with Google App Engine in 2008, the first fully serverless compute service. Since then, Google has evolved more serverless offerings in both application development and analytics.

Google Cloud始终相信无服务器的愿景，它在2008年首次推出了第一个 完全 无服务器的计算服务Google App Engine。 从那时起，谷歌发展了更多的无服务器 应用程序开发和分析中提供的产品。

I started working in Google App Engine in May 2017 and I am going to share my experience with GAE.

我于2017年5月开始在Google App Engine工作，我将与GAE分享我的经验。

The reason we chose Google App Engine is its fully managed, scalable, flexible, easy to use, quick to build and deploy code in to production.

我们选择Google App Engine的原因是其完全托管，可扩展，灵活，易于使用，可快速将代码构建和部署到生产环境中。

We are working in agile and deliver microservices to production every 2 weeks thanks to GAE.

由于GAE，我们正在敏捷地工作，每2周将微服务交付生产。

It is fascinating, unique piece of service Google is offering.

Google提供了令人着迷的独特服务。

** This is before Cloud Function and Cloud Run were introduced.**

**这是在介绍Cloud Function和Cloud Run之前的**。

App Engine的功能 (Features of App Engine)

• Fully managed: A fully managed environment lets you focus on code

  完全托管：完全托管的环境使您可以专注于代码

• Application versioning : Easily host different versions of your app.

  应用程序版本控制 ：轻松托管您应用程序的不同版本。

• Traffic splitting: Route incoming requests to different app versions, A/B test, and do incremental feature rollouts.

  流量拆分 ：将传入的请求路由到不同的应用版本，进行A / B测试，并逐步推出功能。

• Application security: Safeguard application by IAP (Identify Aware Proxy)

  应用程序安全性：通过IAP(识别感知代理)保护应用程序

• Services ecosystem: Excellent integration with different Google Services.

  服务生态系统：与其他Google Services的出色集成。

App Engine类型 (App Engine Types)

You can run the application using App Engine Standard or App Engine Flexible environment.

您可以使用App Engine Standard或App Engine Flexible环境运行该应用程序。

We have split our application in microservices and mostly used Standard environments which scales down to zero instances when there is no traffic. This saves lot of money (which customer wants). We hardly incur costs for our Development and Testing environments.

我们将应用程序拆分为微服务和最常用的标准环境，该环境在没有流量的情况下可缩减为零实例。 这样可以节省很多钱(客户想要的)。 我们几乎不为我们的开发和测试环境招致成本。

Choosing the environment as per Google.

根据Google 选择环境 。

https://cloud.google.com/appengine/docs/the-appengine-environments https://cloud.google.com/appengine/docs/the-appengine-environments

App Engine标准环境 (App Engine Standard Environment)

Standard environment is based on container instance running on Google infrastructure and its preconfigured with the available runtimes. Since we were using Python2.7 and Python 3.7 programming language, we were OK to use Standard environment.

标准环境基于在Google基础架构上运行的容器实例，并且该容器实例已预先配置了可用的运行时。 由于我们使用的是Python2.7和Python 3.7编程语言，因此可以使用Standard环境。

Instance classes: We used F1 instance (256 MB RAM, 600 MHz CPU).

实例类：我们使用了F1实例(256 MB RAM，600 MHz CPU)。

App Engine弹性环境 (App Engine Flexible Environment)

Applications that receive consistent traffic, experience regular traffic fluctuations, or meet the parameters for scaling up and down gradually.

接收一致流量，遇到常规流量波动或满足逐步扩展和缩减参数的应用程序。

It runs on docker container so you can choose any programming language.

它在docker容器上运行，因此您可以选择任何编程语言。

服务和版本 (Services and Versions)

App Engine works on microservice architecture. Each application is divided into multiple services. Therefore, you can run your whole app in a single service or you can design and deploy multiple services to run as a set of microservices.

App Engine适用于微服务架构。 每个应用程序分为多个服务。 因此，您可以在单个服务中运行整个应用程序，也可以设计和部署多个服务以作为一组微服务运行。

The nice thing in App Engine is that everything you deploy has specific version. Each version can be access at https://<version>-dot-<app-id>.appspot.com.

App Engine的好处是，您部署的所有内容都有特定的版本。 可以通过https：// <version> -dot- <app-id> .appspot.com访问每个版本。

Having multiple versions of your app within each service allows you to quickly switch between different versions of that app for rollbacks, testing, or other temporary events.

每个服务中都有多个版本的应用程序，使您可以在该应用程序的不同版本之间快速切换，以进行回滚，测试或其他临时事件。

The versions within your services run on one or more instances. By default, App Engine scales your app to match the load. Your apps will scale up the number of instances that are running to provide consistent performance, or scale down to minimise idle instances and reduces costs.

服务中的版本可在一个或多个实例上运行。 默认情况下，App Engine会缩放您的应用程序以匹配负载。 您的应用程序将按比例缩放正在运行的实例数量以提供一致的性能，或按比例缩小以最小化空闲实例并降低成本。

缩放类型 (Scaling Types)

There are three scaling types by which Google controls how instances are created.

Google可以通过三种缩放类型来控制实例的创建方式。

Automatic: Dynamic as per requests.You can specify idle instances.

自动：根据请求动态。您可以指定空闲实例。

Basic: Dynamic as per requests.

基本：根据要求动态。

Manual: Resident.Detailed comparisons in mentioned in the link

手册：居民。链接中提到的详细比较

service: sample-service


runtime: python38 # or python37 for Python 3.7


instance_class: F2


includes:
- env_variables.yaml

服务帐号 (Service Account)

After you create an App Engine application, the App Engine default service account is created and used as the identity of the App Engine service. The App Engine default service account is associated with your Cloud project and executes tasks on behalf of your apps running in App Engine.

创建App Engine应用程序后，将创建App Engine默认服务帐户 ，并将其用作App Engine服务的身份。 App Engine默认服务帐户与您的Cloud项目相关联，并代表您在App Engine中运行的应用程序执行任务。

By default, the App Engine default service account has the Editor role in the project. This means that any user account with sufficient permissions to deploy changes to the Cloud project can also run code with read/write access to all resources within that project.

默认情况下，App Engine默认服务帐户在项目中具有“编辑者”角色。 这意味着具有足够权限将更改部署到Cloud项目的任何用户帐户也可以运行对项目中所有资源具有读/写访问权限的代码。

Below is sample reference architecture that we develop app using Google App Engine.

以下是我们使用Google App Engine开发应用程序的示例参考架构。

整合方式 (Integrations)

We have integrated Google App Engine with many Google’s services.

我们已将Google App Engine与许多Google服务集成在一起。

数据存储/ Firestore (Datastore / Firestore)

The Datastore is a proprietary NoSQL database, fully managed by Google. Early days, the Datastore was tightly coupled with Google App Engine.

数据存储区是专有的NoSQL数据库，由Google完全管理。 早期，数据存储区与Google App Engine紧密结合。

from google.cloud import datastore


datastore_client = datastore.Client()


kind = 'Employee'
id = 11223344
task_key = datastore_client.key(kind, id)


task = datastore.Entity(key=task_key)
task['name'] = 'Vikram'
task['location'] = 'Leeds'


datastore_client.put(task)

谷歌云存储 (Google Cloud Storage)

The source system uploads files to the bucket. The application gets triggered when file is uploaded.

源系统将文件上传到存储桶。 上传文件时触发应用程序。

from google.cloud import storage


client = storage.Client()


bucket = client.get_bucket('my-bucket-name')
blob = storage.Blob('path/to/blob', bucket)


with open('file-to-download-to') as file_obj:
    client.download_blob_to_file(blob, file_obj)

云任务 (Cloud Tasks)

By using Cloud Tasks, you can enqueue a new task with a payload. No need for custom retry code, Cloud Task will handle it.

通过使用“云任务”，您可以将新任务与有效负载排队。 无需自定义重试代码，Cloud Task会处理它。

from google.cloud import tasks_v2


client = tasks_v2.CloudTasksClient()


parent = client.queue_path(project, location, queue)
task = {
    'app_engine_http_request': {
        'http_method': 'POST',
        'relative_uri': url,
        'headers': {'Content-type': 'application/json'}
    }
    }
response = client.create_task(parent, task)
logging.info('Task created.')

发布/订阅 (Pub/Sub)

The Pub/Sub service allows applications to exchange messages reliably, quickly, and asynchronously. We have multiple subscribers to the topic.

发布/订阅服务允许应用程序可靠，快速和异步地交换消息。 我们有该主题的多个订阅者。

This allows you to send and receive messages between independent applications.

这使您可以在独立的应用程序之间发送和接收消息。

import json
from google.cloud import pubsub_v1


topic_name = payload.get('topic')
data = payload.get('data')
attributes = payload.get('attributes')


publisher = pubsub_v1.PublisherClient()
topic_path = publisher.topic_path(project_id, topic_id)


message = json.dumps(data).encode('utf-8')
future = publisher.publish(topic=topic_name, data=message, attrs=json.dumps(attributes))
print("Published messages with custom attributes. Message id: {}".format(future.result()))

大查询 (BigQuery)

BigQuery is a data warehouse as a service, managed by Google. You import data which can be petabytes and can run analyses via a custom query language.

BigQuery是由Google管理的数据仓库即服务。 您导入的数据可能是PB，并且可以通过自定义查询语言运行分析。

from google.cloud import bigquery


client = bigquery.Client()


table_id = "my_first_project.dataset1.sample_table"


schema = [
    bigquery.SchemaField("full_name", "STRING", mode="REQUIRED"),
    bigquery.SchemaField("age", "INTEGER", mode="REQUIRED"),
]


def create_table():
    table = bigquery.Table(table_id, schema=schema)
    table = client.create_table(table)
    print(
        "Created table {}.{}.{}".format(table.project, table.dataset_id, table.table_id)
    )


def query_table():
    QUERY = f'SELECT name FROM `{table_id}` '
    query_job = client.query(QUERY)  # API request
    rows = query_job.result()  # Waits for query to finish


    return rows

记忆快取(Python2) (Memcache (Python2))

The other crucial way to store data on App Engine is Memcache. By default, you get a shared Memcache. Just annotate an entity with @Cache and it will ask Memcache before the Datastore and save every entity there first. This can have a tremendous effect on performance. Usually Memcache will respond within about 5 ms, which is much faster than the Datastore.

在App Engine上存储数据的另一种至关重要的方法是Memcache。 默认情况下，您将获得一个共享的 Memcache 。 只需使用@Cache注释实体， @Cache在数据存储区之前询问Memcache并首先将每个实体保存在那里。 这会对性能产生巨大影响。 通常，Memcache将在约5毫秒内响应，这比数据存储区快得多。

from google.appengine.api import memcache


def get_data(key, ttl_in_seconds=3600):
    data = memcache.get(key)
    if data is not None:
        return data
    else:
        data = query_for_data()
        memcache.set(key=key, value=data, time=ttl_in_seconds)
    return data

Note: For Python3, Memcache are not available. You need to setup Memorystore for redis.

注意 ：对于Python3，Memcache不可用。 您需要为Redis设置Memorystore 。

API端点 (API Endpoints)

Cloud Endpoints is an API management system that helps you secure, monitor, analyze and set quotas on your APIs.

Cloud Endpoints是一个API管理系统，可帮助您保护，监视，分析和设置API的配额。

We have configured and deployed API using ESP/V1 running services on App Engine Flexible environment.

我们已经在App Engine Flexible环境中使用ESP / V1运行服务配置和部署了API。

swagger: "2.0"
info:
  description: "A simple Google Cloud Endpoints API example."
  title: "Endpoints Example"
  version: "1.0.0"
host: "YOUR-PROJECT-ID.appspot.com"

部署 (Deployments)

Deployment to App Engine very easy.

部署到App Engine非常容易。

 gcloud app deploy

This command automatically builds a container image using Cloud Build service, push it to Google Container Registry and then deploys the image to the Google App Engine.

此命令使用Cloud Build服务自动构建容器映像，将其推送到Google Container Registry ，然后将映像部署到Google App Engine。

—金丝雀发行 (— Canary Release)

Even if you release pipeline is fully automated by all unit tests and integrated tests, you won’t be able to detect all release-related defects until real traffic is hitting to the service. Test environments aren’t 100% identical to production and tests probably don’t cover 100% of possible scenarios.

即使您通过所有单元测试和集成测试完全自动化了发布管道，您也将无法检测到所有与发布相关的缺陷，直到真正的流量流向服务为止。 测试环境与生产环境并非100％相同，并且测试可能无法涵盖100％的可能情况。

The simple canary deployment in Google App Engine is splitting traffic between new versions.

Google App Engine中的简单金丝雀部署可在新版本之间分配流量。

After deploying new version and splitting traffic, you can monitor the logs.

部署新版本并拆分流量后，您可以监视日志。

—蓝绿色部署 (— Blue Green Deployment)

Blue/green deployment maintains two instances of a system: one that is serving traffic (green), and another that is ready to serve traffic (blue). After deploying a new release in the blue environment, you can then move traffic to it. The cutover doesn’t require downtime, and rollback is a trivial reversal of the router change.

蓝色/绿色部署维护系统的两个实例：一个正在为流量提供服务(绿色)，另一个已为流量提供服务(蓝色)。 在蓝色环境中部署新版本后，可以将流量移至该版本。 切换不需要停机，回滚是对路由器更改的简单逆转。

监控/运营 (Monitoring / Operations)

The Google Cloud Console brings a lot of features to diagnose you App’s behaviour in Production.

Google Cloud Console提供了许多功能来诊断您App在生产中的行为。

Cloud Monitoring collect metrics, logs and traces for the application.

云监控收集应用程序的指标，日志和跟踪。

It also provides built-in out of box dashboards and views to monitor the applications.

它还提供了内置的现成仪表板和视图来监视应用程序。

It also give ability to alerts and notification rules.

它还具有警报和通知规则的功能。

Debugger lets you inspect the state of an application at any code location without stopping or slowing it down. The debugger makes it easier to view the application state without adding logging statements.

调试器使您可以在任何代码位置检查应用程序的状态，而无需停止或降低其运行速度。 调试器使您无需添加日志记录语句即可更轻松地查看应用程序状态。

结论 (Conclusion)

Quickly releasing a produce or a service to market is the most important thing for every business. Stimulating the development and maintenance of an app is critical when it comes to deploying the product fast. With the help of Google App Engine you can achieve both.

对于每个企业来说，快速将产品或服务投放市场是最重要的。 在快速部署产品时，刺激应用程序的开发和维护至关重要。 借助Google App Engine，您可以同时实现两者。

Note: Now, we have started using using Cloud Functions and Cloud Run to quickly deliver to production but Google App Engine has always been our first Love…

注意 ：现在，我们已经开始使用Cloud Functions和Cloud Run快速交付生产，但是Google App Engine一直是我们的初恋。

翻译自: https://levelup.gitconnected.com/my-3-years-experience-in-google-cloud-app-engine-python-2-python-3-ad9af7d87ff8

python cloud