高德地图 自动计算缩放
Pioneered by AWS Lambda, serverless computing is the latest paradigm in cloud service. Nowadays, almost all large cloud provider companies are offering Function-as-a-Service (FaaS) products which promise developers the total management of the services by the provider. Some of the current Faas offerings include AWS Lambda, Google Cloud Functions, Google Cloud Run, Azure Functions, and IBM Cloud Functions. There are also other open-source alternatives that mostly run on Kubernetes like OpenFaaS, IronFunctions, Apache OpenWhisk, Kubeless, Fission, OpenLambda, and Knative.
无服务器计算是AWS Lambda的先锋，是云服务中的最新范例。 如今，几乎所有大型云提供商的公司都在提供功能即服务(FaaS)产品，这些产品向开发人员保证将由提供商对服务进行全面管理。 目前的Faas产品包括AWS Lambda，Google Cloud Functions，Google Cloud Run，Azure Functions和IBM Cloud Functions。 还有其他大多数开源替代方案都可以在Kubernetes上运行，例如OpenFaaS，IronFunctions，Apache OpenWhisk，Kubeless，Fission，OpenLambda和Knative。
In this post, I will introduce three different autoscaling patterns used in different serverless computing services which might help you decide which one works best for the project you are planning to start 😉.
在本文中，我将介绍在不同的无服务器计算服务中使用的三种不同的自动缩放模式，这些模式可以帮助您确定哪种模式最适合您计划开始的项目。
自动缩放 (Autoscaling)
One key aspect in the operations which is handled by the provider in FaaS offerings is the autoscaling of the service. When we say a service “austoscales”, we generally mean that when the traffic generated automatically or by the user changes over time, the amount of resources (CPU/RAM/IO/…) allocated to that service will change to adapt accordingly.
FaaS产品中提供商所处理的操作中的一个关键方面是服务的自动扩展。 当我们说一个服务是“自动扩展的”时，通常是指当自动产生的流量或用户产生的流量随时间变化时，分配给该服务的资源量(CPU / RAM / IO /…)将发生变化以相应地适应。
This way, the cloud-based software products can manage their cloud bill while maintaining their availability and responsiveness of their website. Before autoscaling was made possible, companies would need to provision a server with enough resources to handle the largest peaks in the traffic routed to their website, while paying for it the whole time.
这样，基于云的软件产品可以管理其云账单，同时保持其可用性和网站响应能力。 在实现自动扩展之前，公司需要为服务器提供足够的资源，以处理路由到其网站的流量最大的高峰，同时全程付费。
When deploying your services to FaaS products, you are delegating the task of autoscaling (an many other operations tasks) to the cloud provider. However, since life is not perfect and all systems have trade-offs and limitations, you might need to have some insight into how your cloud provider is managing the task behind the scenes. That’s what I hope you will learn if you keep on reading.
将服务部署到FaaS产品时，您会将自动扩展的任务(许多其他操作任务)委派给了云提供商。 但是，由于生活并不完美，并且所有系统都有权衡和限制，因此您可能需要对云提供商如何管理后台任务有一些了解。 这就是我希望您继续阅读所学的知识。
Autoscaling in FaaS usually contains two actions: scaling-in and scaling-out. When you deploy your function to a service (e.g., AWS Lambda), your functions will be deployed to their data centers in “function instances”, which are basically tiny servers that have your code on them. When you are sending more traffic than the system currently can handle, a scale-out event will happen, creating new functions instances to serve the incoming requests in parallel. On the contrary, when the incoming traffic is lower than the current system capacity, a scale-in event will happen, causing the termination of some of those function instances.
FaaS中的自动缩放通常包含两个操作：放大和缩小。 当您将功能部署到服务(例如，AWS Lambda)时，您的功能将被部署到“功能实例”中的数据中心，这些“功能实例”基本上是上面装有代码的微型服务器。 当您发送的流量超出系统当前的处理能力时，将发生横向扩展事件，从而创建新的功能实例来并行处理传入的请求。 相反，当传入流量低于当前系统容量时，将发生放大事件，从而导致其中一些功能实例终止。
自动缩放模式 (Autoscaling Patterns)
Autoscaling pattern is in fact the policy (guideline) that the system will follow to trigger those scaling in and out events. In the most popular serverless computing platforms I’ve seen so far, there are generally three autoscaling patterns: 1) scale-per-request (you might see a different name somewhere else, but this is the name I used in my papers 😊); 2) concurrency value scaling; 3) metrics-based scaling. Now, let’s dig deeper into how each one works.
实际上，自动缩放模式是系统触发这些缩放事件的策略(准则)。 到目前为止，在我所见过的最流行的无服务器计算平台中，通常有三种自动缩放模式：1)按请求缩放(在其他地方可能会看到其他名称，但这是我在论文中使用的名称😊) ; 2)并发值缩放； 3)基于指标的缩放。 现在，让我们更深入地研究每个人的工作方式。
1.按请求缩放自动缩放 (1. Scale-Per-Request Autoscaling)
This is the de-facto autoscaling pattern (and the hardest one to get right) for public FaaS offerings like AWS Lambda, IBM Cloud Functions, Azure Functions, and Google Cloud Functions (not Google Cloud Run, but we will get to that later), and in addition Apache OpenWhisk (which is open source).
这是AWS Lambda，IBM Cloud Functions，Azure Functions和Google Cloud Functions(不是Google Cloud Run，但稍后会介绍)的公共FaaS产品的事实上的自动缩放模式(也是最难解决的一种)。 ，此外还有Apache OpenWhisk(它是开放源代码 )。
In this autoscaling pattern, the system is completely reactive and it needs to be able to react fast, which is why it is the hardest one to get right. Here, when you deploy your application to the cloud, the provider will not run any instances of your function to save costs. Instead, it will wait until a request (or any kind of trigger) comes along for your service. Then, the system will go through the process of creating a function instance and downloading your code onto it real fast, and only then it will process that request. This is commonly known as a cold start and it is how the scaling-out event is handled in these serverless platforms. This process usually takes some time (from a few hundred milliseconds to tens of seconds) and thus will cause a degradation in the quality of service (like response time).
在这种自动缩放模式中，系统是完全React性的，它需要能够快速React，这就是为什么它是最难解决的一个原因。 在这里，当您将应用程序部署到云中时，提供程序将不会运行任何功能实例来节省成本。 相反，它将等待直到为您的服务提出请求(或任何类型的触发器)。 然后，系统将完成创建函数实例并将您的代码真正快速地下载到该函数实例的过程，然后，它将处理该请求。 这通常称为冷启动 ，这是在这些无服务器平台中处理扩展事件的方式。 此过程通常需要一些时间(从几百毫秒到几十秒)，因此将导致服务质量下降(例如响应时间)。
To avoid doing this process for every single request, the serverless platform will not terminate this newly created instance right after the request is done being processed. Instead, it will keep this instance for some period of time (referred here as the termination timeout) before terminating it, causing the scale-in event. In case another request comes for the service before the termination, the system will instead route the request to the warm instance, saving a bit of those precious latencies and computational costs.
为了避免对每个单个请求执行此过程，无服务器平台将不会在处理完请求后立即终止此新创建的实例。 相反，它将在终止实例之前将其保留一段时间(在此称为终止超时 )，从而引起扩展事件。 如果在终止之前另一个请求到达服务，则系统会将请求路由到热实例，从而节省了一些宝贵的等待时间和计算成本。
One key difference between this autoscaling pattern and the other ones (which we will discuss later) is the fact that here there is no queuing for incoming requests here. Instead, the system will either assign a warm instance to the new request or create a new one (leading to a cold start). Thus, there is an upper limit to the latency that each request will experience, which is the maximum time it takes for a cold start request to be processed; as opposed to waiting in a queue of requests for a long time to be processed (maybe even minutes).
这种自动缩放模式与其他自动缩放模式(我们将在后面讨论)之间的一个主要区别是，这里没有传入请求的排队。 相反，系统将为新请求分配一个热实例，或者创建一个新实例(导致冷启动)。 因此，每个请求将经历的等待时间有一个上限，这是处理冷启动请求所花费的最长时间。 而不是在请求队列中等待较长时间(甚至是几分钟)来处理。
2.并发值自动缩放 (2. Concurrency Value Autoscaling)
In concurrency value autoscaling pattern, each function instance can receive multiple requests concurrently. In this pattern, the user specifies a maximum number of requests which are allowed to enter the instance at the same time, and when this number is reached, a new request causes a cold start and consequently the creation of a new function instance. The figure above shows the effect of the maximum concurrency value set by the user.
在并发值自动缩放模式中，每个功能实例可以同时接收多个请求。 在这种模式下，用户指定允许同时进入实例的最大请求数，当达到该数目时，新请求将导致冷启动，并因此创建新的功能实例。 上图显示了用户设置的最大并发值的影响。
This autoscaling pattern might look as a superclass of scale-per-request autoscaling and they might seem to converge to the same concurrency pattern when the maximum concurrency value is set to 1. However, there are fundamental differences between these two autoscaling patterns which have led us to classify them into separate categories. First, concurrency value autoscaling allows queuing of incoming requests in a shared queue, but there is no queuing involved in scale-per-request autoscaling. Besides, scaling out in scale-per-request happens synchronous to incoming requests, meaning the event of creating a new function instance follows the reception of a new request when there are no idle instances available. However, concurrency value autoscaling platforms perform asynchronous scaling on fixed intervals (2 seconds in current implementations) based on average concurrency value measured throughout the stable and panic windows.
这种自动缩放模式可能看起来像是按请求缩放自动缩放的超类，并且当最大并发值设置为1时，它们似乎收敛于相同的并发模式。但是，这两种自动缩放模式之间存在根本差异，导致我们将它们分为不同的类别。 首先，并发值自动缩放允许在共享队列中对传入的请求进行排队，但是按请求缩放自动缩放不涉及排队。 此外，按请求扩展的扩展与传入请求同步发生，这意味着在没有空闲实例可用时，在接收到新请求之后便会创建新的功能实例。 但是，并发值自动缩放平台会根据在整个稳定窗口和紧急窗口内测得的平均并发值，在固定间隔(当前实现中为2秒)上执行异步缩放。
Examples of this autoscaling pattern are Google Cloud Run and some configurations of Knative serving.
这种自动缩放模式的示例包括Google Cloud Run和Knative服务的某些配置。
Examples of this autoscaling pattern are Google Cloud Run and some configurations of Knative serving.
这种自动缩放模式的示例包括Google Cloud Run和Knative服务的某些配置。
3.基于指标的自动缩放 (3. Metrics-Based Autoscaling)
This is the most common autoscaling pattern among general microservices deployed to the cloud. In this pattern, the system tries to keep metrics like CPU, memory usage, throughput, or latency within a predefined range. Most of the on-prem and open-source serverless computing platforms like OpenFaaS, Kubeless, and Fission, and some of the managed platforms like AWS Fargate and Azure Container Instances use this autoscaling patterns. Most of these examples use the Horizontal Pod Autoscaler (HPA) built into Kubernetes which is the de-facto standard for deploying microservices to the cloud.
这是部署到云中的常规微服务中最常见的自动扩展模式。 在这种模式下，系统尝试将指标(例如CPU，内存使用率，吞吐量或延迟)保持在预定义范围内。 大多数本地和开源无服务器计算平台(例如OpenFaaS，Kubeless和Fission)以及某些托管平台(例如AWS Fargate和Azure容器实例)都使用此自动缩放模式。 这些示例大多数使用Kubernetes内置的Horizontal Pod Autoscaler(HPA)，这是将微服务部署到云的实际标准。
Compared to the other two autoscaling patterns, this one has the worst delay in reacting to the bursty workloads but can achieve better resource utilization and cost in services with a high incoming request rate.
与其他两种自动扩展模式相比，该模式在对突发工作负载做出响应时延迟最严重，但可以在具有较高传入请求率的服务中实现更好的资源利用率和成本。
结论 (Conclusion)
The promise of serverless computing is that the developer only needs to provide the code and leave the operations effort to be done by the provider. However, the current generation of serverless computing platforms is still far from perfect. As a result, the developers need to know about the underlying structure to be able to deliver on the required quality for the services they intend to deploy on the platform.
无服务器计算的承诺是，开发人员只需要提供代码，而让操作人员去做就可以了。 但是，当前的无服务器计算平台尚不完善。 结果，开发人员需要了解底层结构，以便能够为他们打算在平台上部署的服务提供所需的质量。
In this article, we reviewed the most common autoscaling patterns, one of the most important operations efforts and promises of serverless, in current serverless computing platforms. Knowing these autoscaling patterns and which platforms use them, I hope that you can make better decisions for deploying your workload.
在本文中，我们回顾了当前无服务器计算平台中最常见的自动缩放模式，无服务器的最重要的操作工作和承诺。 了解了这些自动扩展模式以及哪些平台使用了它们，我希望您可以为部署工作负载做出更好的决策。
关于我 (About Me)
I am a Ph.D. student at the University of Alberta, a visiting researcher at York University, and a part-time Instructor at Seneca College. Day-in day-out I research serverless computing platforms, trying to find ways to improve their performance, reliability, energy consumption, etc., using formal or data-driven methods (fancy words for “I either use mathematics or machine learning to model serverless computing platforms”). What I wrote here is the result of my insight into some of the serverless computing platforms that I have worked with during my research and a brief compilation of their documentation regarding their autoscaling patterns.
我是博士 艾伯塔大学(University of Alberta)的学生，约克大学(York University)的访问学者以及塞内卡学院(Seneca College)的兼职讲师。 我日复一日地研究无服务器计算平台，试图找到使用正规或数据驱动方法来提高其性能，可靠性，能耗等的方法(用“我使用数学或机器学习来建模”无服务器计算平台”)。 我在这里写的内容是我对研究期间使用过的一些无服务器计算平台的深入了解，并简要汇编了有关其自动缩放模式的文档。
In case you want to know more about me, check out my website.
如果您想进一步了解我，请访问我的网站 。
高德地图 自动计算缩放
844
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
