c ++ lambda_使用Zeebe + Lambda进行更清洁的微服务编排

c ++ lambda

In this post I’ll talk about Zeebe. Zeebe is a source available workflow engine that helps you define, orchestrate, and monitor business processes across microservices. I’ll show you how to quickly get a full application that leverages Zeebe and Lambda running on AWS. I’m piggybacking off a post by the company’s co-founder in which he showed how you can use Zeebe with Lambda. You may want to start with that one first for a general overview of Zeebe and how it fits in with Serverless, then if you’re interested in quickly getting all the components running on AWS this post will show you how.

在这篇文章中，我将讨论Zeebe。 Zeebe是源可用的工作流引擎，可帮助您定义，协调和监视微服务之间的业务流程。 我将向您展示如何快速利用在AWS上运行的Zeebe和Lambda的完整应用程序。 我背负着该公司联合创始人的一则帖子，他在其中展示了如何在Lambda上使用Zeebe 。 您可能首先要开始对Zeebe进行一般概述，以及它如何与Serverless配合使用，然后，如果您有兴趣快速让所有组件在AWS上运行，那么这篇文章将向您展示如何。

If you embrace AWS Lambda or are considering adopting it you may see Lambda as an alternative to Kubernetes and prefer the simplicity, reliability, and out of the box scaling that the Serverless way offers. I was motivated to write this for that type of developer. For Zeebe to work we need something running all the time so we can’t be fully “serverless”. My concession is to use ECS to bridge the gap. ECS is a simpler and cheaper alternative to Kubernetes when you’re getting started with containers. If you are a serverless first developer you or your organization might not want to take on the overhead of running and learning k8s if almost all your workloads work fine on Lambda anyway. Zeebe is a super powerful tool that allows you to compose your Lambda functions to build arbitrarily complex stateful applications. Today I’ll show how to get up and running quick with Zeebe and Lambda using ECS.

如果您接受AWS Lambda或正在考虑采用AWS Lambda，则可能会将Lambda视为Kubernetes的替代方案，并且更喜欢无服务器方式提供的简单性，可靠性和即开即用的可扩展性。 我很乐意为那种类型的开发人员写这篇文章。 为了使Zeebe正常工作，我们需要始终保持运行状态，以便不能完全“无服务器”运行。 我的让步是使用ECS弥补差距。 当您开始使用容器时，ECS是Kubernetes的一种更简单，更便宜的替代方案。 如果您是无服务器的第一个开发人员，那么您或您的组织可能不想承担运行和学习k8的开销，即使您几乎所有工作量都可以在Lambda上正常工作。 Zeebe是一个超级强大的工具，可让您组合Lambda函数来构建任意复杂的有状态应用程序。 今天，我将展示如何使用ECS与Zeebe和Lambda一起快速启动和运行。

用例？ (Use Case?)

In case you don’t know about Zeebe I’ll try to describe what it does by describing a use case. A little while ago I was working on a backend that would allow customers to put a down payment on a vehicle then go pick it up at a dealership. The whole process wasn’t too complicated but did involve 5–6 discreet steps involving external API calls. Any of those steps had a number of potential failure paths and based on the outcome of each step you would want to take a different action. On top of that there are different types of errors. There are errors due to conditions in the physical world like a customer putting their address in wrong and there are technical errors like network outages or a bug in the code. You can imagine how the number of scenarios grows exponentially each time you add a step. General purpose programming languages on their own are great at a lot of things, Orchestrating a complicated sequence of steps is not one of them. Using event driven architecture can help us manage emerging complexity by separating business logic into individual microservices whose isolated behavior is simpler to reason about. What we lose is end to end visibility of the business process, so that debugging is a challenge and refactors are scary. This is where a tool like Zeebe, or its predecessor Camunda can help. These tools let us represent our processes as diagrams that are executable. Here is an example:

如果您不了解Zeebe，我将尝试通过描述用例来描述它的作用。 不久前，我正在开发一个后端，该后端允许客户在车辆上支付首付，然后在经销商处取货。 整个过程不太复杂，但确实涉及5–6个谨慎的步骤，涉及外部API调用。 这些步骤中的任何一个都有许多潜在的失败路径，根据每个步骤的结果，您可能希望采取不同的措施。 除此之外，还有不同类型的错误。 物理世界中的情况会导致错误，例如客户将其地址放错了地址，而网络故障或代码中的错误等技术错误也将导致。 您可以想象每添加一个步骤，方案的数量将成倍增长。 通用编程语言本身在很多方面都很出色，编排复杂的步骤序列并不是其中之一。 使用事件驱动的体系结构可以通过将业务逻辑划分为单个微服务来管理新兴的复杂性，这些微服务的孤立行为更易于推论。 我们失去的是业务流程的端到端可见性，因此调试是一个挑战，重构很可怕。 像Zeebe或其前身Camunda这样的工具可以在此提供帮助。 这些工具使我们可以将过程表示为可执行的图。 这是一个例子：

One of the main advantages is that our business logic is now organized into manageable and isolated components. In addition, and in my opinion most importantly, this strategy separates the concern of Orchestration from everything else. This allows us to write functional style microservice components that are easy to read and refactor. All of our orchestration is managed by a dependable and tested framework, and the orchestration logic is cleanly separated as opposed to peppered throughout our microservices (as tends to happen). The boxes with gears are “service tasks” which means they represent pieces of code external to Zeebe that execute. The process engine (Zeebe) is responsible for executing the instructions and will invoke the code we’ve written where specified. In today’s example the service tasks (little boxes with gears ) will be implemented as Lambda functions. There are a lot of other powerful features exposed by BPMN (business process modeling notation), the ones pictured above are just a few.

主要优势之一是我们的业务逻辑现在被组织为可管理和隔离的组件。 另外，在我看来最重要的是，这种策略分开编排的，从一切的关注。 这使我们能够编写易于阅读和重构的功能样式微服务组件。 我们所有的业务流程都由一个可靠且经过测试的框架进行管理，业务流程逻辑是完全独立的，而不是在我们的整个微服务中(通常会发生)。 带齿轮的盒子是“服务任务”，这意味着它们代表Zeebe外部执行的代码段。 流程引擎(Zeebe)负责执行指令，并将在指定的地方调用我们编写的代码。 在今天的示例中， 服务任务 (带齿轮的小盒子)将作为Lambda函数实现。 BPMN(业务流程建模表示法)还提供了许多其他强大的功能，上图只是其中的几个。

We need a way for Zeebe to trigger the Lambda functions because Lambda functions by design are dormant until you trigger them. We will use the code provided by the zeebe-lambda-worker to create a link between the orchestrator (Zeebe) and the business logic (Lamba). The worker will listen for events on the Zeebe broker, pick up work tasks as they come in, and forward the payload to Lambda. The original post covers how all this works. Here is what I’m adding

我们需要让Zeebe触发Lambda函数的方法，因为设计上的Lambda函数在您触发它们之前一直处于Hibernate状态。 我们将使用zeebe-lambda-worker提供的代码在业务流程协调员(Zeebe)和业务逻辑(Lamba)之间创建链接。 该工作人员将在Zeebe代理上侦听事件，在事件进入时接手工作，并将有效负载转发给Lambda。 原始帖子介绍了所有这些工作原理。 这是我要添加的

• How to spin up an ECS cluster and run the zeebe-lambda-worker on it
  如何启动ECS集群并在其上运行zeebe-lambda-worker
• How to use an IAM role for authorization instead of AWS credentials
  如何使用IAM角色而非AWS凭证进行授权

Here are the steps involved in this walkthrough:

这是本演练中涉及的步骤：

1. Get set up with Camunda Cloud

   使用Camunda Cloud进行设置

• Sign up for a Free Camunda Cloud trial account
  注册一个免费的Camunda Cloud试用帐户
• Launch a Zeebe instance
  启动Zeebe实例

2. Use the Serverless Framework to deploy several Lambda functions and an IAM role

2. 使用无服务器框架部署多个Lambda函数和一个IAM角色

3. Get set up on ECS

3. 设置ECS

• Create an ECS cluster using a setup shell script
  使用设置外壳脚本创建ECS集群
• Deploy the zeebe-lambda-worker to ECS
  将zeebe-lambda-worker部署到ECS

4. Test that everything works end to end

4. 测试一切正常运行

注册Camunda Cloud (Sign up for Camunda Cloud)

Camunda and Zeebe are the same company. Zeebe is their newer product and Camunda Cloud is their managed cloud offering which happens to run on the newer Zeebe engine. It’s confusing, I know, probably has something to do with marketing as Camunda is an established and fairly well known brand.

Camunda和Zeebe是同一家公司。 Zeebe是他们的较新产品，Camunda Cloud是他们的托管云产品，恰好在更新的Zeebe引擎上运行。 我知道，这令人困惑，因为Camunda是一个知名的知名品牌，它与市场营销有关。

Getting set up with a development Zeebe instance is fairly straightforward. This post from the zeebe blog walks you through the steps. The whole thing is a good read but you can skip everything after the create a client part if you just want to get up and running. We’ll need the broker address information and the client information later. Once we’re set up with a free Zeebe instance we can deploy the Serverless framework project.

使用开发Zeebe实例进行设置非常简单。 zeebe博客中的这篇文章将引导您完成这些步骤。 整个过程是一本不错的书，但是如果您只想启动并运行， 可以在创建客户端部件之后跳过所有内容。 稍后我们将需要代理地址信息和客户信息。 一旦设置了免费的Zeebe实例，便可以部署无服务器框架项目。

部署无服务器项目 (Deploy the serverless project)

prerequisite: You need to have an AWS account + IAM user set up with admin privileges. This is a great guide if you need help.

前提条件：您需要使用管理员权限设置一个AWS账户+ IAM用户。 如果您需要帮助，这是一个很好的指南 。

clone the repo https://github.com/bmccann36/trip-booking-saga-serverless

克隆仓库https://github.com/bmccann36/trip-booking-saga-serverless

cd into this directory trip-booking-saga-serverless/functions/aws

cd进入该目录trip-booking-saga-serverless/functions/aws

In the serverless.yml file update the region property If you’d like to deploy to a region other than us-east-1.

如果要部署到us-east-1以外的其他区域，请在serverless.yml文件中更新region属性。

I have made barely any changes to this from the original original forked repository. I removed the http event triggers from the functions since we won’t need them for this. This will make the project deploy and tear down faster since there are no API gateway resources. I also defined an IAM role that will give our zeebe-lambda-worker (which we have yet to deploy) permission to invoke the Lambda functions.

我几乎没有对原始的原始分叉存储库进行任何更改。 我从函数中删除了http事件触发器，因为我们不需要它们。 由于没有API网关资源，这将使项目的部署和拆除速度更快。 我还定义了一个IAM角色，该角色将为我们的zeebe-lambda-worker(我们尚未部署)授予调用Lambda函数的权限。

The policy portion of the role we’ll give to the ECS lambda-worker

我们将给ECS lambda-worker角色的政策部分

Run the command sls deploy -v to deploy the Lambda functions (the -v gives you verbose output).

运行命令sls deploy -v部署Lambda函数(-v提供详细输出)。

将Lambda + Zeebe与ECS连接 (Connecting Lambda + Zeebe with ECS)

The last step to get our demo application working end to end is to deploy the zeebe-lambda-worker. The source code can be found here. I also forked this from Zeebe and made a small change which was to add support for IAM role authorization. Since our ECS task will assume a role that gives it permission to invoke the Lambda functions it needs to, we do not need to supply AWS credentials to the worker code. I added an if / else statement so that a role will be used if no AWS credentials are supplied. This works because the AWS sdk sources credentials hierarchically. The role referred to in this snippet is the one we created earlier when we deployed the Serverless framework project.

使我们的演示应用程序端到端工作的最后一步是部署zeebe-lambda-worker。 源代码可以在这里找到。 我还从Zeebe那里得到了分叉，并做了一个小的更改，以增加对IAM角色授权的支持。 由于我们的ECS任务将承担一个赋予其调用所需Lambda函数权限的角色，因此我们无需向工作人员代码提供AWS凭证。 我添加了一条if / else语句，以便在没有提供AWS凭证的情况下使用角色。 之所以可行，是因为AWS sdk 分层地提供凭证 。 此片段中提到的角色是我们先前在部署无服务器框架项目时创建的角色。

If you prefer to use accessKey/secretKey make sure those keys are not the same keys you use as an admin user to work with AWS. You should create a role or user with credentials that allow the least permission possible. In this case we only need permission to invoke a few specific Lambda functions.

如果您更喜欢使用accessKey / secretKey，请确保这些密钥与您用作管理员用户使用AWS的密钥不同。 您应该使用允许尽可能少的权限的凭据来创建角色或用户。 在这种情况下，我们只需要允许调用一些特定的Lambda函数即可。

I have already packaged the zeebe-worker as a Docker image and pushed to a public repo. The ECS setup which we will get to in a moment will pull this image. If you want you can modify the worker source code and/or build the image yourself and publish to your own repository. You’ll need to package it as a .jar first as it is Java code.

我已经将zeebe-worker打包为Docker映像并推送到公共仓库。 稍后我们将介绍的ECS设置将提取此图像。 如果需要，您可以修改工作程序源代码和/或自己构建映像并将其发布到自己的存储库中。 您首先需要将其打包为.jar，因为它是Java代码。

Prerequisite: ecs-cli installed on your machine. You can get it with aws binary or homebrew

先决条件：您的计算机上已安装ecs-cli 。 您可以使用aws二进制或自制软件获取它

If you run into issues at any point look at the AWS walkthrough for reference. I mostly followed the steps outlined there to do this with a few minor modifications. I have chosen to use a launch type of EC2.

如果您在任何时候遇到问题，请查看AWS演练以供参考。 我主要按照此处概述的步骤进行了一些较小的修改。 我选择使用EC2的启动类型。

填写一些配置 (Fill in some configuration)

To make it easy to configure your ECS cluster I’ve included some configuration template files, all of which have the word “SAMPLE” in the file name. To use them, cd into zeebe-event-adapter/ in the trip-booking-saga repo. Then copy all the files with the word SAMPLE in front and save them with “SAMPLE” removed from the file name. This will cause them to be ignored by git. I’ve configured the .gitignore to ignore these files so that you or I don’t accidentally commit sensitive credentials to source control. Next, populate the values in each file as outlined below.

为了简化配置ECS群集的过程，我提供了一些配置模板文件，所有文件名中都带有单词“ SAMPLE”。 要使用它们， zeebe-event-adapter/ cd放入trip-booking-saga回购中的zeebe-event-adapter/中。 然后复制前面带有单词SAMPLE的所有文件，并将其保存为从文件名中删除的“ SAMPLE”。 这将导致它们被git忽略。 我已将.gitignore配置为忽略这些文件，以便您或我不小心将敏感凭据提交给源代码管理。 接下来，如下所述填充每个文件中的值。

(file) aws.env 
AWS_REGION=<FILL IN>---
(file) Camunda.env
ZEEBE_CLIENT_CLOUD_CLUSTERID=<FILL IN>
ZEEBE_CLIENT_CLOUD_CLIENTID=<FILL IN>
ZEEBE_CLIENT_CLOUD_CLIENTSECRET=<FILL IN>--- 
(file) ecs-params.yml
...
# modify this line 
task_role_arn: arn:aws:iam::<YOUR AWS ACCOUNT ID>:role/zeebe-lambda-worker-role
...

The values we supply in Camunda.env will be used by the zeebe-lambda-worker to connect to the Camunda Cloud Zeebe instance we created in the first step.

zeebe-lambda-worker将使用我们在Camunda.env中提供的值连接到我们在第一步中创建的Camunda Cloud Zeebe实例。

Once we’ve filled in the values in each of these files we’re ready to deploy. I’ve combined all the steps into one bash script so everything can be deployed with one command. Just run the setupEcs.sh script and supply your region as the only argument.

一旦在每个文件中填写了值，就可以部署了。 我将所有步骤组合到一个bash脚本中，因此可以使用一个命令来部署所有内容。 只需运行setupEcs.sh脚本并提供您的区域作为唯一参数。

i.e. bash setupEcs.sh us-east-1

即bash setupEcs.sh us-east-1

If you run into problems at any step you may want to try running the commands in the script one by one manually, or refer back to the AWS tutorial.

如果您在任何步骤遇到问题，都可能希望尝试手动一步一步地运行脚本中的命令，或者参考AWS教程。

# note you may get this warning INFO[0010] (service aws) was unable to place a task because no container instance met all of its requirements.Don’t get impatient and kill the process, this just means your infra isn’t quite ready yet, it should resolve on its own.

＃注意，您可能会收到此警告，提示INFO[0010] (service aws) was unable to place a task because no container instance met all of its requirements. 不要不耐烦地杀死进程，这只是意味着您的基础设施还没有准备好，它应该自行解决。

If you are used to working with Kubernetes the ECS terms will be confusing. An ECS service is not the same as K8s service at all and has nothing to do with networking. Instead in ECS a service just defines that you want to keep a “task” (which is basically a container with some config attached) running. It’s the same idea as a service you’d set up on a Linux server.

如果您习惯使用Kubernetes，则ECS术语会令人困惑。 ECS服务与K8s服务完全不同，并且与网络无关。 相反，在ECS中，服务仅定义您要保持“任务”(它基本上是带有某些配置的容器)运行。 这与您在Linux服务器上设置的服务的想法相同。

测试它是否有效 (Test that it works)

• *disclaimer The main purpose of this post is to show how to easily deploy the zeebe-lamba-connector to work with Lambda functions so I won’t cover Zeebe usage much. If you want to learn more about Zeebe check out Bernd Rücker’s posts or try the zeebe quickstart.

  *免责声明这篇文章的主要目的是演示如何轻松部署zeebe-lamba-connector以与Lambda函数一起使用，因此我不会过多介绍Zeebe的用法。 如果您想了解更多关于Zeebe退房贝恩德·吕克尔的帖子或尝试zeebe快速入门 。

Probably the simplest way to deploy and start the workflow is with the Zeebe Modeler tool which you can download here . You can also use the command line tool zbctl. Once you have the modeler downloaded, just open the trip-booking.bpmn file (in zeebe/aws directory) in the zeebe modeler. Use the upload and play buttons to upload the workflow to the cluster and start an instance of the process.

部署和启动工作流的最简单方法就是使用Zeebe Modeler工具，您可以在此处下载该工具。 您也可以使用命令行工具zbctl 。 下载建模器后，只需打开zeebe建模器中的trip-booking.bpmn文件(在zeebe / aws目录中)。 使用上载和播放按钮将工作流程上载到集群并启动流程实例。

If the process instance worked successfully end to end you will see the completed instance in the operate console.

如果流程实例首尾成功运行，您将在操作控制台中看到完成的实例。

If we navigate to ECS, select the service we’ve deployed and click on logs, we can see the successful results from the Lambda functions being passed back and forth through the zeebe-lambda-worker.

如果导航到ECS，选择我们已部署的服务，然后单击日志，则可以看到通过zeebe-lambda-worker来回传递的Lambda函数的成功结果。

Also if we navigate to the Lambda console, select one of the Lambdas involved in our workflow then select monitoring → view logs in cloudWatch, we can see the logs of the Lambda function itself.

另外，如果我们导航到Lambda控制台，请选择工作流程中涉及的Lambda之一，然后选择“监视”→在cloudWatch中查看日志，我们可以看到Lambda函数本身的日志。

Cleaning up ECS resources when you don’t want them anymore

当您不再需要ECS资源时清理它们

take down the service ecs-cli compose service rm — cluster-config zb-cfg

取消服务ecs-cli compose service rm — cluster-config zb-cfg

remove the whole cluster ecs-cli down — force — cluster-config zb-cfg

删除整个集群ecs-cli down — force — cluster-config zb-cfg

总结思想 (Closing thoughts)

I am curious to hear what others think about pairing Zeebe with Lambda. Does the combination make sense? What would you do differently? Also for anyone who uses or has used step functions I am curious to hear how this solution compares.

我很想知道其他人对将Zeebe和Lambda配对的看法。 组合有意义吗？ 你将做点什么不同的？ 对于使用或使用过步进功能的任何人，我都很想知道此解决方案的比较情况。

I had not used ECS much before this and I was pleasantly surprised by how easy it was to provision and use. It seems like they’ve added a lot of features since the last time I looked at it to try to keep pace with Kubernetes. I also like how they use a docker-compose file and docker like commands so you can work with your containers in the cloud more or less the same way you would locally. My one big concern and hesitation on getting good at ECS is that its clearly not the industry standard. Practically as engineers it just makes way more sense to use what everyone else is using because thats what everyone already knows.

在此之前，我没有使用ECS太多，而配置和使用起来如此容易，让我感到惊喜。 自从我上次查看它以来，他们似乎已经添加了许多功能以尝试与Kubernetes保持同步。 我还喜欢它们如何使用docker-compose文件和docker like命令，因此您可以或多或少以与本地相同的方式使用云中的容器。 我对擅长ECS的最大担忧和犹豫是，这显然不是行业标准。 实际上，作为工程师，使用所有人都在使用的东西才有意义，因为那是每个人都已经知道的。

I think Zeebe is a very promising new product, I also think they have their work cut out for them because there are a lot of competitors in this space. Some popular tools that fill the same Niche are Netflix’s Conductor and AWS’s own Step Functions. They face competition not just from competitor frameworks but also homespun orchestration solutions using event buses and other lower level orchestration/choreography tools. I think that a lot of orgs and developers don’t even realize that there is a tool that can help them glue together or orchestrate microservices. In a lot of cases people tend to write this code themselves. In many cases it is a conscious decision because they feel that adding in another Library just creates additional complexity, makes their app harder to understand for new developers, or perhaps even limits what they can do. I totally get that concern as I’ve had these problems myself. For my personal development and collaboration style I think Zeebe makes a lot of sense for a broad range of tech problems.

我认为Zeebe是一个非常有前途的新产品，我也认为他们的工作很辛苦，因为在这个领域有很多竞争对手。 填充相同细分市场的一些流行工具是Netflix的Conductor和AWS自己的Step Functions。 他们不仅面临来自竞争对手框架的竞争，还面临着使用事件总线和其他较低级别的编排/编排工具的本地编排解决方案的竞争。 我认为许多组织和开发人员甚至都没有意识到有一种工具可以帮助他们粘合或协调微服务。 在很多情况下，人们倾向于自己编写此代码。 在许多情况下，这是一个有意识的决定，因为他们觉得在另一个Library中添加只会增加额外的复杂性，使新开发人员难以理解其应用，甚至可能限制了他们的工作。 因为我自己遇到了这些问题，所以我完全感到担忧。 对于我的个人发展和协作风格，我认为Zeebe对于广泛的技术问题很有用。

I hear about a lot of people reaching for Kafka to achieve EDA. I have nothing against Kafka and I think it is a technically impressive piece of engineering. An experienced dev can probably get up and running with it in a day but they definitely won’t be using it optimally or even getting benefits from it for a long time because it is so complicated with tons of use case specific config required. Kafka is more low level than Zeebe and therefore applies to a broader range of use cases. However I think that if you are just using Kafka as an event bus to drive a collection of choreographed microservices, Zeebe may offer a simpler and actually more powerful solution. With Kafka things like event replay, retry strategies, and monitoring solutions at a business process level, are capabilities we need to build ourselves. Zeebe is appealing to me because they’ve found a way to generalize these needs. This allows us as developers to simply use their API to endow our apps with these advanced features and ultimately solve a business problem.

我听说有很多人希望获得Kafka来实现EDA。 我对卡夫卡一无所知，我认为这是一项技术上令人印象深刻的工程。 一个有经验的开发人员可能可以在一天内启动并运行它，但是他们肯定不会最佳使用它，甚至很长时间都不会从中获得好处，因为它非常复杂，需要大量的用例特定配置。 Kafka比Zeebe的等级低，因此适用于更广泛的用例。 但是我认为，如果您只是将Kafka用作事件总线来驱动经过精心设计的微服务集合，则Zeebe可能会提供一个更简单，实际上更强大的解决方案。 使用Kafka，诸如事件重播，重试策略以及在业务流程级别监视解决方案之类的东西，是我们构建自身所需的能力。 Zeebe之所以吸引我，是因为他们找到了一种概括这些需求的方法。 这使我们作为开发人员可以简单地使用其API为我们的应用程序赋予这些高级功能，并最终解决业务问题。

翻译自: https://medium.com/swlh/cleaner-microservice-orchestration-with-zeebe-lambda-b8ea673305cc

c ++ lambda