lambda函数实现循环_lambda函数和cloudwatch事件实现自动化

lambda函数实现循环

sam init HelloWorld

sam init HelloWorld

Lambdas are great. It lets you run your code without having to worry about provisioning or managing servers. You pay only for the compute time you consume. With Lambda, you can run code for various types of applications or backend services, all with zero administration. You just upload your code and Lambda takes care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app.

大号 ambdas是巨大的。 它使您可以运行代码,而不必担心配置或管理服务器。 您只需为您消耗的计算时间付费。 借助Lambda,您可以为各种类型的应用程序或后端服务运行代码,而所有这些都将实现零管理。 您只需上传代码,Lambda便可以处理以高可用性运行和扩展代码所需的一切。 您可以将代码设置为自动从其他AWS服务触发,或直接从任何Web或移动应用程序调用它。

TLDR; ♂‍♂️ (TLDR; 🤷‍♂️)

I created a simple workflow using AWS Lambda, CloudWatch Events, SSM Parameter Store and SNS that sends email notification when certain criteria is met on a predefined recurring schedule.

我使用AWS Lambda,CloudWatch Events,SSM参数存储和SNS创建了一个简单的工作流,当在预定义的定期计划上满足特定条件时,该电子邮件发送电子邮件通知。

让我们开始吧 (Let’s start 🚀)

Recently, I was preparing for the AWS Developer Associate certification. As part of the preparation, I was reading white-papers, articles, watching tutorials videos and tinkering with the AWS console to get some hands-on experience. I have built a chrome extension that I monitor its download count and check for any reviews or support questions once in a while. I thought instead on me visiting the site, it would be good if I can get notified wherever the download count changes. As a sample application, I decided to build a simple workflow using Lambda and CloudWatch Events that will send out an email notification whenever the download count changes. This is a very basic scenario but using Lambda you can build complex applications.

最近,我正在准备AWS开发人员助理认证。 作为准备工作的一部分,我正在阅读白皮书,文章,观看教程视频以及对AWS控制台进行修补,以获得一些动手经验。 我构建了一个chrome扩展程序,可以监视其下载数量,并偶尔检查是否有任何评论或支持问题。 我本来以为我可以访问该站点,所以无论下载计数如何变化,都可以得到通知,那将是很好的。 作为一个示例应用程序,我决定使用Lambda和CloudWatch Events构建一个简单的工作流,只要下载次数发生变化,它们就会发送电子邮件通知。 这是一个非常基本的方案,但是使用Lambda可以构建复杂的应用程序。

无服务器应用程序模型(SAM) (Serverless Application Model (SAM))

I started with the SAM, which lets you quickly get started with a Serverless project available in various languages. As part of the boilerplate template, you get an API hosted in API Gateway, a Lambda function with basic logging in CloudWatch.

我从SAM开始,它使您可以快速入门各种语言的无服务器项目。 作为样板模板的一部分,您将获得托管在API Gateway中的API,这是一个Lambda函数,在CloudWatch中具有基本日志记录。

I choose dotnet core 3.1 as the runtime but it can be built using any supported language. I had a rough idea about how to build the application. Start by invoking the Lambda function, that will make a request to the extension details page in Chrome Web Store, using regex pattern matching, extract the string or keyword from the response and compare it with previously saved value for the extension downloads. When the saved downloads do not match the current downloads from the response call, send a notification 🔔.

我选择dotnet core 3.1作为运行时,但是可以使用任何受支持的语言来构建它。 我对如何构建应用程序有一个粗略的想法。 首先调用Lambda函数,该函数将使用正则表达式模式匹配来请求Chrome Web Store中的扩展程序详细信息页面,从响应中提取字符串或关键字,并将其与先前保存的扩展程序下载值进行比较。 如果保存的下载与响应调用中的当前下载不匹配,请发送通知🔔。

Let’s break down the rough idea in a bit more detail. Where to save the download count result from the api call and how to send notifications. Also how to trigger this workflow automatically based on a recurring schedule.

让我们更详细地分解一下粗略的想法。 api调用将下载计数结果保存到何处以及如何发送通知。 以及如何根据定期计划自动触发此工作流程。

  1. For saving the download count, I decided to use the Secure Systems Manager (SSM) Parameter Store. It’s free, you could save up to 10,000 parameters in the free standard tier and can have different versions of your parameter.

    为了节省下载次数,我决定使用安全系统管理器(SSM)参数存储 。 它是免费的,您可以在免费的标准层中最多保存10,000个参数,并且可以使用不同版本的参数。

  2. For sending notifications, I chose Simple Notification Service (SNS) with a subscription with Email ✉️ as the protocol.

    为了发送通知,我选择了简单通知服务(SNS),并已订阅Email✉️作为协议。

  3. For automation of the workflow, I used CloudWatch Events 🕛 trigger with an SNS topic as the target. CloudWatch Events delivers a near real-time stream of system events that describe changes in AWS resources. CloudWatch Events becomes aware of operational changes as they occur. Using simple rules, you can quickly set up a trigger which can match events and route them to one or more target functions.

    为了实现工作流程的自动化,我使用了以SNS主题为目标的CloudWatch Events触发器。 CloudWatch Events提供了几乎实时的系统事件流,这些流描述了AWS资源的变化。 CloudWatch Events会在发生操作更改时意识到它们。 使用简单的规则,您可以快速设置一个触发器,该触发器可以匹配事件并将其路由到一个或多个目标函数。

Below is a high-level diagram that shows various steps in the workflow.

下面是显示工作流程中各个步骤的高级示意图。

Image for post
Created using Creately
使用Creately创建

让我们开始编码👨‍💻 (Let’s get coding 👨‍💻)

Enough of the talk, let’s look at some code. Below is the boilerplate code generated by the SAM cli using .NET Core template. The code defined in FunctionHandler method calls Amazon’s CheckIP website, removes the empty line from the response and simply returns a hello world message with the IP address.

足够多的讨论,让我们看一些代码。 以下是SAM cli使用.NET Core模板生成的样板代码。 FunctionHandler方法中定义的代码将调用Amazon的CheckIP网站,从响应中删除空行,并仅返回具有IP地址的hello world消息。

using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using System.Net.Http;
using Newtonsoft.Json;


using Amazon.Lambda.Core;
using Amazon.Lambda.APIGatewayEvents;


// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.Json.JsonSerializer))]


namespace HelloWorld
{
    public class Function
    {
        private static readonly HttpClient client = new HttpClient();


        private static async Task<string> GetCallingIP()
        {
            client.DefaultRequestHeaders.Accept.Clear();
            client.DefaultRequestHeaders.Add("User-Agent", "AWS Lambda .Net Client");


            var msg = await client.GetStringAsync("http://checkip.amazonaws.com/").ConfigureAwait(continueOnCapturedContext:false);


            return msg.Replace("\n","");
        }


        public async Task<APIGatewayProxyResponse> FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context)
        {
            var location = await GetCallingIP();
            var body = new Dictionary<string, string>
            {
                { "message", "hello world" },
                { "location", location }
            };


            return new APIGatewayProxyResponse
            {
                Body = JsonConvert.SerializeObject(body),
                StatusCode = 200,
                Headers = new Dictionary<string, string> { { "Content-Type", "application/json" } }
            };
        }
    }
}

After running sam build command and then sam local invoke (requires Docker 🐳), we get the results from the Lambda function execution running locally on your machine.

运行sam build命令,然后执行sam local invoke (需要Docker, )后 ,我们从在您的计算机上本地运行的Lambda函数执行中获取结果。

Image for post
sam build output
山姆建立输出
Image for post
sam local invoke output — mock IP address
sam本地调用输出-模拟IP地址

We can achieve the same outcome by starting the API locally by running sam local start-api (again, requires Docker 🐳) command and then executing curl localhost:3000/hello

我们可以通过运行sam local start-api (再次需要Docker command)命令在本地启动API ,然后执行curl localhost:3000 / hello来实现相同的结果

Image for post
sam local start-api and invocation output for boilerplate SAM code
用于样板SAM代码的sam本地start-api和调用输出

This works great, now let’s add a method that makes a GET request to the extension page and extracts the download count using a regular expression.

效果很好,现在让我们添加一个方法,该方法向扩展页面发出GET请求,并使用正则表达式提取下载次数。

public static async Task<string> GetExtensionDownloadCount()
{
    var websiteUrl = Environment.GetEnvironmentVariable("WEBSITE_URL");
    var websiteContent = await client.GetStringAsync(websiteUrl)
        .ConfigureAwait(continueOnCapturedContext: false);


    Regex pattern = new Regex(@"UserDownloads:(?<downloads>\d+)");
    Match match = pattern.Match(websiteContent);


    return match.Groups["downloads"].Value;
}
Image for post
Chrome Extension page HTML content
Chrome扩展程序页面HTML内容
Image for post
sam local start-api and invocation output
sam本地start-api和调用输出

Finally, let’s save the download count value is SSM Parameter Store. To be able to use the AWS SSM SDK, let’s add the package to the project by running

最后,让我们将下载计数值保存为SSM Parameter Store 。 为了能够使用AWS SSM SDK,让我们通过运行将包添加到项目中

dotnet add HelloWorld.csproj package AWSSDK.SimpleSystemsManagement
public static async Task UpdateParameter(string value, string parameterKey)
{
    var request = new PutParameterRequest()
    {
        Name = parameterKey,
        Value = value,
        Type = ParameterType.String,
        Overwrite = true,
    };


    try
    {
        using (var client = new AmazonSimpleSystemsManagementClient())
        {
            await client.PutParameterAsync(request);


            Console.WriteLine($"Parameter {parameterKey} updated successfully");
        }
    }
    catch (Exception ex)
    {
        Console.Error.WriteLine($"Failed to put parameter {parameterKey}, Error message: {ex.Message}");
    }
}

If we start the api locally and hit the endpoint, we will get The security token included in the request is invalid error.

如果我们在本地启动api并命中端点,则将获得请求中包含的安全令牌无效错误。

Image for post
sam local start-api invocation output with security token error
带有安全令牌错误的sam本地start-api调用输出

As a workaround suggested in this thread https://github.com/aws/aws-sam-cli/issues/2135, I have set an AWS_SESSION_TOKEN environment variable to an empty string.

作为此线程https://github.com/aws/aws-sam-cli/issues/2135中建议的解决方法,我已将AWS_SESSION_TOKEN环境变量设置为空字符串。

Next, let’s add a method that can retrieve the saved parameter from SSM Parameter Store using the key.

接下来,让我们添加一个方法,该方法可以使用键从SSM参数存储中检索保存的参数。

public static async Task<string> GetParameterValue(string parameterKey)
{
    var request = new GetParameterRequest()
    {
        Name = parameterKey
    };


    var extensionDownloadCount = String.Empty;


    try
    {
        using (var client = new AmazonSimpleSystemsManagementClient())
        {
            var response = await client.GetParameterAsync(request);
            extensionDownloadCount = response.Parameter.Value;


            Console.WriteLine($"Parameter {request.Name} value is: {extensionDownloadCount}");
        }
    }
    catch (Exception ex)
    {
        Console.Error.WriteLine($"Error occurred: {ex.Message}");
    }


    return extensionDownloadCount;
}

We want to compare the values of the current extension download count against the saved value in the parameter store. If the values are not the same, then either the download count has gone up or down and as a result of this, we would want to send a notification. For sending a notification, we will publish a message to SNS topic which will have an email subscription subscribed to the topic. Once a message is published to the SNS topic, an email will be sent to the defined email address. Let’s install the AWS SimpleNotificationService package to the project by running

我们想将当前扩展下载计数的值与参数存储中保存的值进行比较。 如果值不相同,则下载计数增加或减少,因此,我们希望发送通知。 为了发送通知,我们将向SNS主题发布一条消息,该消息将通过电子邮件订阅该主题。 将消息发布到SNS主题后,会将电子邮件发送到已定义的电子邮件地址。 让我们通过运行将AWS SimpleNotificationService软件包安装到项目中

dotnet add HelloWorld.csproj package \ AWSSDK.SimpleNotificationService
public async Task<APIGatewayProxyResponse> FunctionHandler(APIGatewayProxyRequest apigProxyEvent, ILambdaContext context)
{
    var currentExtensionDownloadCount = await GetExtensionDownloadCount();
    var ssmParameterKey = Environment.GetEnvironmentVariable("SSM_PARAMETER_KEY");
    var savedDownloadCount = await GetParameterValue(ssmParameterKey);


    if (currentExtensionDownloadCount != savedDownloadCount)
    {
        await UpdateParameter(currentExtensionDownloadCount, ssmParameterKey);


        string message = $"Extension download count, last download count: {savedDownloadCount}, " +
                   $"new download count: {currentExtensionDownloadCount}, updated at: {DateTime.UtcNow.ToLongDateString()}";


        var request = new PublishRequest
        {
            Message = message,
            TopicArn = "arn:aws:sns:ap-southeast-2:aws-account-id:topic-name",
        };


        try
        {
            using (var client = new AmazonSimpleNotificationServiceClient())
            {
                var topicResponse = await client.PublishAsync(request);


                Console.WriteLine($"Message published, id: {topicResponse.MessageId}");
            }
        }
        catch (Exception ex)
        {
            Console.Error.WriteLine($"Unable to publish message: {ex.Message}");
        }
    }


    var body = new Dictionary<string, string>
    {
        { "Extension saved download count", savedDownloadCount },
        { "Extension new download count", currentExtensionDownloadCount }
    };


    return new APIGatewayProxyResponse
    {
        Body = JsonConvert.SerializeObject(body),
        StatusCode = 200,
        Headers = new Dictionary<string, string> { { "Content-Type", "application/json" } }
    };
}

For now, I will create the SNS topic and email subscription manually, however, this can be created using CloudFormation. Let's start the api locally by running sam local start-api command and issue a curl request against the endpoint.

现在,我将手动创建SNS主题和电子邮件订阅,但是,可以使用CloudFormation创建它。 让我们通过运行sam local start-api命令在本地启动api ,并对端点发出curl请求。

Image for post

Once the code finishes executing, we see the output response of the saved and the download count from the extension page. Happy with the outcome, now let’s deploy 🚀 it to AWS using sam deploy — —guided cli command.

代码执行完后,我们将从扩展页面上看到已保存的输出响应和下载计数。 对结果感到满意,现在让我们使用sam deploy —引导的 cli命令将它部署到AWS。

Image for post
Image for post
Image for post
Image for post
sam deploy — guided output
sam deploy —指导输出

If we grab the URL from the deployed stack output and hit the endpoint, we can see it is not working quite as expected. This is because the LambdaExecutionRole does not have permission to get or put SSM Parameters nor it has permission to publish to our SNS Topic.

如果我们从已部署的堆栈输出中获取URL并命中端点,则可以看到它没有按预期工作。 这是因为LambdaExecutionRole没有获得或放置SSM参数的权限,也没有发布到我们的SNS Topic的权限。

Image for post
API Gateway Endpoint invocation
API网关端点调用

After granting the Lambda function proper permission, the lambda function now runs as expected and we can see the logs in CloudWatch. The email message is successfully received as well.

授予Lambda函数适当的权限后,lambda函数现在将按预期运行,我们可以在CloudWatch中查看日志。 电子邮件也已成功接收。

Image for post
API Gateway Endpoint invocation
API网关端点调用
Image for post
Email from SNS
来自SNS的电子邮件
Image for post

CloudWatch Events规则触发器 (CloudWatch Events rule trigger)

For the automation part, we can set up a recurring schedule in CloudWatch Events that will invoke the lambda function. We can either set up a fixed rate or cron syntax schedule and specify our CheckContent-HelloWorldFunction Lambda as the target invocation. Specifying a cron expression 0 12 * * ? * will invoke the Lambda function every day at noon GMT.

对于自动化部分,我们可以在CloudWatch Events中设置一个重复计划,以调用lambda函数。 我们可以设置固定费率或cron语法计划,然后将CheckContent-HelloWorldFunction Lambda指定为目标调用。 指定cron表达式0 12 * * ? * 0 12 * * ? *将在格林尼治标准时间每天中午调用Lambda函数。

Image for post
CloudWatch Events setup
CloudWatch Events设置

结论 (Conclusion)

This is a simple workflow but obviously very implementations like data extraction, image manipulation and a lot more can be achieved using Lambda functions. Fork or clone aws-sam-dotnet-core repo available on my GitHub account. 🙏

这是一个简单的工作流程,但显然可以使用Lambda函数实现诸如数据提取,图像处理等更多实现。 在我的GitHub帐户上可以使用fork或克隆aws-sam-dotnet-core仓库。 🙏

翻译自: https://medium.com/@chandankkrr/automation-with-lambda-function-and-cloudwatch-events-b7a525b7bd03

lambda函数实现循环

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值