aws lambda测试_我如何为AWS Lambda函数编写有意义的测试

aws lambda测试

苛刻的真相 (A Harsh Truth)

If you are going to write meaningless unit tests that are more likely to mask errors than expose them, you are better off skipping the exercise altogether.

如果您要编写无意义的单元测试，而这些单元测试更可能掩盖错误而不是暴露错误，那么最好完全跳过该练习。

There, I said it.

在那里，我说了。

Your time is precious and could be spent on better things than achieving a hollow coverage percentage.

您的时间很宝贵，可以花在比达成空缺的覆盖率更好的事情上。

Effective testing of code has long been a challenging problem in programming, and newer tools like AWS Lambda seem to bring out the worst in developers when it comes to writing tests.

有效的代码测试长期以来一直是编程中的挑战性问题，而诸如AWS Lambda之类的新工具似乎在编写测试方面给开发人员带来了最糟糕的情况。

I think the main reason for this is that it’s more difficult (or at least less intuitive) to mirror the Lambda production environment locally. And as a result, some developers abstain from testing entirely.

我认为这样做的主要原因是，在本地镜像Lambda生产环境更加困难(或至少不那么直观)。 结果，一些开发人员完全放弃了测试。

I know because I’ve done it myself, even for projects in production. Instead, testing was done integration-style only after code was already deployed to the cloud.

我知道，因为我自己做过，即使是在生产项目中也是如此。 相反，仅在代码已部署到云中之后才进行集成样式的测试。

This is extremely manual and wastes time in the long run.

从长远来看，这是非常手动的操作，浪费时间。

Another approach I’ve seen results in tests that look something like this:

我见过的另一种方法导致测试看起来像这样：

import pytest
from lambda_function import *


def test_lambda_handler():


    try:
        response = lambda_handler({'Records': []}, {})
    except Exception as e:
        print(e)


    assert True == True

This is the unmistakeable sign of an engineering team with a test coverage requirement but a lack of accountability. And no explanation is needed that the above is a no-no.

这是具有测试覆盖率要求但缺乏责任感的工程团队的必经之路。 并且不需要解释上述内容是否可以通过。

So, how do we go about transforming the sad test_lambda_function.py file above into something meaningful?

那么，如何将上述可悲的test_lambda_function.py文件转换为有意义的东西呢？

Before we can dive right into testing our Lambda code, there are a couple hurdles in the way. We’ll cover each of these individually and determine how to best handle them. Once dealt with, we are then free to test Lambdas to our heart’s content!

在我们继续深入测试Lambda代码之前，有两个障碍。 我们将逐一介绍这些内容，并确定如何最好地处理它们。 一经处理，我们便可以自由地测试Lambda了！

Note: I’ll be including small snippets of code throughout the article for clarity. But at the end there will be a full working code example to reference.

注意：为了清楚起见，我将在整篇文章中包含一些小代码段。 但最后将有完整的工作代码示例可供参考。

障碍1：Lambda触发事件 (Hurdle #1: The Lambda Trigger Event)

Every Lambda function gets invoked in response to a pre-defined trigger that passes specific event data into the default lambda_handler() method. And your first task for effectively testing a Lambda function is to create a realistic input event to test with.

每个Lambda函数都会被调用以响应预定义的触发器，该触发器将特定event数据传递到默认的lambda_handler()方法中。 有效测试Lambda函数的首要任务是创建一个实际的输入事件进行测试。

The format of this event depends on the type of trigger. As of the time of writing there are 16 distinct AWS services that can act as the invocation trigger for Lambda.

此事件的格式取决于触发器的类型。 在撰写本文时，有16种不同的AWS服务可以充当Lambda的调用触发器。

Below is a code snippet with several examples of inputs that I most commonly use:

下面是一个代码片段，其中包含一些我最常用的输入示例：

import pytest
from lambda_function import *


test_kinesis_event = {
    "Records": [
        {
            "kinesis": {
                base64.b64encode(json.dumps({'a': 'dog', 'b': 'cat'}))
            }
     }]}


test_s3_event = {
    "Records": [{
        "s3": {
            'bucket': {'name': 'test_bucket'},
            'object': {
                'key': 'example/s3/path/key'
            }
        }
    }]}


test_sqs_event = {
    "Records": [{
        "body": json.dumps({
            "user_id": "B9B3022F98Fjvjs83AB8/80C185D8", 
            "updated_timstamp": "2020-03-03T00:50:47"
        })
    }]}


def test_lambda_handler():


    try:
        response = lambda_handler(event=test_sqs_event, context={})
    except Exception as e:
        print(e)


    assert True == True

The full list of sample input events can be found in the AWS documentation. Alternatively, you can also print the event variable in your lambda_handler code after deploying and view the payload in CloudWatch Logs:

样本输入事件的完整列表可以在AWS文档中找到。 另外，您还可以在部署后在lambda_handler代码中打印事件变量，并在CloudWatch Logs中查看有效负载：

Lambda event triggered from Kinesis in CloudWatch Logs example | Image by author

CloudWatch Logs示例中从Kinesis触发的Lambda事件| 图片作者

Once you have that example, simply hardcode it in your test file as shown above and we’re off to a fantastic start!

有了该示例后，只需按上面所示将其硬编码到您的测试文件中，我们就可以开始了！

Next up…

接下来...

障碍2：AWS服务互动 (Hurdle #2: AWS Service Interactions)

Almost inevitably, a Lambda function interacts with other AWS services. Maybe you are writing data to a DynamoDB table. Or posting a message to an SNS topic. Or simply sending a metric to CloudWatch. Or a combination of all three!

Lambda函数几乎不可避免地与其他AWS服务进行交互。 也许您正在将数据写入DynamoDB表。 或将消息发布到SNS主题。 或者只是将指标发送到CloudWatch。 或这三者的结合！

When testing it is not a good idea to send data or alter actual AWS resources used in production. To get around this problem, one approach is to set up and later tear down separate test resources.

测试时，发送数据或更改生产中使用的实际AWS资源不是一个好主意。 为了解决这个问题，一种方法是设置并随后拆除单独的测试资源。

A cleaner approach though, is to mock interactions with AWS services. And since this is a common problem, a package has been developed to solve this specific problem. And what’s better is it does so in a super elegant way.

不过，更干净的方法是模拟与AWS服务的交互。 并且由于这是一个普遍的问题，因此已经开发出一种包装来解决该特定问题。 而且更好的是它以一种超级优雅的方式进行。

It’s name is moto (a portmanteau of mock & boto) and its elegance is derived from two main features:

它的名称是moto (模拟和boto的Portmanteau)，其优雅源自两个主要特征：

1. It patches and mocks boto clients in tests automatically.

   它会在测试中自动修补和模拟boto客户端。

2. It maintains the state of pseudo AWS resources.
   它维护伪AWS资源的状态。

What does this look like? All that’s needed is some decorator magic and a little set up!

看起来像什么？ 所需要的只是一些装饰魔术和一些设置！

Say we read data from S3 in our Lambda. Instead of creating and populating a test bucket in S3, we can use moto to create a fake S3 bucket — one that looks and behaves exactly like a real one — without actually touching AWS.

假设我们从Lambda中的S3中读取数据。 无需在S3中创建和填充测试存储桶，我们可以使用moto创建一个伪造的S3存储桶-一个外观和行为与真实存储桶完全相同的存储桶-无需实际接触AWS。

And the best part is we can do this using standard boto3 syntax, as seen in the example below when calling the create_bucket and put_object methods:

最好的部分是我们可以使用标准的boto3语法来做到这一点，如下面的示例在调用create_bucket和put_object方法时所示：

import boto3
import pytest
from moto import mock_s3


from lambda_function import *




@mock_s3
def test_lambda_handler():


    # set up test bucket
    s3_client = boto3.client('s3')
    test_bucket_name = 'test_bucket'
    test_data = b'col_1,col_2\n1,2\n3,4\n'


    s3_client.create_bucket(Bucket=test_bucket_name)    
    s3_client.put_object(Body=test_data, Bucket=test_bucket_name, Key=f'example/s3/path/key/test_data.csv')
    
    response = lambda_handler(event=test_event, context={})


    assert response['status'] == 'success'

Similarly, if we write data to DynamoDB, we could set up our test by creating a fake Dynamo table first:

同样，如果我们将数据写入DynamoDB，则可以通过首先创建一个伪造的Dynamo表来设置测试：

import boto3
import pytest
from moto import mock_dynamodb2
from lambda_function import *




@mock_dynamodb2
def test_lambda_handler():


    table_name = 'user_spending'
    dynamodb = boto3.resource('dynamodb', 'us-east-1')


    table = dynamodb.create_table(
        TableName=table_name,
        KeySchema=[{'AttributeName': 'user_id', 'KeyType': 'HASH'}],
        AttributeDefinitions=[{'AttributeName': 'user_id','AttributeType': 'S'}],
        ProvisionedThroughput={'ReadCapacityUnits': 5, 'WriteCapacityUnits': 5}
    )


    response = lambda_handler(event=kinesis_test_event, context={})


    table = dynamodb.Table(table_name)
    response = table.get_item(
        Key={
            'user_id': 'B9B3022F98Fjvjs83AB8a80C185D'
        }
    )
    
    item = response['Item']
    # {'user_id': 'B9B3022F98Fjvjs83AB8a80C185D', 'amount': Decimal('10'), 'updated_at': '2020-08-13T21:44:50.908455'}

It requires a bit of trust, but if the test passes, you can be confident your code will work in production, too.

它需要一点信任，但是如果测试通过，您可以确信您的代码也可以在生产中使用。

好的，但是moto并不能涵盖所有内容... (Okay, but not everything is covered by moto…)

Yes, it is true that moto doesn’t maintain parity with every AWS API. For example, if your Lambda function interacts with AWS Glue, odds are moto will leave you high and dry since it is only 5% implemented for the Glue service.

是的，的确，moto不会与每个 AWS API保持同等地位。 例如，如果您的Lambda函数与AWS Glue进行交互，那么由于Glue服务仅实现了5％， 因此moto将使您无所适从。

This is where we need to roll up our sleeves and do the dirty work of mocking calls ourselves by monkeypatching. This is true whether we’re talking about AWS-related calls or any external service your Lambda may touch, like when posting a message to Slack, for example.

在这里我们需要卷起袖子，并通过猴子修补做嘲笑自己的肮脏工作。 无论我们是在谈论与AWS相关的呼叫还是Lambda可能接触的任何外部服务，例如在向Slack发布消息时，都是如此。

Admittedly the terminology and concepts around this get dense, so it is best explained via an example. Let’s stick with AWS Glue and say we have a burning desire to list our account’s Glue crawlers with the following code:

不可否认，围绕此的术语和概念比较繁琐，因此最好通过一个示例进行解释。 让我们坚持使用AWS Glue，说我们迫切希望使用以下代码列出我们帐户的Glue搜寻器：

session = boto3.session.Session()
glue_client = session.client("glue", region_name='us-east-1')
glue_client.list_crawlers()['CrawlerNames']
# {“CrawlerNames”: [“crawler1”, "crawler2",...]}

If we don’t want the success or failure of our illustrious test to depend on the list_crawlers() response, we can hardcode a return value like so:

如果我们不希望杰出测试的成功或失败取决于list_crawlers()响应，则可以对返回值进行硬编码，如下所示：

import boto3
import pytest
from lambda_function import *


class MockBotoSession:
    class Session:


        def client(self, service_name, region_name):
            # create a flexible Client class for any boto3 client
            class Client:
                def list_crawlers(self):
                    # hardcode the list_crawlers response
                    return {'CrawlerNames': ['crawler_1', 'crawler_2',]}


            return Client()
          
          
def test_lambda_handler(monkeypatch):
  
  # patch boto session with a mock client
  monkeypatch.setattr(boto3, 'session', MockBotoSession)
  
  response = lambda_handler(event=test_event, context={})
  assert response == 'something'

By leveraging the setattr method of the pytest monkeypatch fixture, we allow glue client code in a lambda_handler to dynamically at runtime access the hardcoded list_clusters response from the MockBotoSession class.

通过利用setattr的pytest方法猴补丁夹具，我们允许在胶客户端代码lambda_handler在运行时动态存取硬编码list_clusters从响应MockBotoSession类。

What’s nice about this solution is that is it flexible enough to work for any boto client.

这种解决方案的优点是它足够灵活，可以适用于任何 boto客户端。

整理灯具 (Tidying Up with Fixtures)

We’ve already covered how to deal with event inputs and external dependencies in Lambda tests. Another tip I’d like to share involves the use of pytest fixtures to maintain an organized testing file.

我们已经介绍了如何在Lambda测试中处理事件输入和外部依赖关系。 我想分享的另一个技巧是使用pytest 固定装置来维护组织良好的测试文件。

The code examples thus far have shown set up code directly in the test_lambda_handler method itself. A better pattern, however, is to create a separate set_up function as a pytest fixture that gets passed into any test method that needs to use it.

到目前为止的代码示例已直接在test_lambda_handler方法本身中显示了设置代码。 但是，更好的模式是创建一个单独的set_up函数作为pytest固定装置，并将其传递给需要使用它的任何测试方法。

For the final code snippet, let’s show an example of this fixture structure using the @pytest.fixture decorator and combine everything covered:

对于最后的代码段，让我们使用@pytest.fixture装饰器展示此灯具结构的示例，并结合涵盖的所有内容：

import boto3
import json
import pytest
from moto import mock_dynamodb2
from lambda_function import *




@pytest.fixture
def test_sqs_event():
    test_sqs_event = {
        "Records": [{
            "body": json.dumps({
                "user_id": "B9B3022F98Fjvjs83AB8/80C185D8",
                "amount": 10 
                "updated_at": "2020-03-03T00:50:47"
            })
        }]}
    return test_sqs_event
    


class MockBotoSession:
    class Session:
        def client(self, service_name, region_name):
            class Client:
                def list_crawlers(self):
                    return {'CrawlerNames': ['crawler_1', 'crawler_2',]}
            return Client()
            


@mock_dynamodb2
@pytest.fixture(autouse=True)
def set_up(monkeypatch):


    dynamodb = boto3.resource('dynamodb', 'us-east-1')


    table = dynamodb.create_table(
        TableName='user_spending',
        KeySchema=[{'AttributeName': 'user_id', 'KeyType': 'HASH'}],
        AttributeDefinitions=[{'AttributeName': 'user_id','AttributeType': 'S'}],
        ProvisionedThroughput={'ReadCapacityUnits': 5, 'WriteCapacityUnits': 5}
    )


    monkeypatch.setattr(boto3, 'session', MockBotoSession)
    yield




def test_lambda_handler(test_sqs_event, set_up):
    
    response = lambda_handler(event=test_sqs_event, context={})


    table = dynamodb.Table(table_name)
    response = table.get_item(
        Key={'user_id': 'B9B3022F98Fjvjs83AB8a80C185D'}
    )
     
    item = response['Item']


    assert item['amount'] == 10
    assert 'updated_at' in item.keys()

We’ve come a long way from the empty test file at the beginning of the article, no?

我们距离本文开头的空白测试文件还有很长的路要走，不是吗？

As a reminder, this code tests a Lambda function that:

提醒一下，此代码测试了Lambda函数，该函数具有：

1. Triggers off an sqs message event
   触发sqs消息事件
2. Writes the message to a Dynamo Table
   将消息写入Dynamo表
3. Lists available Glue Crawlers
   列出可用的胶水爬行器
4. And finally reads back the data written to Dynamo and asserts that the final values match the input.
   最后读回写入Dynamo的数据，并断言最终值与输入匹配。

By employing these strategies though, you should feel confident testing a Lambda triggered from any event type, and one that interacts with any AWS service.

但是，通过采用这些策略，您应该自信地测试由任何事件类型触发的Lambda，以及与任何 AWS服务进行交互的Lambda。

最后的想法 (Final Thoughts)

If you’ve been struggling to test your Lambda functions, my hope is this article showed you a few tips to help you do so in a useful way.

如果您一直在努力测试Lambda函数，那么我希望本文向您显示了一些技巧，可以帮助您以有用的方式进行操作。

While we spent a lot of time on common issues and how you shouldn’t test a Lambda, we didn’t get a chance to cover the opposite, yet equally important aspect of this topic— namely what should you test, and how can you structure your Lambda function’s code to make it easier to do so.

尽管我们在常见问题上花了很多时间，以及您不应该如何测试Lambda，但我们没有机会涵盖该主题的相反但同样重要的方面-即您应该测试什么，以及如何进行测试结构化Lambda函数的代码，使其更容易实现。

I look forward to hearing from you and discussing how you test Lambda functions!

我期待着您的来信，并讨论如何测试Lambda函数！

Thank you to Vamshi Rapolu for inspiration and feedback on this article.

感谢Vamshi Rapolu对本文的启发和反馈。

其他参考 (Other References)

翻译自: https://towardsdatascience.com/how-i-write-meaningful-tests-for-aws-lambda-functions-f009f0a9c587

aws lambda测试