消费者驱动的契约测试_消费者驱动的合同测试的魔力

消费者驱动的契约测试

前言 (Preface)

Why hello there! My goal in this article is to share what Consumer-Driven Contract Tests are, how to design them, how we use them at Hootsuite, and talk about some of the technical challenges that come with these tests. For simplicity, I’ll refer to Consumer-Driven Contract Tests as contract tests. So let’s get started!

为什么在那里！ 我在本文中的目标是分享什么是消费者驱动的合同测试，如何设计它们，如何在Hootsuite中使用它们，并讨论这些测试所带来的一些技术挑战。 为简单起见，我将消费者驱动的合同测试称为合同测试。 因此，让我们开始吧！

什么是合同测试？ (What are Contract Tests?)

Now before you think you are signing your life away, contract tests are quite an integral part of creating a reliable service-oriented architecture.

现在，在您认为自己要放弃自己的生活之前，合同测试是创建可靠的面向服务的体系结构不可或缺的一部分。

As an application evolves, code changes often result in side effects. These could be schema change of core application models, change in values in each respective fields, or changes that resulting in a different side effect for your application. In a monolithic application, good unit tests and integration tests would catch breaking modifications. However, unit tests and integration tests simply aren’t enough for microservices. Often, services are maintained by different teams. The more teams there are the more changes that can occur on any data schema i.e. events sent between services, and the higher the chance that breaking changes can happen. This is where contract tests shine.

随着应用程序的发展，代码更改通常会带来副作用。 这些可能是核心应用程序模型的架构更改，每个相应字段中值的更改或导致应用程序产生不同副作用的更改。 在单片应用程序中，良好的单元测试和集成测试会捕获重大的修改。 但是，单元测试和集成测试对于微服务而言还远远不够。 通常，服务由不同的团队维护。 团队越多，在任何数据模式(即服务之间发送的事件)上发生的更改就越多，发生重大更改的机会就越大。 这就是合同测试的亮点。

Contract testing first and foremost is a design pattern for testing that does NOT require frameworks to create! A lot of the time developers would look for frameworks or tools to create tests, but contract tests are not restricted to a particular language and do not require additional tools or frameworks.

合同测试首先是不需要框架即可创建的测试设计模式！ 很多时候，开发人员会寻找框架或工具来创建测试，但是合同测试并不限于特定的语言，并且不需要其他工具或框架。

Contract tests can be thought of as integration tests between services that depend on each other. These tests ensure the link between a service providing some functionality (producer) and the service (consumer) which depends on these functions is not broken. This can be an HTTP endpoint for which consumers call on or producers produce an event that the consumer consumes. In this article, we are focusing on contract testing for “asynchronous” flows (producers produce an event that the consumers later consume). By no means are contract tests end-to-end testing of your entire application. Instead, they will test and guarantee parts of your application are working correctly by probing parts of your application, and they are meant to be faster than end-to-end testing!

合同测试可以看作是相互依赖的服务之间的集成测试。 这些测试确保了提供某些功能的服务(生产者)和依赖于这些功能的服务(消费者)之间的链接没有中断。 这可以是消费者调用的HTTP端点，或者生产者可以产生消费者消费的事件。 在本文中，我们将重点放在针对“异步”流的合同测试上(生产者产生了一个事件，消费者后来消费了)。 合同测试绝不是整个应用程序的端到端测试。 相反，它们将通过探测应用程序的各个部分来测试并保证应用程序的某些部分正常工作，并且它们比端到端测试要快！

Here in this article, we focus on consumer-driven contract tests. What does this mean? In each test case, consumers are essentially telling the producers “hey, you are required to behave exactly in this manner, and don’t you dare change that!”. This way tests are focused on ensuring the consumer’s demands are met.

在本文中，我们重点关注消费者驱动的合同测试。 这是什么意思？ 在每个测试案例中，消费者实际上是在告诉生产者“嘿，您必须严格按照这种方式行事，并且您不敢改变它！”。 这种方式的测试着重于确保满足消费者的需求。

合同测试用例配方 (Contract Test Case Recipe)

So, what makes up an individual contract test case? Well, let’s take a look at the following diagram…

那么，由什么构成单个合同测试用例？ 好吧，让我们看一下下图...

(Helpful definitions)

(有用的定义)

• Events — Data resulting from an action or activity that is generated by one service and meant to be sent/consumed by other services
  事件-由一项服务生成并打算由其他服务发送/消费的操作或活动产生的数据
• Producers — Services Emitting events
  生产者-服务发射事件
• Broker — Intermediate service that holds emitted events and provides them to consumers
  经纪人-中间服务，用于保存发出的事件并将其提供给消费者
• Consumers — Services Consuming events
  消费者-服务消费事件

At Hootsuite, contract tests run against our live staging infrastructure, using the services and Kafka brokers deployed on staging.

在Hootsuite，使用部署在分阶段的服务和Kafka经纪人，对我们的实时分阶段基础结构进行合同测试。

At a high level, the end-to-end flow of a contract consists of 3 phases:

在较高的层次上，合同的端到端流程包括三个阶段：

Phase 1 — Emit Test Data

第1阶段 -发射测试数据

This normally involves defining and generating/mocking the necessary data by using the producer service. This can be done either through SDKs or directly interacting with a service’s endpoints to create the data. When that is not possible, mocking the data within the test case is an option, however, it’s better to test with live data. The goal here is to then emit the data to the brokers.

这通常涉及使用生产者服务定义和生成/模拟必要的数据。 这可以通过SDK进行，也可以直接与服务的端点进行交互以创建数据。 如果不可能的话，可以在测试用例中模拟数据，但是，最好使用实时数据进行测试。 目的是将数据发送给代理。

Phase 2 — Broker Receives Data

第二阶段 -经纪人接收数据

Great, when data is created and emitted, a broker acts as the intermediary in microservice architectures. At Hootsuite, we use Kafka as the broker between the producers and consumers. A producer service places the data or “event” into Kafka, and services listening on to Kafka will grab events that they want and when they want it. Here in phase 2, we want to be able to check whether the event has been emitted and stored in Kafka. Though this is optional as a similar check can be performed in phase 3.

很好，当创建和发出数据时，代理充当微服务体系结构的中介。 在Hootsuite，我们使用Kafka作为生产者和消费者之间的中间人。 生产者服务将数据或“事件”放置到Kafka中，侦听Kafka的服务将获取他们想要的事件以及他们想要的事件。 在第2阶段的此处，我们希望能够检查事件是否已经发出并存储在Kafka中。 尽管这是可选的，因为可以在阶段3中执行类似的检查。

Phase 3 — Consumers Receives Data

第三阶段 -消费者接收数据

Consumers listen to Kafka and identify events it wants and will consume the event into its service. Once the data is consumed, the consumer will perform actions on the data to yield either an expected data or behavior for your application. It is this data or behavior that you want to assert on!

消费者聆听Kafka并识别其想要的事件，并将该事件用于其服务中。 消费完数据后，使用者将对数据执行操作以产生应用程序的预期数据或行为。 您要声明的数据或行为！

Currently, most contract tests are embedded as part of both the producer and consumer’s build pipelines at Hootsuite. When a build on either the producer or consumer service runs, the contract tests are cloned from the repo, built, and executed. This way, when there is any breaking change in either the producer or consumer service, it is caught before any code gets deployed. To a certain degree, these also serve as documentation to developers modifying any service codebase. Though any tests added to any codebase should technically also serve as good documentation …. However, that’s not the only place you can place contract tests! Some additional ideas:

目前，大多数合同测试嵌入作为生产者和消费者来自HootSuite的构建既管道的一部分。 当运行基于生产者服务或消费者服务的构建时，合同测试将从存储库中克隆，构建并执行。 这样，当生产者服务或使用者服务中发生任何重大更改时，都会在部署任何代码之前捕获该更改。 在一定程度上，它们还充当开发人员修改任何服务代码库的文档。 尽管从技术上讲，任何添加到任何代码库中的测试也应作为良好的文档……。 但是，这不是唯一可以进行合同测试的地方！ 一些其他想法：

• Jenkins Scheduled Build Jobs — Running on intervals
  Jenkins计划的构建作业-定期运行
• Create a contract testing service that executes them at scheduled intervals — proactive and can serve as health status for your services
  创建一个合同测试服务，该服务可以按计划的时间间隔执行它们-主动并且可以用作服务的健康状态
• Embed the tests inside a docker container, this way the time it takes to perform setup for contract test execution can be reduced.
  将测试嵌入docker容器中，这样可以减少执行合同测试执行设置所需的时间。

Hootsuite合同测试 (Hootsuite Contract Tests with a Twist)

At Hootsuite, Contract Tests are done with a little bit more complexity. To achieve more purposeful testing, phase one in the previous section has a bit of a twist. Instead of mocking the data, we can generate the necessary data for the producers by hitting the APIs of major social networks — Facebook, Twitter, Instagram, LinkedIn …etc. Once a Tweet or Post is created, if the social network accounts are linked to Hootsuite, our services will receive the creation event information and use that as the data for the contract tests! LIVE DATA!

在Hootsuite，合同测试要稍微复杂一些。 为了实现更有针对性的测试，上一节中的第一阶段有些曲折。 通过模拟主要社交网络的API(Facebook，Twitter，Instagram，LinkedIn…等)，我们可以为生产者生成必要的数据，而不是模拟数据。 创建推文或帖子后，如果社交网络帐户已链接到Hootsuite，我们的服务将接收创建事件信息，并将其用作合同测试的数据！ 实时数据！

(Pretty cool isn’t it? 😉)

(不是很酷吗？)

Though this is cool, this adds 3 additional challenges to writing reliable contract tests:

尽管这很酷，但这给编写可靠的合同测试增加了3个其他挑战：

Data Schema Changes

数据架构变更

• It’s no surprise API’s get deprecated or changed all the time, this means contract tests, depending on how the API’s data is formatted and which fields are used would often require changes
  总是不建议使用或更改API并不奇怪，这意味着要进行合同测试，具体取决于API数据的格式设置和使用的字段通常需要进行更改

Data correctness

数据正确性

• People make mistakes, API’s underlying changes can sometimes go sideways, you can always end up receiving faulty data. So sometimes your tests aren’t failing because you’ve made a mistake but the social network you depend on has a bug!
  人们会犯错误，API的基础更改有时可能会横冲直撞，您总会收到错误的数据。 因此有时您的测试不会失败，因为您犯了一个错误，但是您所依赖的社交网络却有一个错误！

Timing of Data delivery

数据传送时间

• Probably more “on-time” than your average package from Canada Post, there can always be a delay in when you receive the data generated by external APIs …. especially when traffic is worse than the morning rush
  “准时”时间可能比您从Canada Post获得的平均包裹时间要多，在接收外部API生成的数据时总会出现延迟……。 特别是在交通状况不如早上高峰时

Here’s the key point

这是关键

Hootsuite’s contract tests rely on live data provided by Social Network APIs

Hootsuite的合同测试依赖于社交网络API提供的实时数据

This opens the door to a lot of cool testing strategies that can be done. However, at the same time, introduces additional points of failure for test cases. Yet, the pros outweigh the cons as the changes from upstream third party API can also be captured as part of these tests, implicitly contributing to the application’s test coverage and reliability.

这为许多很酷的测试策略打开了大门。 但是，与此同时，为测试用例引入了更多的故障点。 但是，优点也胜过缺点，因为上游第三方API的更改也可以作为这些测试的一部分被捕获，从而隐含地提高了应用程序的测试覆盖范围和可靠性。

Twitter合约测试 (Twitter Contract Tests)

Recently, at Hootsuite, we did work to migrate our Twitter services from polling to Webhook, referred to as Twitter Migration. So why was this important for building out Contract Tests?

最近，在Hootsuite，我们确实致力于将Twitter服务从轮询迁移到Webhook，称为Twitter迁移。 那么，为什么这对于建立合同测试很重要？

Well, to begin with, Webhooks provide a more reliable and instantaneous response to any of Twitter-related activities! Thus, we would be able to create live data directly from the Twitter APIs to test. This made the tests more meaningful, robust, and reliable with data coming faster than polling reducing the possibility that the tests may fail due to time outs.

好吧，首先，Webhooks对任何与Twitter相关的活动都提供了更可靠，即时的响应！ 因此，我们将能够直接从Twitter API创建实时数据进行测试。 这使测试更有意义，更健壮和更可靠，数据的发送速度比轮询更快，从而减少了由于超时而导致测试失败的可能性。

Here’s how we do it…. with a diagram 😎

这是我们的操作方法…。 带有图😎

Within the contract test suite (the same test suite should also be executed in producer service).

在合同测试套件中(同一测试套件也应在生产者服务中执行)。

1. Generate Live Data (Producer)

   生成实时数据(生产者)

At the beginning of the test case, we make calls to generate Twitter Mention or DMs through a service that specifically handles interactions with the Twitter API. This will allow us to supply dynamic content to be tested. The message content should be dynamic and changes on each request if you are generating live data.

在测试用例的开始，我们通过专门处理与Twitter API交互的服务进行调用，以生成Twitter Mention或DM。 这将使我们能够提供要测试的动态内容。 消息内容应该是动态的，并且如果您正在生成实时数据，则每个请求都将更改。

2. Check if Data is Present in Kafka (Broker)

2.检查Kafka(经纪人)中是否存在数据

Once the service responsible for emitting twitter related events (Producer Service) receives the event via Webhook, the data is packaged into an event object and placed into the Kafka brokers. Here, we can assert whether the event was generated and placed into Kafka.

一旦负责发布Twitter相关事件的服务(生产者服务)通过Webhook接收到该事件，该数据便被打包到一个事件对象中，并放入Kafka代理中。 在这里，我们可以断言事件是否已生成并放置到Kafka中。

3. Check Expected Behavior (Consumer)

3.检查预期行为(消费者)

Once we are guaranteed an event is in Kafka, we can test whether the event generates an expected behavior in the consumer services. Note here, that despite the service for determining message assignment and tagging matching the definition of consumer service, it is easier to test the behavior of its consumer services (services that actually performed the assigning and tagging of messages). Utilizing the SDK from the downstream services, we can check whether the correct assignment or tagging occurred.

一旦确定某个事件在Kafka中，我们就可以测试该事件是否在消费者服务中产生了预期的行为。 在此请注意，尽管用于确定消息分配和标记的服务与使用者服务的定义相匹配，但是测试其使用者服务(实际上执行消息的分配和标记的服务)的行为更容易。 利用下游服务中的SDK，我们可以检查是否发生了正确的分配或标记。

Alright, this can’t be a technical article without some code …. so……. let’s take a look at an example! Here’s something we’re using for Twitter contract tests!

好吧，如果没有一些代码，这不可能是一篇技术文章...。 所以……。 让我们来看一个例子！ 这是我们用于Twitter合约测试的东西！

/*
    Scenario: A registered user direct mentions a registered user
    Expected Result: The receiving registered user should be assigned.
*/
"Creating a Twitter Mention" should "generate an assignment" in new TestCase {
// Preparing the request with the unique text we want to tweet
    val request = generateCreatePostRequest(
      messageBody = s"@$receiverName $randomDayQuote",
      socialProfileId = senderSocialProfileId
    )// PHASE 1
// Create the tweet 
    whenReady(scumApi.createPosts(List(request))) {
// Wait for the resulting confirmation that a tweet has been made
// and save the response fields so we can use it to validate within
// our own system.
      result: List[CreatePostResponse] =>
        val response = result.headOption.get
        response.postId shouldBe defined// PHASE 2
// Checking broker
        eventually {
// check if the upstream producer service, whether it successfully 
// obtained a response and provided the correct event inside kafka
// by validating if the postId we got from tweet creation response
// matches the event's postId.
          validateEventBus(response.postId.get)
        }// PHASE 3
// Checking if downstream services performed expected action and 
// assert on them 
        eventually {
// check if message assigning service correctly assigned a
// message/post to a user
         whenReady(isAssigned(response.postId.get, receiverOrgId)){
            isAssigned =>
              isAssigned shouldBe true
          }
        }        eventually {
// check if message has been tagged correctly and the correct number // of tags have been applied. 
          whenReady(getTags(response.postId.get)) { response =>
            response.tags.length shouldBe 1
            response.tags.head.id shouldBe tagId
          }
        }
    }
  }

Couple things to note:

需要注意的几件事：

Eventually blocks

最终块

These are critical to performing these tests in Scala. We can retry policies to handle scenarios like a race condition, where the actual result hasn’t been generated yet.

这些对于在Scala中执行这些测试至关重要。 我们可以重试策略以处理诸如竞赛条件之类的情况，而实际结果尚未生成。

Imagine having two endpoints for

想象一下，有两个端点

• Creating a message
  建立讯息
• Retrieving a message
  检索消息

If I requested to Create a message and immediately make a request to retrieve the message, if the message hasn’t been created we’ve got an empty response. However, does this mean the message can’t be created? Nope! It may just not have been created by the time your request for retrieving the message was made. Thus, using an eventually block with a retry policy specifying the interval of retry and the max number of retries will allow you to check the retrieve message endpoint at a later point to see if the retrieve message endpoint can return a non-empty response!

如果我请求创建一条消息并立即发出请求以检索该消息，如果尚未创建该消息，我们将得到一个空的响应。 但是，这是否意味着无法创建消息？ 不！ 您提出检索消息的请求时，可能尚未创建该消息。 因此，使用带有重试策略的finally块并指定重试间隔和最大重试次数，将使您可以在以后检查检索消息端点，以查看检索消息端点是否可以返回非空响应！

Define your policy carefully

仔细定义您的政策

It may be alright for calls to internal Hootsuite services to have more frequent retry policies, but when it comes to third-party API’s? Have longer intervals between retries and fewer retries!

调用内部Hootsuite服务具有更频繁的重试策略可能还不错，但是涉及第三方API呢？ 重试间隔较长，重试次数更少！

WhenReady blocks

WhenReady块

This can be thought of like awaits from JavaScript, just handling async responses! That’s it!

可以认为这就像从JavaScript中等待，只是处理异步响应！ 而已！

技术挑战 (Technical Challenges)

Now, a couple of things I’ve encountered while creating these contract tests.

现在，在创建这些合同测试时遇到了几件事。

Testing Flakiness — Many moving parts within in contract tests can result in test flakiness

测试剥落感-合同测试中的许多运动部件可能导致测试剥落感

• Do your technical due diligence, wait for a reasonable time for network calls to return, and set a good retry policy that maximizes the success rate.
  进行技术尽职调查，等待合理的时间以使网络呼叫返回，并设置良好的重试策略以最大程度地提高成功率。
• Accept the fact some tests may fail due to flakiness, so long as when evaluating the value of the test case, this test still highlights and covers an important part of your code, this may be an acceptable flakiness
  接受以下事实：某些测试可能会由于脆弱性而失败，只要评估测试用例的值，该测试仍然突出显示并覆盖了代码的重要部分，这可能就是可接受的脆弱性

Identifying points of failure

确定故障点

Relating to the point above. Sure, contract tests can alert developers that something is wrong. However, it often becomes difficult to pinpoint where exactly is the problem. This still requires manual debugging, tracing an event’s path to identify what could have gone wrong. For example, sometimes the failure is not related to your tests or your services, but the upstream APIs have failed. It may take quite a bit of time until you identify that. To combat that, really an SOP (Standard Operation Procedures) can be created to let developers know where to look first. This way, it’ll be easier to identify the points of failure!

关于以上几点。 当然，合同测试可以提醒开发人员某些问题。 但是，通常很难精确地确定问题出在哪里。 这仍然需要手动调试，以跟踪事件的路径来确定可能出了什么问题。 例如，有时故障与您的测试或服务无关，但上游API已失败。 您可能需要花费很多时间才能识别出来。 为了解决这个问题，实际上可以创建一个SOP(标准操作程序)来让开发人员知道首先要看的地方。 这样，可以更轻松地确定故障点！

Don’t Spam! You aren’t a malicious bot!

不要垃圾邮件！ 您不是恶意机器人！

If you are depending on external APIs or services, you may encounter rate-limiting or spam detection for the data you are emitting. Be courteous and mindful of how frequently you run the tests and randomize your data with some meaningful text and change it up in each call! I’m sure there’s a list of jokes somewhere on the internet you could use to randomize what you post 😂

如果您依赖于外部API或服务，则可能会遇到速率限制或垃圾邮件检测所发送的数据。 保持礼貌并注意您运行测试的频率，并使用一些有意义的文本来随机化数据，并在每次通话中进行更改！ 我敢肯定，互联网上的某个地方有一个笑话清单，您可以用来随机分配您的帖子what

它为什么如此重要？ (Why is it Important?)

Additional Functions of Contract Tests

合同测试的附加功能

• Serve as documentation for use case scenarios for critical components of your application
  用作应用程序关键组件的用例场景的文档
• Provide confidence in future changes (avoiding breaking changes) of these critical components
  对这些关键组件的未来更改(避免发生重大更改)充满信心
• Help drive Test-Driven Development (TDD) — Once one test exists, it’ll open the door to more tests to write!
  帮助推动测试驱动开发(TDD)-一旦存在一个测试，它将为编写更多测试打开大门！

Thinking back to my personal experience with the Twitter Migration work. It was certainly bumpy, as for a simple Tweet, one must also account for scenarios such as retweets, retweet with mentions, replies, nested replies, single mention, multi-mentions, and the list goes on. For a new developer who’s never touched Twitter itself, this can get truly confusing, and something would be missed.

回想我在Twitter迁移工作中的个人经历。 对于一条简单的Tweet来说肯定是坎bump不平的，还必须考虑到以下场景：转发，以提及方式转发，回复，嵌套回复，单提及，多提及，并且列表还在继续。 对于从未接触过Twitter本身的新开发人员，这可能会造成真正的混乱，并且会遗漏一些东西。

结束语 (Concluding Remarks)

Sure, contract tests may consume a good chunk of a developer’s time to implement and may be expensive to maintain. However, the value it can provide when applied to critical components of your application can be extremely high. Having effective and tests allow you to scale your product, teams, and company efficiently with reduced application outages! Your SLO’s will thank you later 😂

当然，合同测试可能会占用开发人员大量的实施时间，并且维护成本可能很高。 但是，将其应用于应用程序的关键组件时可以提供的价值可能会非常高。 通过有效的测试，您可以减少应用程序停机，从而有效地扩展产品，团队和公司！ 您的SLO将在稍后感谢您😂

翻译自: https://medium.com/hootsuite-engineering/the-magic-of-consumer-driven-contract-tests-493145dee2f8

消费者驱动的契约测试