队列管理器的死信队列_管理死信队列

最新推荐文章于 2022-06-24 17:58:23 发布

weixin_26729165

最新推荐文章于 2022-06-24 17:58:23 发布

阅读量281

点赞数

原文链接：https://medium.com/dev-genius/managing-dead-letter-queues-be30672fc73f

版权

队列管理器的死信队列

Exchanging messages from a microservice to another has never been easier. With the presence of modern web frameworks we are now able to bootstrap microservices from ground up with minimal configuration, e.g. Spring Boot which favors convention over configuration. It is also worth mentioning Spring JMS API — an abstraction layer to JMS that makes development easier because all connection, session, and JNDI contexts are all abstracted away. Combine this with Spring Integration Framework which allows services to communicate via message channels and in return makes the overall architecture loosely coupled.

从微服务到另一服务的消息交换从未如此简单。随着现代Web框架的出现，我们现在能够以最少的配置从头开始引导微服务，例如Spring Boot倾向于使用约定而非配置。还值得一提的是Spring JMS API，它是JMS的抽象层，因为所有连接，会话和JNDI上下文都被抽象了，因此使开发更加容易。将此与Spring Integration Framework结合使用，该框架允许服务通过消息通道进行通信，从而使整个体系结构松散耦合。

While the things that I’ve mentioned are all good and great we can’t ignore the trade offs that are being made.

尽管我提到的东西都很好，但我们不能忽略正在做出的取舍。

最终一致性 (Eventual Consistency)

Image for post — Diagram created in miro.com

Regarding transactions within a Monolith — Atomicity is easily achieved. In Spring it might be as simple as annotating a method with @Transactional, but this is not the case with Asynchronous Messaging. We might opt to limit the transactions only within bounded context which is good but we can’t avoid it entirely i.e. a feature request related to Fund Transfer to a different bank. Considering the scenario wherein the amount to be transferred is already debited from the payor but then an exception occurred on the other bank service while trying to credit it to the payee — we need to perform some kind of rollback to attain eventual consistency and for this we’ll be utilizing a Dead Letter Queue.

关于Monolith中的交易-容易实现原子性。在Spring中，使用@Transactional注释方法可能很简单，但是异步消息却并非如此。我们可能会选择仅在有限的范围内限制交易，这很好，但我们不能完全避免，即与将Fund Transfer到另一家银行有关的功能请求。考虑到以下情况：要转移的金额已从付款人处扣除，但随后另一家银行服务发生异常，同时试图将其贷记给收款人，我们需要执行某种回滚操作以实现最终的一致性，为此，我们将使用Dead Letter Queue 。

Disclaimer: The solution I’m about to demo is oversimplified and does not reflect the complex business rule around financial systems. This article’s main focus is on managing dead letter queues and I’ll be providing a step by step guide for anyone who would like to follow.

免责声明：我要演示的解决方案过于简单，并且没有反映出围绕金融系统的复杂业务规则。本文的主要重点是管理死信队列，我将为想要遵循的任何人提供逐步指南。

如何将邮件移至死信队列？ (How Messages are moved to a Dead Letter Queue?)

Note: We’ll be using ActiveMQ for this demo.

注意：我们将在此演示中使用ActiveMQ。

When a consumer tried processing a message and failed several times — as per ActiveMQ it would try up to seven times — and if all those attempts fail, then the message would be moved to the Default Dead Letter Queue, named ActiveMQ.DLQ.

当使用者尝试处理一条消息并失败几次时(按照ActiveMQ它将尝试最多七次)，并且如果所有这些尝试都失败了，则该消息将被移到名为ActiveMQ.DLQ的Default Dead Letter Queue 。

Let’s start with clean slate and set everything up from scratch. I’ll install ActiveMQ on docker (a personal preference so it’s easy to delete).

让我们从干净的石板开始，并从头开始设置所有内容。我将在码头上安装ActiveMQ(个人喜好，因此很容易删除)。

1.安装ActiveMQ (1. Installing ActiveMQ)

docker pull rmohr/activemq
docker run --name activemq -p 61616:61616 -p 8161:8161 -d rmohr/activemq

The JMX broker listens on port 61616 and the Web Console on port 8161with a container name of activemq and runs as a background process (-d). Visit http://localhost:8161 and you’ll see ActiveMQ is now up and running.

JMX broker使用容器名称activemq侦听端口61616和Web Console侦听端口8161上的Web Console ，并作为后台进程(-d)运行。访问http：// localhost：8161 ，您将看到ActiveMQ现在已启动并正在运行。

Since we’re here at ActiveMQ’s Web Console, let’s try sending a message using the Web Console in preparation for the Receiver service — so it will have a message to consume. We can access that facility by clicking on the “Manage ActiveMQ broker” then click on the “Send” tab.

因为我们在ActiveMQ的Web控制台中，所以让我们尝试使用Web控制台发送消息以准备Receiver服务-这样它将有一条消息可供使用。我们可以通过单击“管理ActiveMQ代理”，然后单击“发送”选项卡来访问该工具。

Make sure to check the Persistent Delivery. Non-persistent messages are by default — not being routed to a Dead Letter Queue.

确保检查“ Persistent Delivery 。默认情况下，非持久消息是不被发送到死信队列。

2.创建一个Spring Boot应用程序(消费者) (2. Creating a Spring Boot Application (Consumer))

I bet most us already know how to create a maven/gradle project or use the Spring Initialzr. I’ll skip that part and proceed with the dependencies and the application properties needed.

我敢打赌，大多数我们已经知道如何创建Maven / Gradle项目或使用Spring Initialzr。我将跳过这一部分，继续进行所需的依赖关系和应用程序属性。

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>


    <groupId>com.consumer</groupId>
    <artifactId>message-consumer</artifactId>
    <version>1.0-SNAPSHOT</version>


    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.0.3.RELEASE</version>
    </parent>


    <dependencies>
        <dependency>
            <groupId>org.springframework.integration</groupId>
            <artifactId>spring-integration-jms</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-activemq</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
        </dependency>
    </dependencies>


</project>

spring.activemq.broker-url=tcp://localhost:61616
spring.activemq.username=admin
spring.activemq.password=admin


fund.transfer.queue.name=fund.transfer

Then, create the Main class. If you are using Spring Initialzr, the Main class might be already present. There isn’t anything in particular that worth our attention with this Main class, regardless I’ll include it so everyone can see that there isn’t any extra annotation needed.

然后，创建Main类。如果您正在使用Spring Initialzr，则Main类可能已经存在。 Main类没有什么特别值得我们注意的，无论我将其包括在内，这样每个人都可以看到不需要任何额外的注释。

package com.consumer;


import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;


@SpringBootApplication
public class Main {


    public static void main(String[] args) {
        SpringApplication.run(Main.class, args);
    }


}

Since we’re using Spring Integration, let’s proceed with the configuration which will allow consumption of messages via channels.

由于我们使用的是Spring Integration ，因此让我们继续进行配置，该配置将允许通过channels消耗消息。

package com.consumer;


import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.integration.jms.ChannelPublishingJmsMessageListener;
import org.springframework.integration.jms.JmsInboundGateway;
import org.springframework.jms.listener.SimpleMessageListenerContainer;
import org.springframework.messaging.MessageChannel;


import javax.jms.ConnectionFactory;


@Configuration
public class IntegrationConfig {


    @Value("${fund.transfer.queue.name}")
    private String fundTransferQueueName;


    @Autowired
    private ConnectionFactory connectionFactory;


    @Bean
    public JmsInboundGateway inboundGateway() {
        JmsInboundGateway inboundGateway = new JmsInboundGateway(messageListenerContainer(), channelPublishingJmsMessageListener());
        inboundGateway.setRequestChannel(channel());
        return inboundGateway;
    }


    @Bean
    public SimpleMessageListenerContainer messageListenerContainer() {
        SimpleMessageListenerContainer messageListenerContainer = new SimpleMessageListenerContainer();
        messageListenerContainer.setConnectionFactory(connectionFactory);
        messageListenerContainer.setDestinationName(fundTransferQueueName);
        messageListenerContainer.setSessionTransacted(true); // IMPORTANT! this enables transaction management, retry won't happen without this
        return messageListenerContainer;                     // same as setting the transaction manager on JmsContainerFactory
    }


    @Bean
    public ChannelPublishingJmsMessageListener channelPublishingJmsMessageListener() {
        ChannelPublishingJmsMessageListener channelPublishingJmsMessageListener = new ChannelPublishingJmsMessageListener();
        channelPublishingJmsMessageListener.setExpectReply(true);
        return channelPublishingJmsMessageListener;
    }


    @Bean
    public MessageChannel channel() {
        return new DirectChannel();
    }


}

You might have noticed the comment on line 37. With Spring Integration, it is the way to enable rolling back of messages using transactions.

您可能已经注意到第line 37的评论。借助Spring Integration，它是启用使用事务回滚消息的方法。

Lastly let’s define the Receiver service, where we explicitly throw a RuntimeException to simulate the scenario in our diagram. Adding @Transactional annotation for the receive method would be redundant, because the receive method implicitly uses the transaction management from the messageListenerContainer.

最后，让我们定义Receiver服务，在这里我们显式抛出RuntimeException来模拟图中的场景。为接收方法添加@Transactional注释将是多余的，因为receive方法隐式地使用来自messageListenerContainer的事务管理。

package com.consumer;


import org.springframework.integration.annotation.MessageEndpoint;
import org.springframework.integration.annotation.ServiceActivator;


@MessageEndpoint
public class ReceiverService {


    @ServiceActivator(inputChannel = "channel")
    public void receive(String message) {
        throw new RuntimeException("an exception occured");
    }


}

Assuming that a message is present in the fund.transfer queue — Upon running the application we should now see a message in a Dead Letter Queue named ActiveMQ.DLQ.

假设在fund.transfer队列中存在一条消息—运行应用程序后，我们现在应该在名为ActiveMQ.DLQ死信队列中看到一条消息。

As mentioned earlier, ActiveMQ.DLQ is the default Dead Letter Queue. You might wonder — what if we are processing several messages from different queues? How are we going to handle each of those messages differently? — Use Individual Dead Letter Strategy.

如前所述， ActiveMQ.DLQ是默认的死信队列。您可能想知道-如果我们正在处理来自不同队列的几条消息怎么办？我们将如何以不同方式处理这些消息？ -使用Individual Dead Letter Strategy 。

使用个人死信策略 (Using Individual Dead Letter Strategy)

Assuming this strategy is in place — ActiveMQ will automatically create an individual dead letter queue for each queue, following the format specified in the configuration.

假设采用了该策略— ActiveMQ将按照配置中指定的格式为每个队列自动创建一个单独的死信队列。

Before creating the configuration, let’s take a step back and identify first if the broker (ActiveMQ) that we are using is Embedded or an External one. I’m mentioning this because I’ve made a mistake back then, assuming that I would be able to configure the dead letter strategy within the application while overlooking the fact that I’m using an External ActiveMQ.

在创建配置之前，让我们退后一步，首先确定我们正在使用的代理(ActiveMQ)是Embedded还是External 。我之所以这样说是因为我当时犯了一个错误，假设我能够在应用程序内配置死信策略，而忽略了我正在使用External ActiveMQ的事实。

For this demo — clearly we are using an External ActiveMQ which is installed using Docker.

对于此演示，显然，我们使用的是使用Docker安装的External ActiveMQ。

配置ActiveMQ以使用个人死信策略 (Configure ActiveMQ to use Individual Dead Letter Strategy)

Let’s access the ActiveMQ’s installation within the dockerized container and look for a file named activemq.xml in this directory: /opt/apache-activemq-${version}/conf. Then replace the <destinationPolicy> element which is inside the <broker> element with this Destination Policy.

让我们在dockerized容器中访问ActiveMQ的安装，并在以下目录activemq.xml查找一个名为activemq.xml的文件： /opt/apache-activemq-${version}/conf 。然后，使用此目标策略替换<broker>元素内的<destinationPolicy>元素。

<destinationPolicy>
    <policyMap>
      <policyEntries>
        <!-- Set the following policy on all queues using the '>' wildcard -->
        <policyEntry queue=">">
          <deadLetterStrategy>
            <!--
              Use the prefix 'DLQ.' for the destination name, and make
              the DLQ a queue rather than a topic
            -->
            <individualDeadLetterStrategy queuePrefix="DLQ." useQueueForQueueMessages="true"/>
          </deadLetterStrategy>
        </policyEntry>
      </policyEntries>
    </policyMap>
  </destinationPolicy>

To avoid confusion I only included the relevant part but if for some reason you would like to see the entire configuration, here it is: activemq.xml. Based on this configuration, each Dead Letter Queue will now have a prefix of DLQ..

为了避免混淆，我仅包括相关部分，但是如果出于某种原因您希望查看整个配置，则为： activemq.xml 。基于此配置，每个“死信队列”现在将具有DLQ.前缀DLQ. 。

Note: Restarting ActiveMQ is necessary for this configuration to take effect.

注意：必须重新启动ActiveMQ，此配置才能生效。

DLQ.fund.transfer is now present in the Queues. On a side note, consuming dead letter queues is basically the same as consuming any other queues.

DLQ.fund.transfer现在在队列中。附带说明，使用死信队列与使用其他队列基本相同。

结论 (Conclusion)

Dealing with Asynchronous Messaging, we can achieve Eventual Consistency using Dead Letter Queues. Furthermore, using Individual Dead Letter Strategy would allow us to have a separate Dead Letter Queue for each queue.

在处理异步消息时，我们可以使用死信队列来实现最终一致性。此外，使用“个人死信策略”将使我们可以为每个队列使用单独的死信队列。

I hope you’ve enjoyed reading this article.

希望您喜欢阅读本文。

Happy Coding!

编码愉快！

翻译自: https://medium.com/dev-genius/managing-dead-letter-queues-be30672fc73f

队列管理器的死信队列

weixin_26729165

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
复制链接

分享到 QQ

分享到新浪微博

扫一扫