RabbitMQ（五）延时队列

最新推荐文章于 2024-07-30 17:01:52 发布

stopping5

最新推荐文章于 2024-07-30 17:01:52 发布

阅读量833

点赞数

分类专栏： RabbitMQ 文章标签： rabbitmq

本文链接：https://blog.csdn.net/stopping5/article/details/123675131

版权

RabbitMQ 专栏收录该内容

5 篇文章 0 订阅

订阅专栏

一、SpringBoot整合RabbitMQ

通过Springboot整合RabbitMQ，实现延迟队列处理。x是普通交换机分别通过XA、XB绑定到队列QA、QB。其中QA队列超时时间为10s、QB为40s。两个队列都通过YD路由键绑定到yExchange死信队列中。一旦超时则转发消息到QD队列，从而实现延时队列消息处理。
在这里插入图片描述

1. 新增依赖

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
    <version>2.2.2.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-amqp</artifactId>
    <version>2.4.4</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <version>2.2.2.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <version>1.18.10</version>
</dependency>

2. 添加配置

application.yml
spring:
  application:
    name: rabbit-consumer
  rabbitmq:
    host: RabbitMQ-IP
    port: 34566
    username: stopping
    password: 123456

3. 声明交换机、队列、绑定关系

package com.stopping.config;

import com.stopping.common.RabbitMQConfig;
import org.springframework.amqp.core.*;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
/**
 * 延时队列配置
 * */
@Configuration
public class TTLQueueConfig {
    /**
     * 声明x交换机
     * */
    @Bean
    public DirectExchange xExchange(){
        return new DirectExchange(RabbitMQConfig.X_EXCHANGE);
    }
    /**
     * 声明y交换机 - 死信交换机
     * */
    @Bean
    public DirectExchange yExchange(){
        return new DirectExchange(RabbitMQConfig.Y_EXCHANGE);
    }

    /**
     * 声明队列A
     * */
    @Bean
    public Queue queueA(){
        return QueueBuilder
                .durable(RabbitMQConfig.QA_QUEUE)
                //设置超时时间
                .ttl(10000)
                //设置死信交换机
                .deadLetterExchange(RabbitMQConfig.Y_EXCHANGE)
                //设置routing-key
                .deadLetterRoutingKey(RabbitMQConfig.QD_RK)
                .build();
    }

    @Bean
    public Queue queueB(){
        return QueueBuilder
                .durable(RabbitMQConfig.QB_QUEUE)
                .ttl(2000)
                .deadLetterExchange(RabbitMQConfig.Y_EXCHANGE)
                .deadLetterRoutingKey(RabbitMQConfig.QD_RK)
                .build();
    }
    /**
     * 声明队列D - 死信队列
     * */
    @Bean
    public Queue queueD(){
        return QueueBuilder
                .durable(RabbitMQConfig.QD_QUEUE)
                .build();
    }

    @Bean
    public Binding queueABindingX(Queue queueA,DirectExchange xExchange){
        return BindingBuilder.bind(queueA).to(xExchange).with(RabbitMQConfig.QA_RK);
    }

    @Bean
    public Binding queueBBindingX(Queue queueB, DirectExchange xExchange){
        return BindingBuilder.bind(queueB).to(xExchange).with(RabbitMQConfig.QB_RK);
    }

    @Bean
    public Binding queueDBindingY(Queue queueD, DirectExchange yExchange){
        return BindingBuilder.bind(queueD).to(yExchange).with(RabbitMQConfig.QD_RK);
    }
}

变量声明

package com.stopping.common;
/**
 * rabbit mq 配置信息
 * */
public class RabbitMQConfig {

    public static String QUEUE = "RABBIT_DEMO_QUEUE";

    public static String EXCHANGE = "RABBIT_DEMO_EXCHANGE";

    public static String ACK_QUEUE = "RABBIT_ACK_QUEUE";

    public static String ACK_EXCHANGE = "RABBIT_ACK_EXCHANGE";

    public static String NORMAL_EXCHANGE = "NORMAL_EXCHANGE";

    public static String NORMAL_QUEUE = "NORMAL_QUEUE";

    public static String DEAD_EXCHANGE = "DEAD_EXCHANGE";

    public static String DEAD_QUEUE = "DEAD_QUEUE";

    public static String X_EXCHANGE = "X";

    public static String Y_EXCHANGE = "Y";

    public static String QA_QUEUE = "QA";

    public static String QB_QUEUE = "QB";

    public static String QD_QUEUE = "QD";

    public static String QD_RK = "dead-message";

    public static String QA_RK = "XA";

    public static String QB_RK = "XB";
}

4. 生产者

发送消息给XExchange，但是不同的路由键分发到不同的队列中。不同的队列有不同的超时时间，所以到达死信队列的时间也不一样。

@Service
@Slf4j
public class SendMessageServiceImpl {
    @Resource
    private RabbitTemplate rabbitTemplate;

    public void send(String msg){
        rabbitTemplate.convertAndSend(RabbitMQConfig.X_EXCHANGE,RabbitMQConfig.QA_RK,msg);
        rabbitTemplate.convertAndSend(RabbitMQConfig.X_EXCHANGE,RabbitMQConfig.QB_RK,msg);
        log.info("发送成功");
    }
}

5. 消费者监听

通过@RabbitListener(queues = “QD”) 监听队列QD，当消息超时发送到死信交换机yExchange的时候，QD就开始处理超时消息。

@Slf4j
@Component
public class DeadQueueCustomer {

    @RabbitListener(queues = "QD")
    public void receive(Message message){
        String msg = new String(message.getBody());
        log.info("接收消息:{}",msg);
    }
}

6. 缺陷

每次需要新的超时时间都需要新建一个队列
若不需要新建队列则设置超时属性在消息上，但是由于队列先进先出的特性，导致即使超时短的消息都要等待超时长的消息过期后才能发送到死信队列。

7. 解决方案

使用延迟队列插件

二、延迟队列插件

下载插件地址：https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/releases
官方地址：https://www.rabbitmq.com/community-plugins.html

支持RabbitMQ版本：3.8+

1. Docker安裝延迟队列插件

将下载的插件上传到物理机
复制插件到docker容器的plugins
rabbit-mq 是容器的名称，也可以用容器的ID

docker cp /download/rabbitmq_delayed_message_exchange-3.9.0.ez rabbit-mq:/plugins

进入容器查看插件并且启用插件
查看插件是否复制

root@685f4d665714:/plugins# ls | grep delay
rabbitmq_delayed_message_exchange-3.9.0.ez

启用插件

root@685f4d665714:/plugins# rabbitmq-plugins enable rabbitmq_delayed_message_exchange

Enabling plugins on node rabbit@685f4d665714:
rabbitmq_delayed_message_exchange
The following plugins have been configured:
  rabbitmq_delayed_message_exchange
  rabbitmq_management
  rabbitmq_management_agent
  rabbitmq_prometheus
  rabbitmq_web_dispatch
Applying plugin configuration to rabbit@685f4d665714...
The following plugins have been enabled:
  rabbitmq_delayed_message_exchange

重启rabbitmq容器

docker restart rabbit-mq

确认插件是否生效
Rabbit MQ 管理界面新建exchange查看是否有延迟队列类型

2. 使用方式

声明交互机，设置交换机类型为x-delayed-message ，设置参数"x-delayed-type" 。当然交换机类型可以设置direct 、topic 。

Map<String, Object> args = new HashMap<String, Object>();
args.put("x-delayed-type", "direct");
channel.exchangeDeclare("my-exchange", "x-delayed-message", true, false, args);

发送消息时，设置消息延迟属性这里需要区别于超时属性。

byte[] messageBodyBytes = "delayed payload".getBytes("UTF-8");Map<String, Object> headers = new HashMap<String, Object>();
headers.put("x-delay", 5000);
AMQP.BasicProperties.Builder props = new AMQP.BasicProperties.Builder().headers(headers);
channel.basicPublish("my-exchange", "", props.build(), messageBodyBytes);

byte[] messageBodyBytes2 = "more delayed payload".getBytes("UTF-8");Map<String, Object> headers2 = new HashMap<String, Object>();
headers2.put("x-delay", 1000);
AMQP.BasicProperties.Builder props2 = new AMQP.BasicProperties.Builder().headers(headers2);
channel.basicPublish("my-exchange", "", props2.build(), messageBodyBytes2);

性能影响

Due to the “x-delayed-type” argument, one could use this exchange in place of other exchanges, since the “x-delayed-message” exchange will just act as proxy. Note that there might be some performance implications if you do this.
For each message that crosses an “x-delayed-message” exchange, the plugin will try to determine if the message has to be expired by making sure the delay is within range, ie: Delay > 0, Delay =< ?ERL_MAX_T (In Erlang a timer can be set up to (2^32)-1 milliseconds in the future).
If the previous condition holds, then the message will be persisted to Mnesia and some other logic will kick in to determine if this particular message delay needs to replace the current scheduled timer and so on.
This means that while one could use this exchange in place of a direct or fanout exchange (or any other exchange for that matter), it will be slower than using the actual exchange. If you don’t need to delay messages, then use the actual exchange.

局限性

Delayed messages are stored in a Mnesia table (also see Limitations below) with a single disk replica on the current node. They will survive a node restart. While timer(s) that triggered scheduled delivery are not persisted, it will be re-initialised during plugin activation on node start. Obviously, only having one copy of a scheduled message in a cluster means that losing that node or disabling the plugin on it will lose the messages residing on that node.
This plugin was created with disk nodes in mind. RAM nodes are currently unsupported and adding support for them is not a priority (if you aren’t sure what RAM nodes are and whether you need to use them, you almost certainly don’t).
The plugin only performs one attempt at publishing each message but since publishing is local, in practice the only issue that may prevent delivery is the lack of queues (or bindings) to route to.
Closely related to the above, the mandatory flag is not supported by this exchange: we cannot be sure that at the future publishing point in time

there is at least one queue we can route to
the original connection is still around to send a basic.return to
Current design of this plugin doesn’t really fit scenarios with a high number of delayed messages (e.g. >100s of thousands or millions). See #72 for details.

三、通过延时插件实现延时队列

1. 配置声明

这里主要根据官方的说明，设置交换机的参数，其他的没有什么区别。具体如何延时发送需要在发送消息的时候设置延迟时间。

/**
 * 延迟队列
 * */
@Configuration
public class DelayConfig {
    public static String DELAY_EXCHANGE = "DELAY.EXCHANGE";

    public static String DELAY_QUEUE = "DELAY.QUEUE";

    public static String DELAY_ROUTING_KEY = "delay-message";

    @Bean
    public CustomExchange delayExchange(){
        Map<String,Object> arg = new HashMap<>(3);
        arg.put("x-delayed-type","direct");
        CustomExchange customExchange = new CustomExchange(DELAY_EXCHANGE,"x-delayed-message",false,false,arg);
        return customExchange;
    }

    @Bean
    public Queue delayQueue(){
        return QueueBuilder
                .durable(DELAY_QUEUE)
                .build();
    }

    @Bean
    public Binding delayBind(CustomExchange delayExchange,Queue delayQueue){
        return BindingBuilder.bind(delayQueue).to(delayExchange).with(DELAY_ROUTING_KEY).noargs();
    }
}

2. 消息生产者

设置了消息的delay 属性。

public void sendDelay(String msg,Integer delayTime){
    rabbitTemplate.convertAndSend(DelayConfig.DELAY_EXCHANGE,DelayConfig.DELAY_ROUTING_KEY,msg,message -> {
        message.getMessageProperties().setDelay(delayTime);
        return message;
    });
    log.info("当前时间:{},发送消息:{},延时:{}",new Date().toString(),msg,delayTime);
}

3. 消息消费者

跟普通消费者一致，监听DELAY.QUEUE

@RabbitListener(queues = "DELAY.QUEUE")
public void receiveDelay(Message message){
    String msg = new String(message.getBody());
    log.info("当前时间:{},消费消息:{}",new Date().toString(),msg);
}

4. 测试

当前时间:Tue Mar 22 17:19:50 CST 2022,发送消息:延迟消息50s,延时:50000
当前时间:Tue Mar 22 17:19:55 CST 2022,发送消息:延迟消息20s,延时:20000
当前时间:Tue Mar 22 17:19:59 CST 2022,发送消息:延迟消息10s,延时:10000
当前时间:Tue Mar 22 17:20:09 CST 2022,消费消息:延迟消息10s
当前时间:Tue Mar 22 17:20:15 CST 2022,消费消息:延迟消息20s
当前时间:Tue Mar 22 17:20:40 CST 2022,消费消息:延迟消息50s