4.RabbitMQ死信队列

RabbitMQ死信队列

概念介绍

相关概念

1. DLX：(dead-letter-exchange的缩写)死信队列交换机
2. DLK：(dead-letter-routing-key的缩写)死信队列routingKey
3. TTL：(time-to-live的缩写)存活时间

• DLX、DLK、TTL三者关系的解释
  • 当一个队列A设置了这三个属性时，如果队列A里面的消息变成了死信时，会被路由到与DLX交换机进行绑定的队列上，并且routingKey是DLK指定的。
  • 什么时候消息会变成死信消息呢？
    • 消息被拒绝（basic.reject或basic.nack）并且requeue=false.
    • 消息在原队列上存活了TTL这么长时间
    • 队列达到最大长度（原队列长度达到上限）
  • 如果没有设置DLK
    • 没有设置DLK的话，死信消息就会根据原来的routingKey在死信交换机上进行路由。假如说有一个队列A设置了DLX和TTL，没有设置DLK的话，如果现在队列A上有一个消息变成了死信消息，那么这个消息首先会通过原来的routingKey路由到与死信交换机进行绑定并且绑定的routingKey与这个死信消息的原来的routingKey相匹配。
  • 注意：死信消息再次路由时，它首先是被死信队列接收后才会从原来的队列里面被删除，这样原因可能是尽量避免消息的丢失。
    

死信队列相关配置

Map<String, Object> args=new HashMap();
// DLX(死信交换机)
args.put("x-dead-letter-exchange", "死信队列交换机的名称");
// DLK(死信路由key)
args.put("x-dead-letter-routing-key", "死信消息路由的routingKey");
// TTL(time-to-live存活时间)
args.put("x-message-ttl", 10000);
return new Queue(env.getProperty("simple.dead.queue.name"),true,false,false,args);

个人理解

站在TTL角度来看，当消息在原来的队列里面存活TTL长时间后，被路由到另一交换机(死信交换机)所绑定的队列(死信队列)，这个死信交换机和死信队列也可以理解为延迟交换机和延迟队列。(其实死信交换机和死信队列也是普通的交换机和队列)

示例代码

背景

• SpringBoot和RabbitMQ的整合

maven依赖

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
    <java.version>1.8</java.version>

    <slf4j.version>1.7.13</slf4j.version>
    <log4j.version>1.2.17</log4j.version>
    <mybatis-spring-boot.version>1.1.1</mybatis-spring-boot.version>
    <mybatis-pagehelper.version>4.1.2</mybatis-pagehelper.version>
    <druid.version>1.0.16</druid.version>
    <mysql.version>5.1.37</mysql.version>
    <okhttp.version>3.1.2</okhttp.version>
    <retrofit.version>2.1.0</retrofit.version>
    <guava.version>19.0</guava.version>
    <java-mail.version>1.6.0</java-mail.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <!--log start-->
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>${slf4j.version}</version>
    </dependency>
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-log4j12</artifactId>
        <version>${slf4j.version}</version>
    </dependency>
    <dependency>
        <groupId>log4j</groupId>
        <artifactId>log4j</artifactId>
        <version>${log4j.version}</version>
    </dependency>
    <!--log end-->
    <!--spring-mybatis-->
    <dependency>
        <groupId>org.mybatis.spring.boot</groupId>
        <artifactId>mybatis-spring-boot-starter</artifactId>
        <version>${mybatis-spring-boot.version}</version>
    </dependency>
    <!--for page-->
    <dependency>
        <groupId>com.github.pagehelper</groupId>
        <artifactId>pagehelper</artifactId>
        <version>${mybatis-pagehelper.version}</version>
    </dependency>
    <!--druid-->
    <dependency>
        <groupId>com.alibaba</groupId>
        <artifactId>druid</artifactId>
        <version>${druid.version}</version>
    </dependency>
    <!--mysql-->
    <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
        <version>${mysql.version}</version>
    </dependency>
    <!--rabbitmq-->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-amqp</artifactId>
        <version>${parent.version}</version>
    </dependency>
    <!-- okhttp -->
    <dependency>
        <groupId>com.squareup.okhttp3</groupId>
        <artifactId>okhttp</artifactId>
        <version>${okhttp.version}</version>
    </dependency>
    <dependency>
        <groupId>com.squareup.retrofit2</groupId>
        <artifactId>retrofit</artifactId>
        <version>${retrofit.version}</version>
    </dependency>
    <dependency>
        <groupId>com.squareup.retrofit2</groupId>
        <artifactId>converter-jackson</artifactId>
        <version>${retrofit.version}</version>
    </dependency>
    <!--guava-->
    <dependency>
        <groupId>com.google.guava</groupId>
        <artifactId>guava</artifactId>
        <version>${guava.version}</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-test</artifactId>
        <version>2.1.5.RELEASE</version>
        <scope>test</scope>
    </dependency>
</dependencies>

Spring配置文件application.properties

server.port=9092
server.context-path=/study_mq
#logging
logging.file.path=D:\\logs\\SpringBoot-RabbitMQ
logging.file.name=springboot-rabbitmq-01

spring.mvc.view.prefix=/WEB-INF/views/
spring.mvc.view.suffix=.jsp
multipart.max-request-size=20Mb
multipart.max-file-size=10Mb

logging.level.org.springframework = INFO
logging.level.com.fasterxml.jackson = INFO
logging.level.com.debug.steadyjack = DEBUG

spring.jackson.date-format=yyyy-MM-dd HH:mm:ss
spring.jackson.time-zone=GMT+8
spring.datasource.initialize=false
spring.jmx.enabled=false

#数据库
spring.datasource.type=com.alibaba.druid.pool.DruidDataSource
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://127.0.0.1:3306/study_rabbitmq?characterEncoding=utf8
spring.datasource.username=root
spring.datasource.password=123456

#mybatis
mybatis.config-location=classpath:mybatis-config.xml
mybatis.checkConfigLocation = true
mybatis.mapper-locations=classpath:mappers/*.xml

#rabbitmq
spring.rabbitmq.host=39.105.91.158
spring.rabbitmq.port=5672
spring.rabbitmq.username=jack
spring.rabbitmq.password=123456
spring.rabbitmq.virtual-host=/test

# 配置并发是多消费者
# 这个是什么意思目前也没了解清楚?
spring.rabbitmq.listener.concurrency=10
# 最多有多少个消费者
spring.rabbitmq.listener.max-concurrency=20
# 每个消费者预处理多少个数据
spring.rabbitmq.listener.prefetch=50

# 死信队列的配置
simple.dead.queue.name=${mq.env}.simple.dead.queue
simple.dead.exchange.name=${mq.env}.simple.dead.exchange
simple.dead.routing.key.name=${mq.env}.simple.dead.routing.key

simple.produce.exchange.name=${mq.env}.simple.produce.exchange
simple.produce.routing.key.name=${mq.env}.simple.produce.routing.key
simple.dead.real.queue.name=${mq.env}.simple.dead.real.queue

配置类-对RabbitMQ进行设置以及创建和绑定Queue、Exchange

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.core.AcknowledgeMode;
import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.DirectExchange;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.core.Queue;
import org.springframework.amqp.core.TopicExchange;
import org.springframework.amqp.rabbit.config.SimpleRabbitListenerContainerFactory;
import org.springframework.amqp.rabbit.connection.CachingConnectionFactory;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.amqp.rabbit.support.CorrelationData;
import org.springframework.amqp.support.converter.Jackson2JsonMessageConverter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.amqp.SimpleRabbitListenerContainerFactoryConfigurer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.env.Environment;

import java.util.HashMap;
import java.util.Map;


/**
 * Created by steadyjack on 2018/8/20.
 */
@Configuration
public class RabbitmqConfig {
    private static final Logger log= LoggerFactory.getLogger(RabbitmqConfig.class);
    @Autowired
    private Environment env;
    @Autowired
    private CachingConnectionFactory connectionFactory;
    @Autowired
    private SimpleRabbitListenerContainerFactoryConfigurer factoryConfigurer;
    @Autowired
    private RabbitTemplate rabbitTemplate;

    /**
     * 单一消费者
     * @return
     */
    @Bean(name = "singleListenerContainer")
    public SimpleRabbitListenerContainerFactory listenerContainer(){
        SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
        factory.setConnectionFactory(connectionFactory);
        factory.setMessageConverter(new Jackson2JsonMessageConverter());
        factory.setConcurrentConsumers(1);
        factory.setMaxConcurrentConsumers(1);
        factory.setPrefetchCount(1);
        factory.setTxSize(1);
        return factory;
    }

    /**
     * 多个消费者
     * @return
     */
    @Bean(name = "multiListenerContainer")
    public SimpleRabbitListenerContainerFactory multiListenerContainer(){
        SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
        factoryConfigurer.configure(factory,connectionFactory);
        factory.setMessageConverter(new Jackson2JsonMessageConverter());
        factory.setAcknowledgeMode(AcknowledgeMode.NONE);
        factory.setConcurrentConsumers(env.getProperty("spring.rabbitmq.listener.concurrency",int.class));
        factory.setMaxConcurrentConsumers(env.getProperty("spring.rabbitmq.listener.max-concurrency",int.class));
        factory.setPrefetchCount(env.getProperty("spring.rabbitmq.listener.prefetch",int.class));
        return factory;
    }

    @Bean
    public RabbitTemplate rabbitTemplate(){
        connectionFactory.setPublisherConfirms(true);
        connectionFactory.setPublisherReturns(true);
        RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
        rabbitTemplate.setMandatory(true);
        // 设置消息发送到rabbitMQ,rabbitMQ接收到这个消息后的回调
        rabbitTemplate.setConfirmCallback(new RabbitTemplate.ConfirmCallback() {
            @Override
            public void confirm(CorrelationData correlationData, boolean ack, String cause) {
                log.info("消息发送成功:correlationData({}),ack({}),cause({})",correlationData,ack,cause);
            }
        });
        // 设置消息发送到rabbitMQ,rabbitMQ找不到对应的队列发送这个消息时 将消息返回给生产者的回调
        rabbitTemplate.setReturnCallback(new RabbitTemplate.ReturnCallback() {
            @Override
            public void returnedMessage(Message message, int replyCode, String replyText, String exchange, String routingKey) {
                log.info("消息丢失:exchange({}),route({}),replyCode({}),replyText({}),message:{}",exchange,routingKey,replyCode,replyText,message);
            }
        });
        return rabbitTemplate;
    }
    
    //TODO：死信队列消息模型
    //创建死信队列
    @Bean
    public Queue simpleDeadQueue(){
        Map<String, Object> args=new HashMap();
        // DLX(死信交换机)
        args.put("x-dead-letter-exchange", env.getProperty("simple.dead.exchange.name"));
        // DLK(死信路由key)
        args.put("x-dead-letter-routing-key", env.getProperty("simple.dead.routing.key.name"));
        // TTL(time-to-live存活时间)
        args.put("x-message-ttl", 10000);
        return new Queue(env.getProperty("simple.dead.queue.name"),true,false,false,args);
    }
    //绑定死信队列-面向生产端
    @Bean
    public TopicExchange simpleDeadExchange(){
        return new TopicExchange(env.getProperty("simple.produce.exchange.name"),true,false);
    }
    @Bean
    public Binding simpleDeadBinding(){
        return BindingBuilder.bind(simpleDeadQueue()).to(simpleDeadExchange()).with(env.getProperty("simple.produce.routing.key.name"));
    }
    //TODO: 给死信队列 创建并绑定实际监听消费队列
    @Bean
    public Queue simpleDeadRealQueue() {
        return new Queue(env.getProperty("simple.dead.real.queue.name"));
    }
    @Bean
    public TopicExchange simpleDeadRealExchange() {
        return new TopicExchange(env.getProperty("simple.dead.exchange.name"));
    }
    @Bean
    public Binding simpleDeadRealBinding() {
        return BindingBuilder.bind(simpleDeadRealQueue()).to(simpleDeadRealExchange()).with(env.getProperty("simple.dead.routing.key.name"));
    }
}    

发送消息

import com.jack.response.BaseResponse;
import com.jack.response.StatusCode;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.core.MessageBuilder;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.env.Environment;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

/**
 * Created by Administrator on 2018/9/1.
 */
@RestController
public class DeadQueueController {

    private static final Logger log= LoggerFactory.getLogger(DeadQueueController.class);
    private static final String Prefix="dead/queue";

    @Autowired
    private RabbitTemplate rabbitTemplate;
    @Autowired
    private Environment env;

    @RequestMapping(value = Prefix+"/send",method = RequestMethod.GET)
    public BaseResponse sendMail(){
        BaseResponse response=new BaseResponse(StatusCode.Success);
        try {
            rabbitTemplate.setExchange(env.getProperty("simple.produce.exchange.name"));
            rabbitTemplate.setRoutingKey(env.getProperty("simple.produce.routing.key.name"));

            String str="死信队列的消息";
            Message message= MessageBuilder.withBody(str.getBytes("UTF-8")).build();
            rabbitTemplate.convertAndSend(message);

        }catch (Exception e){
            e.printStackTrace();
        }
        log.info("发送消息完毕----");
        return response;
    }
}

监听和消费

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.messaging.handler.annotation.Payload;
import org.springframework.stereotype.Service;

/**
 * @author cwl
 * @description: TODO
 * @date 2020/8/26 20:59
 */
@Service
public class CommonMqService {
    private static final Logger log = LoggerFactory.getLogger(CommonMqService.class);
    
        /**
     * 监听消费死信队列中的消息
     * @param message
     */
    @RabbitListener(queues = "${simple.dead.real.queue.name}",containerFactory = "singleListenerContainer")
    public void consumeDeadQueue(@Payload byte[] message){
        try {
            log.info("监听消费死信队列中的消息： {} ",new String(message,"UTF-8"));
        }catch (Exception e){
            e.printStackTrace();
        }
    }
}

运行这个代码并调用dead/queue//send接口发送数据到RabbitMQ时，在RabbitMQ的控制台可以发现，消息首先在local.simple.dead.queue队列，过了10000ms后，消息被路由到了local.simple.dead.real.queue队列