RabbitMQ进阶-SprintBoot集成RabbitMQ使用
1.构建项目
1.1 Spring Init创建项目
我们之选了RabbitMQ、Web、Lombok插件简单的几个组成
项目组成
SpringBoot 2.4.4 + RabbitMQ AMQP版本 2.4.4
1.2 新建项目包
项目结构如下所示
-java
--com.jzj.bootmqtest
---bean
---config
--controller
--rabbitmq
--task
--BootmqtestApplication
-resources
--static
--templates
--application.yml
2.初始化RabbitMQ
2.1 项目配置Pom文件及application.yml
Pom文件
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.4.4</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.jzj</groupId>
<artifactId>bootmqtest</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>bootmqtest</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.amqp</groupId>
<artifactId>spring-rabbit-test</artifactId>
<scope>test</scope>
</dependency>
<!-- hutool 工具类-->
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>5.6.1</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<excludes>
<exclude>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</exclude>
</excludes>
</configuration>
</plugin>
</plugins>
</build>
</project>
application.yml 配置
#项目端口
server:
port: 9672
spring:
#应用名称
application:
name: jzj-sprintboot-mqtest
#mq配置
rabbitmq:
addresses: 127.0.0.1
port: 5672
username: guest
password: guest
#开启消息确认机制 confirms
publisher-confirm-type: correlated
publisher-returns: true
#采用手动应答方式
listener:
simple:
acknowledge-mode: manual
2.2 新建实体Bean
在com.jaj.bootmqtest.bean下新建Student实体Bean,用来作为消息体,发送Object的消息体
package com.jzj.bootmqtest.bean;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.Setter;
@Setter
@Getter
@AllArgsConstructor
public class Student {
private Integer id;
private String name;
private Integer age;
}
在controller下面新建测试SiteController,用来测试消息
package com.jzj.bootmqtest.controller;
import com.jzj.bootmqtest.bean.Student;
import com.jzj.bootmqtest.config.QueueConfig;
import com.jzj.bootmqtest.rabbitmq.BaseMqProducer;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;
import javax.annotation.Resource;
@ResponseBody
@RestController
@RequestMapping("test")
public class SiteController {
@Resource
private BaseMqProducer mqProducer;
@RequestMapping("ping")
public Object ping() {
return "pong";
}
@RequestMapping("sendA")
public Object mqtestA() {
Student student = new Student(1, "jiazijie-AAA", 18);
mqProducer.sendMsg(student, QueueConfig.RK_QUEUE_TEST_A);
return "AAA";
}
@RequestMapping("sendB")
public Object mqtestB() {
Student student = new Student(2, "jiazijie-BBB", 30);
mqProducer.sendMsg(student, QueueConfig.RK_QUEUE_TEST_B);
return "BBB";
}
}
2.3 RabbitMQ配置
2.3.1 Exchange交换机配置
ExchangeConfig.java
package com.jzj.bootmqtest.config;
import org.springframework.amqp.core.DirectExchange;
import org.springframework.amqp.core.FanoutExchange;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class ExchangeConfig {
public static final String MAIN_EXCHANGE = "mainExchange";
public static final String DEAD_EXCHANGE = "deadExchange";
/**
* 声明一个主题交换机,作为默认交换机 ,名字: mainExchange
*
* @return
*/
@Bean(name = MAIN_EXCHANGE)
public DirectExchange mainExchange() {
return new DirectExchange(MAIN_EXCHANGE);
}
/**
* 声明一个直连交换机、作为死亡交换机,名字deadExchange,为了方便接收消息,我们声明fanout类型,有消息就转发
*
* @return
*/
@Bean(name = DEAD_EXCHANGE)
public FanoutExchange deadExchange() {
return new FanoutExchange(DEAD_EXCHANGE);
}
}
2.3.2 Queue绑定信息
QueueConfig.java
package com.jzj.bootmqtest.config;
import org.springframework.amqp.core.*;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.HashMap;
import java.util.Map;
import static com.jzj.bootmqtest.config.ExchangeConfig.DEAD_EXCHANGE;
import static com.jzj.bootmqtest.config.ExchangeConfig.MAIN_EXCHANGE;
@Configuration
public class QueueConfig {
public static final String QUEUE_TEST_A = "queue_test_AAA";
public static final String RK_QUEUE_TEST_A = "rk.queue_test_AAA";
public static final String QUEUE_TEST_B = "queue_test_BBB";
public static final String RK_QUEUE_TEST_B = "rk.queue_test_BBB";
@Autowired
@Qualifier(MAIN_EXCHANGE)
DirectExchange mainExchange;
@Autowired
@Qualifier(DEAD_EXCHANGE)
FanoutExchange deadExchange;
/**
* 声明一个
*
* @return
*/
@Bean
@Qualifier("queueAAA")
public Queue queueAAA() {
//声明 队列名称及队列持久化
return new Queue(QUEUE_TEST_A);
}
/**
* 声明队列,绑定死信队列交换机参数
*/
@Bean
@Qualifier("queueBBB")
public Queue queueBBB() {
Map<String, Object> args = new HashMap<>();
args.put("x-dead-letter-exchange",DEAD_EXCHANGE);
args.put("x-dead-letter-routing-key", RK_QUEUE_TEST_B);
return new Queue(QUEUE_TEST_B, true, false, false, args);
}
/**
* 声明对立绑定的Exchange 及 Routingkey
*
* @return
*/
@Bean
public Binding bindingQueueTest1() {
return BindingBuilder.bind(queueAAA()).to(mainExchange).with(RK_QUEUE_TEST_A);
}
/**
* 声明队列绑定,绑定 mainExchange 及 RoutingKey
*/
@Bean
public Binding bindingQueueTest2() {
return BindingBuilder.bind(queueBBB()).to(mainExchange).with(RK_QUEUE_TEST_B);
}
/**
* 声明死信队列,用于接收 死信交换机过来的消息
*/
@Bean
@Qualifier("deadQueue")
public Queue deadQueue() {
return new Queue("dead_queue", true);
}
/**
* 声明死信队列绑定,只要是rk开头的全都接收
*/
@Bean
public Binding bindingDeadQueue() {
return BindingBuilder.bind(deadQueue()).to(deadExchange);
}
}
2.3.3 生产者
在此rabbitmq包下,新建生产者,用基类生产者来定义MQ生产者,消息发送时候只需要注入 BaseMqConsumer就可以了
BaseMqProducer.java
package com.jzj.bootmqtest.rabbitmq;
import cn.hutool.json.JSONUtil;
import com.jzj.bootmqtest.task.SendMqTask;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.AmqpException;
import org.springframework.amqp.core.AmqpTemplate;
import org.springframework.amqp.core.DirectExchange;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.core.MessageBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.task.TaskRejectedException;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
import org.springframework.stereotype.Component;
import java.util.Date;
import java.util.UUID;
/**
* <mq消息发送>.
*
* @author peng cheng
* @date 2019/12/23
* @since
*/
@Component
public class BaseMqProducer {
private static final Logger LOGGER = LoggerFactory.getLogger(BaseMqProducer.class);
/**
* 最大线程数目
*/
private static final Integer RETRY_TIMES = 50;
/**
* 消息优先级设置
*/
private Integer msgPriority;
@Autowired
private AmqpTemplate amqpTemplate;
@Autowired
private ThreadPoolTaskExecutor mqSenderExecutor;
@Autowired
private DirectExchange mainExchange;
/**
* 发送消息 采用默认的交换机
*
* @param msg
* @param routingKey
*/
public void sendMsg(Object msg, String routingKey) {
sendMsg(msg, mainExchange.getName(), routingKey);
}
/**
* 发送消息 指定交换机和路由键
*
* @param exchange
* @param routingKey
* @param mqMsg
*/
public void sendMsg(Object mqMsg, String exchange, String routingKey) {
if (null == mqMsg) {
return;
}
try {
Message message = generateMessage(mqMsg);
LOGGER.info("RabbitMQ sendMsg exchange={},routingKey={},messageId={},msgBody={}", exchange, routingKey, message.getMessageProperties().getMessageId(),
new String(message.getBody()));
threadExecute(new SendMqTask(exchange, routingKey, message, amqpTemplate));
} catch (AmqpException e) {
LOGGER.error("booster-notify sendMsq failed exchange=" + exchange + ",routingKey=" + routingKey + ",mqMsg=" + JSONUtil.toJsonStr(mqMsg), e);
}
}
/**
* 消息处理
*
* @param msg
* @return
*/
public Message generateMessage(Object msg) {
String msgBody = null;
try {
msgBody = JSONUtil.toJsonStr(msg);
return MessageBuilder.withBody(msgBody.getBytes())
.setMessageId(UUID.randomUUID().toString())
.setTimestamp(new Date())
.setPriority(msgPriority)
.build();
} catch (Exception e) {
LOGGER.error("generateMessage error :" + e.getMessage(), e);
}
return null;
}
/**
* thread execute
*
* @param
*/
private void threadExecute(Runnable var) {
for (int i = 0; i < RETRY_TIMES; i++) {
try {
mqSenderExecutor.execute(var);
return;
} catch (TaskRejectedException e) {
LOGGER.error("RabbitMQ线程池满请等待,等待次数:" + i + " 错误信息:" + e.getMessage());
// sleep 100 ms
try {
Thread.sleep(100);
} catch (InterruptedException e2) {
LOGGER.error("sleep error! " + e2.getMessage(), e2);
}
}
}
mqSenderExecutor.execute(var);
}
public void setMsgPriority(Integer msgPriority) {
this.msgPriority = msgPriority;
}
}
2.3.4 消费者
我们模板设计模式、制定消费者抽象基类,所有的消费者都继承实现基类就可以了
BaseMqConsumer.java
package com.jzj.bootmqtest.rabbitmq;
import com.rabbitmq.client.Channel;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.core.MessageProperties;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
/**
* 封装消费者抽象类
* 每个消费者只需要关注自己的Listener 及 消息的处理,其余的工作都放在抽象模板中做
*/
public abstract class BaseMqConsumer {
private static final Logger LOGGER = LoggerFactory.getLogger(BaseMqConsumer.class);
/**
* @param message
* @return
* @throws Exception
*/
protected abstract boolean handlerMessage(Message message) throws Exception;
/**
* 获取消息消费异常后,重试的次数
*/
protected abstract int initMaxRetryTimes();
/**
* 处理公共信息
*
* @param channel 通道
* @param message 消息体
*/
public void onMessage(Channel channel, Message message) throws Exception {
String msgBody = new String(message.getBody(), StandardCharsets.UTF_8);
// 获取重试次数
final int maxRetryTimes = initMaxRetryTimes();
String messageId = message.getMessageProperties().getMessageId();
LOGGER.info(String.format("RabbitMQ receive msg, messageId:%s message:%s", messageId, msgBody));
//先设置 处理结果为False
boolean processResult = false;
try {
//开始处理消息
processResult = handlerMessage(message);
} catch (Exception e) {
String errorMsg = e.getMessage() == null ? "" : e.getMessage();
LOGGER.error(String.format("Rabbitmq Comsumer process error, messageId:%s,messageBody:%s, errorMsg:%s", messageId, msgBody, errorMsg), e);
} finally {
Long retryNum = getRetryCountFromMessage(message.getMessageProperties());
if (retryNum > maxRetryTimes) {
processResult = true;
LOGGER.error(String.format("Rabbitmq 当前消息体重试操作超过[" + maxRetryTimes + "]次, 放弃重试!, messageId:%s retryNum:%s message:%s", messageId, retryNum, msgBody));
}
ackHandler(channel, message, processResult);
}
}
protected void ackHandler(Channel channel, Message message, boolean processResult) throws IOException {
if (processResult) {
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
} else {
// 不成功, 丢到dead letter (second param true:requeue false:dead letter)
channel.basicReject(message.getMessageProperties().getDeliveryTag(), false);
}
}
private Long getRetryCountFromMessage(MessageProperties properties) {
//此处最好把 Message中的消息ID放入Redis,用分布式锁获取Redis的值 来判断当前的MessageId来到过几次了,用来解决幂等性
// 方便测试,后面我们测试时候,设置为3次,表示这是第3次到来,防止消息无线消费,导致CPU暴增
return 0L;
}
}
消费者A,对应A队列 QueueAListener.java
package com.jzj.bootmqtest.rabbitmq;
import cn.hutool.json.JSONUtil;
import com.jzj.bootmqtest.bean.Student;
import com.jzj.bootmqtest.config.QueueConfig;
import com.rabbitmq.client.Channel;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitHandler;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;
@Component
public class QueueAListener extends BaseMqConsumer {
private static final Logger LOGGER = LoggerFactory.getLogger(QueueAListener.class);
/**
* 绑定队列
*/
@Override
@RabbitHandler
@RabbitListener(queues = QueueConfig.QUEUE_TEST_A)
public void onMessage(Channel channel, Message message) throws Exception {
super.onMessage(channel, message);
}
@Override
protected int initMaxRetryTimes() {
return 1;
}
@Override
protected boolean handlerMessage(Message message) {
boolean result = false;
try {
result = processSendResult(message);
} catch (Exception e) {
LOGGER.error("邮件消息 处理 异常 ", e);
}
return result;
}
/**
* 解析邮件发送结果,且尝试计数
* <p>
* 此处 返回的都是 正确发送的消息,失败重试后 避免重新路由
*
* @param message 邮件消息
* @return 处理结果
*/
private boolean processSendResult(Message message) {
// 解析MQ 邮件消息
String msgBody = new String(message.getBody());
Student msg = JSONUtil.toBean(msgBody, Student.class);
if (null == msg) {
LOGGER.error("转化失败 msg:{}", msgBody);
throw new RuntimeException("邮件消息解析失败");
}
LOGGER.info("AAA处理消息:" + msgBody);
return true;
}
}
消费者B,对应B队列 QueueBListener.java
package com.jzj.bootmqtest.rabbitmq;
import cn.hutool.json.JSONUtil;
import com.jzj.bootmqtest.bean.Student;
import com.jzj.bootmqtest.config.QueueConfig;
import com.rabbitmq.client.Channel;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitHandler;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;
@Component
public class QueueBListener extends BaseMqConsumer {
private static final Logger LOGGER = LoggerFactory.getLogger(QueueBListener.class);
/**
* 绑定队列
*/
@Override
@RabbitHandler
@RabbitListener(queues = QueueConfig.QUEUE_TEST_B)
public void onMessage(Channel channel, Message message) throws Exception {
super.onMessage(channel, message);
}
@Override
protected int initMaxRetryTimes() {
return 2;
}
@Override
protected boolean handlerMessage(Message message) {
boolean result = false;
try {
result = processSendResult(message);
} catch (Exception e) {
LOGGER.error("邮件消息 处理 异常 ", e);
}
return result;
}
/**
* 解析邮件发送结果,且尝试计数
* <p>
* 此处 返回的都是 正确发送的消息,失败重试后 避免重新路由
*
* @param message 邮件消息
* @return 处理结果
*/
private boolean processSendResult(Message message) {
// 解析MQ 邮件消息
String msgBody = new String(message.getBody());
Student msg = JSONUtil.toBean(msgBody, Student.class);
if (null == msg) {
LOGGER.error("转化失败 msg:{}", msgBody);
throw new RuntimeException("邮件消息解析失败");
}
//测试异常信息,直接抛出异常
// throw new RuntimeException("邮件消息BBB解析失败");
LOGGER.info("BBB 处理消息:" + msgBody);
return true;
}
}
2.3.5 MQ任务
设计SendMqTask 用来接收MQ消息发送者的消息,实现消息的发送
package com.jzj.bootmqtest.task;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.amqp.AmqpException;
import org.springframework.amqp.core.AmqpTemplate;
import org.springframework.amqp.core.Message;
/**
* 单独发送的线程
*/
public class SendMqTask implements Runnable {
private static final Logger LOGGER = LoggerFactory.getLogger(SendMqTask.class);
/**
* 交换机
*/
private String exchange;
/**
* Routingkey
*/
private String rk;
/**
* 发送的消息
*/
private Message message;
/**
* amqpTemplate
*/
private AmqpTemplate amqpTemplate;
public SendMqTask(String exchange, String rk, Message message, AmqpTemplate amqpTemplate) {
this.exchange = exchange;
this.rk = rk;
this.message = message;
this.amqpTemplate = amqpTemplate;
}
@Override
public void run() {
amqpSend(exchange, rk, message);
}
/**
* amqp发送封装逻辑
*
* @param rk
* @param message
*/
private void amqpSend(final String exchange, final String rk, final Message message) {
try {
amqpTemplate.convertAndSend(exchange, rk, message);
} catch (AmqpException e) {
LOGGER.info(String.format("send error!!! messageId:%s message:%s",
message.getMessageProperties().getMessageId(), new String(message.getBody())));
LOGGER.error("send message error: " + e.getMessage(), e);
}
}
}
2.4 启动类
启动类BootmqtestApplication.java
package com.jzj.bootmqtest;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class BootmqtestApplication {
public static void main(String[] args) {
SpringApplication.run(BootmqtestApplication.class, args);
}
}
3.启动测试
3.1 项目启动
项目成功启动
3.2 Rabbitmq界面
交换机信息如下:
3个队列信息
3.3 执行结果
执行Controller sendA 给A队列发一条消息 curl 127.0.0.1:9672/test/sendA
执行结果可以看到,先发送消息,然后Listener接收消息,然后处理消息,处理成功消费完毕,B队列也一样
3.4 异常测试
3.4.1 basicReject 拒绝测试
现在基类BaseMqConsumer中 逻辑是消息处理成功,就确认消息,不成功就Reject消息
而且Reject中的参数requeue是false,表明拒绝后到私信队列,我们制造个异常,让他处理失败,看看
修改B的Listener的processSendResult方法,因为B有死信队列配置,让他抛出异常,处理失败,走下面的else分支
private boolean processSendResult(Message message) {
// 解析MQ 邮件消息
String msgBody = new String(message.getBody());
Student msg = JSONUtil.toBean(msgBody, Student.class);
if (null == msg) {
LOGGER.error("转化失败 msg:{}", msgBody);
throw new RuntimeException("邮件消息解析失败");
}
//测试异常信息,直接抛出异常
throw new RuntimeException("邮件消息BBB解析失败");
// LOGGER.info("BBB 处理消息:" + msgBody);
// return true;
}
重启项目,再次发送sendB消息,发送完、接收消息,处理失败,抛出异常
死信队列中接收到消息
3.4.2 basicReject 拒绝、重新路由测试
我们可以看到 channel.basicReject(message.getMessageProperties().getDeliveryTag(), false);
消息拒绝后是 requeue是false,表明不再路由,我们改为true看看,是否会重新路由
!!!!!!! 注意,此处会有死循环,抛出异常->basicReject->重新路由->再次消费->再抛出异常…无限循环
protected void ackHandler(Channel channel, Message message, boolean processResult) throws IOException {
if (processResult) {
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
} else {
// 不成功, 丢到dead letter (second param true:requeue false:dead letter)
channel.basicReject(message.getMessageProperties().getDeliveryTag(), true);
}
}
重启项目再次发送B
我们现在把循环次数变为3尝试一下,重启,可以看到,消息消费了3次后,就自动丢弃了,不会再无限循环
private Long getRetryCountFromMessage(MessageProperties properties) {
//此处最好把 Message中的消息ID放入Redis,用分布式锁获取Redis的值 来判断当前的MessageId来到过几次了,用来解决幂等性
// 方便测试,后面我们测试时候,设置为3次,表示这是第3次到来,防止消息无线消费,导致CPU暴增
return 3L;
}
重试结果,Rabbitmq 当前消息体重试操作超过[2]次, 放弃重试!
4.常见错误
4.1 reply-text=PRECONDITION_FAILED - unknown delivery tag 1
2021-04-06 21:46:41.708 ERROR 7560 — [ 127.0.0.1:5672] o.s.a.r.c.CachingConnectionFactory : Shutdown Signal: channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - unknown delivery tag 1, class-id=60, method-id=80)
该错误是因为 没有配置手动消息确认导致的,需要在配置文件application.yml中新增
publisher-confirm-type: correlated
publisher-returns: true
#采用手动应答方式
listener:
simple:
acknowledge-mode: manual
spring:
#应用名称
application:
name: jzj-sprintboot-mqtest
#mq配置
rabbitmq:
addresses: 127.0.0.1
port: 5672
username: guest
password: guest
#开启消息确认机制 confirms
publisher-confirm-type: correlated
publisher-returns: true
#采用手动应答方式
listener:
simple:
acknowledge-mode: manual
4.2 spring.rabbitmq.publisher-confirms过时解决
该错误是因为 rabbitmq升级后,这对消息确认的方式变了,将原来的publisher-confirms 修改为 publisher-confirm-type 配置
publisher-confirm-type: correlated
至此 我们的 Rabbitmq结合SpringBoot 简单完毕,下一节我们讲一下 批量发送消息