三、RabbitMQ高级特性
7. 消费端ACK与重回队列
7.1 消费端的手工ACK和NACK
消费端进行消费的时候,如果由于业务异常我们可以进行日志的记录,然后进行补偿!
如果由于服务器宕机
等严重问题,那我们就需要手工进行ACK
保障消费端消费成功!
7.2 消费端的重回队列
消费端重回队列
是为了对没有处理成功
的消息,把消息重新传递给Broker
!
一般我们在实际应用中,都会关闭重回队列
,也就是设置为False.
7.3 代码实例
/**
*生产者
**/
public class Producer {
public static void main(String[] args) throws Exception {
//1创建ConnectionFactory
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
String exchange = "test_ack_exchange";
String routingKey = "ack.save";
for(int i =0; i<5; i ++){
Map<String, Object> headers = new HashMap<String, Object>();
headers.put("num", i);
//添加属性,后续会使用到
AMQP.BasicProperties properties = new AMQP.BasicProperties.Builder()
.deliveryMode(2) //投递模式,持久化
.contentEncoding("UTF-8")
.headers(headers)
.build();
String msg = "Hello RabbitMQ ACK Message " + i;
channel.basicPublish(exchange, routingKey, true, properties, msg.getBytes());
}
}
}
//-------------------------------------------------------------------------------
/**
* 消费者
**/
public class Consumer {
public static void main(String[] args) throws Exception {
//1创建ConnectionFactory
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
String exchangeName = "test_ack_exchange";
String queueName = "test_ack_queue";
String routingKey = "ack.#";
channel.exchangeDeclare(exchangeName, "topic", true, false, null);
channel.queueDeclare(queueName, true, false, false, null);
channel.queueBind(queueName, exchangeName, routingKey);
// 手工签收 必须要关闭 autoAck = false
channel.basicConsume(queueName, false, new MyConsumer(channel));
}
}
//-------------------------------------------------------------------------------
/**
* 自定义类:MyConsumer
**/
public class MyConsumer extends DefaultConsumer {
private Channel channel ;
public MyConsumer(Channel channel) {
super(channel);
this.channel = channel;
}
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.err.println("-----------consume message----------");
System.err.println("body: " + new String(body));
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
if((Integer)properties.getHeaders().get("num") == 0) {
//Nack三个参数 第二个参数:是否是批量,第三个参数:是否重回队列(需要注意可能会发生重复消费,造成死循环)
channel.basicNack(envelope.getDeliveryTag(), false, true);
} else {
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
}
//------------------------------------------------------------------------------------
**
* 连接工具类
**/
public class ConnectionUtils {
public static Connection getConnection() throws IOException, TimeoutException {
//定义连接工厂
ConnectionFactory factory = new ConnectionFactory();
//设置服务地址
factory.setHost("localhost");
//端口
factory.setPort(5672);//amqp协议 端口 类似与mysql的3306
//设置账号信息,用户名、密码、vhost
factory.setVirtualHost("/");
factory.setUsername("gn");
factory.setPassword("123456");
// 通过工程获取连接
Connection connection = factory.newConnection();
return connection;
}
}
7.4 控制台打印结果
注意:
可以看到重回队列会出现重复消费导致死循环的问题,这时候最好设置重试次数,比如超过三次后,消息还是消费失败,就将消息丢弃。
8. TTL队列/消息
8.1 TTL
TTL
是Time To Live的缩写,也就是生存时间
- RabbitMQ支持
消息``的
过期时间,在消息发送时可以进行
指定``` - RabbitMQ支持
队列
的过期时间
,从消息入队列开始计算,只要超过了队列的``超时时间```配置,那么消息会自动的清除
8.2 代码实例 (直接管控台添加)
-
添加queue
Arguments中填写x-max-length=3000,x-message-ttl =10000,名字test_ttl
通过管控台创建一个队列
x-max-length
队列的最大大小
x-message-ttl
设置10秒钟,如果消息还没有被消费的话,就会被清除。 -
添加exchange
虚拟主机、名称test_ttl_exchange
、类型topic
、持久化
-
Queue与Exchange进行绑定
点击test_ttl_exchange
进行绑定 -
查看是否绑定成功
点击exchanges下Bindings下的Bind按钮
-
通过管控台发送消息
-
消息未处理自动清除
-
生产端设置过期时间
AMQP.BasicProperties properties = new AMQP.BasicProperties.Builder()
.deliveryMode(2)
.contentEncoding("UTF-8")
.expiration("10000")
.headers(headers)
.build();
这两个属性并不相同,一个对应的是消息体,一个对应的是队列的过期。
9. 死信队列
9.1 what 死信队列(DLX,Dead-Letter-Exchange)
RabbitMQ的死信队里与Exchange相关
- 利用DLX,当消息在一个队列中变成死信(dead message)之后,它能被重新publish到另一个Exchange,这个Exchange就是DLX
消息变成死信有以下几种情况
- 消息被拒绝(basic.reject/basic.nack)并且requeue=false
- 消息TTL过期
- 队列达到最大长度
DLX也是一个正常的Exchange,和一般的Exchange没有区别,它能在任何的队列上被指定,实际上就是设置某个队列的属性
当这个队列中有死信时,RabbitMQ就会自动的将这个消息重新发布到设置的Exchange上去,进而被路由到另一个队列。
可以监听这个队列中消息做相应的处理,这个特征可以弥补RabbitMQ3.0以前支持的immediate参数的功能。
9.2 代码实例
死信队列设置:
- 首先需要设置死信队列的exchange和queue,然后进行绑定:
Exchange:dlx.exchange
Queue:dlx.queue
RoutingKey:# - 然后我们进行正常声明交换机、队列、绑定,只不过我们需要在队列加上一个参数即可:arguments.put(“x-dead-letter-exchange”,“dlx.exchange”);
- 这样消息在过期、requeue、队列在达到最大长度时,消息就可以直接路由到死信队列!
/**
* 生产者
**/
public class Producer {
public static void main(String[] args) throws Exception {
//创建ConnectionFactory
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
String exchange = "test_dlx_exchange";
String routingKey = "dlx.save";
String msg = "Hello RabbitMQ DLX Message";
for(int i =0; i<1; i ++){
AMQP.BasicProperties properties = new AMQP.BasicProperties.Builder()
.deliveryMode(2)
.contentEncoding("UTF-8")
.expiration("10000")
.build();
channel.basicPublish(exchange, routingKey, true, properties, msg.getBytes());
}
}
}
//--------------------------------------------------------------------------------
/**
* 消费者
**/
public class Consumer {
public static void main(String[] args) throws Exception {
//创建ConnectionFactory
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
// 这就是一个普通的交换机 和 队列 以及路由
String exchangeName = "test_dlx_exchange";
String routingKey = "dlx.#";
String queueName = "test_dlx_queue";
channel.exchangeDeclare(exchangeName, "topic", true, false, null);
Map<String, Object> agruments = new HashMap<String, Object>();
agruments.put("x-dead-letter-exchange", "dlx.exchange");
//这个agruments属性,要设置到声明队列上
channel.queueDeclare(queueName, true, false, false, agruments);
channel.queueBind(queueName, exchangeName, routingKey);
//要进行死信队列的声明:
channel.exchangeDeclare("dlx.exchange", "topic", true, false, null);
channel.queueDeclare("dlx.queue", true, false, false, null);
channel.queueBind("dlx.queue", "dlx.exchange", "#");
channel.basicConsume(queueName, true, new MyConsumer(channel));
}
}
//-------------------------------------------------------------------------------
/**
* 自定义类:MyConsumer
**/
public class MyConsumer extends DefaultConsumer {
public MyConsumer(Channel channel) {
super(channel);
}
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.err.println("-----------consume message----------");
System.err.println("consumerTag: " + consumerTag);
System.err.println("envelope: " + envelope);
System.err.println("properties: " + properties);
System.err.println("body: " + new String(body));
}
}
//------------------------------------------------------------------------------------
**
* 连接工具类
**/
public class ConnectionUtils {
public static Connection getConnection() throws IOException, TimeoutException {
//定义连接工厂
ConnectionFactory factory = new ConnectionFactory();
//设置服务地址
factory.setHost("localhost");
//端口
factory.setPort(5672);//amqp协议 端口 类似与mysql的3306
//设置账号信息,用户名、密码、vhost
factory.setVirtualHost("/");
factory.setUsername("gn");
factory.setPassword("123456");
// 通过工程获取连接
Connection connection = factory.newConnection();
return connection;
}
}
9.3 测试结果
1.运行Consumer,查看管控台Exchanges,包含一个名为dlx.exchange,test_dlx_exchange的死信队列
2.查看Queues,可以看到test_dlx_queue多了DLX的标识,表示当队列中出现死信的时候,会将消息发送到死信队列dlx_queue中
3.关闭Consumer,只运行Producer,过10秒钟后,消息过期,test_dlx_queue中没有消息,dlx.queue增加一条
4.在工作中,死信队列非常重要,用于消息没有消费者,处于死信状态。我们可以才用补偿机制
四、 SpringBoot整合RabbitMQ
1. SpringBoot整合配置详解
-
publisher-confirms,实现一个监听器用于监听Broker端给我们返回的确认请求:RabbitTemplate.ConfirmCallback
-
publisher-returns,保证消息对Broker端是可达的,如果出现路由键不可达的情况,则使用监听器对不可达的消息进行后续的处理,保证消息的路由成功:RabbitTemplate.ReturnCallback
-
在发送消息的时候对template进行配置mandatory=true保证监听有效
-
生产端还可以配置其他属性,比如发送重试,超时时间,次数,间隔等
2.代码实例
2.1 生产端实例
1.新建项目springboot-producer
//pom 依赖包
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.gao</groupId>
<artifactId>springboot-producer</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>springboot-producer</name>
<description>springboot-producer</description>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.4.RELEASE</version>
<relativePath/>
<!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
//------------------------------------------------------------------------------------
/**
* 消息生产者
**/
@Component
public class RabbitSender {
//自动注入RabbitTemplate模板类
@Autowired
private RabbitTemplate rabbitTemplate;
//回调函数: confirm确认
final ConfirmCallback confirmCallback = new RabbitTemplate.ConfirmCallback() {
@Override
public void confirm(CorrelationData correlationData, boolean ack, String cause) {
System.err.println("correlationData: " + correlationData);
System.err.println("ack: " + ack);
if(!ack){
//可以进行日志记录、异常处理、补偿处理等
System.err.println("异常处理....");
}else {
//更新数据库,可靠性投递机制
}
}
};
//回调函数: return返回
final ReturnCallback returnCallback = new RabbitTemplate.ReturnCallback() {
@Override
public void returnedMessage(org.springframework.amqp.core.Message message, int replyCode, String replyText,
String exchange, String routingKey) {
System.err.println("return exchange: " + exchange + ", routingKey: "
+ routingKey + ", replyCode: " + replyCode + ", replyText: " + replyText);
}
};
//发送消息方法调用: 构建Message消息
public void send(Object message, Map<String, Object> properties) throws Exception {
MessageHeaders mhs = new MessageHeaders(properties);
Message msg = MessageBuilder.createMessage(message, mhs);
rabbitTemplate.setConfirmCallback(confirmCallback);
rabbitTemplate.setReturnCallback(returnCallback);
//id + 时间戳 全局唯一 用于ack保证唯一一条消息,这边做测试写死一个。但是在做补偿策略的时候,必须保证这是全局唯一的消息
CorrelationData correlationData = new CorrelationData("1234567890");
rabbitTemplate.convertAndSend("exchange-1", "springboot.abc", msg, correlationData);
}
}
//---------------------------------------------------------------------------------------
/**
* application.properties
**/
spring.rabbitmq.addresses=localhost:5672
spring.rabbitmq.username=guest
spring.rabbitmq.password=guest
spring.rabbitmq.virtual-host=/vhost_cp
spring.rabbitmq.connection-timeout=15000
spring.rabbitmq.publisher-confirms=true
spring.rabbitmq.publisher-returns=true
spring.rabbitmq.template.mandatory=true
2.2 管控台
1.添加Exchanges
2.添加Queue
3.Exchange绑定Queue
修改routingKey,springboot改为spring,则进入的是returnCallback方法
2.3 解决ack为false问题
当方法结束,rabbitmq相关的资源也就关闭了,虽然消息发送出去,但异步的ConfirmCallback却由于资源关闭而出现了上面的问题。加入Thread.sleep()即可解决。
@Test
public void testSender1() throws Exception {
Map<String, Object> properties = new HashMap<>();
properties.put("number", "12345");
properties.put("send_time", simpleDateFormat.format(new Date()));
rabbitSender.send("Hello RabbitMQ For Spring Boot!", properties);
Thread.sleep(2000);
}
2.4 消费端实例
1.消费端核心配置:
- 签收模式-手工签收
spring.rabbitmq.listener.simple.acknowledge-mode=manual
- 设置监听限制:最大10,默认5
spring.rabbitmq.listener.simple.concurrency=5
spring.rabbitmq.listener.simple.max-concurrency=10
-
首先配置手工确认模式,用于ACK的手工处理,这样我们可以保证消息的可靠性送达,或者再消费端消费失败的时候可以做到重回队列(不建议)、根据业务记录日志等处理。
-
可以设置消费端的监听个数和最大个数,用于监控消费端的并发情况
2.@RabbitListener注解使用
- 消费端监听@RabbitListener注解,这个对于在实际工作中非常的好用
- @RabbitListener是一个组合注解,里面可以注解配置
- @QueueBinding、@Queue、@Exchange直接通过这个组合注解一次性搞定消费端交换机、队列、绑定、路由、并且配置监听功能等。
@RabbitListener(bindings=@QueueBinding(
value = @Queue(value = "queue-1",durable = "true"),
exchange = @Exchange(value = "exchange-1",
durable = "true",
type = "topic",
ignoreDeclarationException = "true"),
key = "springboot.*")
@RabbitHandler
public void onMessage(Message message,Channel channel){
}
在方法onMessage上加@RabbitListener注解,同时需要加另外一个注解@RabbitHandler,代码被消费者监听。
建立绑定,在Value上写上队列,设置Exchange,是否持久化,设置Exchange的类型、表达式设置为true以及路由key。通过这种简单的方式,可以完成之前很复杂的代码逻辑。同时建议将配置放入到配置文件中,动态获取。如果mq中没有相应的队列、Exchange等,注解声明也可以创建它们。
3.新建项目springboot-consumer
//pom 依赖包
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.gao</groupId>
<artifactId>springboot-consumer</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>springboot-consumer</name>
<description>springboot-consumer</description>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.2.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
//-----------------------------------------------------------------------------------------------------
/**
* 消息生产者
**/
@Component
public class RabbitReceiver {
@RabbitListener(bindings = @QueueBinding(
value = @Queue(value = "queue-1",
durable="true"),
exchange = @Exchange(value = "exchange-1",
durable="true",
type= "topic",
ignoreDeclarationExceptions = "true"),
key = "springboot.*"
)
)
@RabbitHandler
public void onMessage(Message message, Channel channel) throws Exception {
System.err.println("--------------------------------------");
System.err.println("消费端Payload: " + message.getPayload());
Long deliveryTag = (Long)message.getHeaders().get(AmqpHeaders.DELIVERY_TAG);
//手工ACK,获取deliveryTag
channel.basicAck(deliveryTag, false);
}
}
//-------------------------------------------------------------------------------------
/**
* application.properties
**/
spring.rabbitmq.addresses=localhost:5672
spring.rabbitmq.username=user_gn
spring.rabbitmq.password=123456
spring.rabbitmq.virtual-host=/vhost_gn
spring.rabbitmq.connection-timeout=15000
spring.rabbitmq.listener.simple.acknowledge-mode=manual
spring.rabbitmq.listener.simple.concurrency=5
spring.rabbitmq.listener.simple.max-concurrency=10
运行Application,查看之前在生产端发送的消息,是否能被消费
2.5 结果
3. 代码优化
- 自定义Java对象消息
- @RabbitListener注解中的配置改为动态配置
@Payload:指定具体的消息体Body。
@Headers: 获取Headers。
3.1 消费端优化
- 传输对象的时候,必须序列化。否则会传输失败
- 已经将配置写入到了application.properties中,进行动态获取
/**
* 定义一个Order对象
**/
public class Order implements Serializable {
private String id;
private String name;
public Order() {
}
public Order(String id, String name) {
super();
this.id = id;
this.name = name;
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
//-------------------------------------------------------------------------------------
/**
* RabbitReceiver添加监听
* spring.rabbitmq.listener.order.queue.name=queue-2
* spring.rabbitmq.listener.order.queue.durable=true
* spring.rabbitmq.listener.order.exchange.name=exchange-2
* spring.rabbitmq.listener.order.exchange.durable=true
* spring.rabbitmq.listener.order.exchange.type=topic
* spring.rabbitmq.listener.order.exchange.ignoreDeclarationExceptions=true
* spring.rabbitmq.listener.order.key=springboot.*
* @param order
* @param channel
* @param headers
* @throws Exception
*/
@RabbitListener(bindings = @QueueBinding(
value = @Queue(value = "${spring.rabbitmq.listener.order.queue.name}",
durable="${spring.rabbitmq.listener.order.queue.durable}"),
exchange = @Exchange(value = "${spring.rabbitmq.listener.order.exchange.name}",
durable="${spring.rabbitmq.listener.order.exchange.durable}",
type= "${spring.rabbitmq.listener.order.exchange.type}",
ignoreDeclarationExceptions = "${spring.rabbitmq.listener.order.exchange.ignoreDeclarationExceptions}"),
key = "${spring.rabbitmq.listener.order.key}"
)
)
@RabbitHandler
public void onOrderMessage(@Payload com.cp.springboot.entity.Order order,
Channel channel,
@Headers Map<String, Object> headers) throws Exception {
System.err.println("--------------------------------------");
System.err.println("消费端order: " + order.getId());
Long deliveryTag = (Long)headers.get(AmqpHeaders.DELIVERY_TAG);
//手工ACK
channel.basicAck(deliveryTag, false);
}
/**
* application.properties
**/
spring.rabbitmq.listener.order.queue.name=queue-2
spring.rabbitmq.listener.order.queue.durable=true
spring.rabbitmq.listener.order.exchange.name=exchange-2
spring.rabbitmq.listener.order.exchange.durable=true
spring.rabbitmq.listener.order.exchange.type=topic
spring.rabbitmq.listener.order.exchange.ignoreDeclarationExceptions=true
spring.rabbitmq.listener.order.key=springboot.*
3.2 生产端优化
- 同样是一个Order对象,必须跟消费端的保持一致
- RabbitSender添加发送消息
//发送消息方法调用: 构建自定义对象消息
public void sendOrder(Order order) throws Exception {
rabbitTemplate.setConfirmCallback(confirmCallback);
rabbitTemplate.setReturnCallback(returnCallback);
//id + 时间戳 全局唯一
CorrelationData correlationData = new CorrelationData("0987654321");
rabbitTemplate.convertAndSend("exchange-2", "springboot.def", order, correlationData);
}
- 添加测试方法
@Test
public void testSender() throws Exception {
Order order = new Order("001", "第一个订单");
rabbitSender.sendOrder(order);
//防止资源提前关闭,ConfirmCallback异步回调失败
Thread.sleep(2000);
}
4.结果
运行testSender()方法。
生产端打印:
消费端打印: