084:RabbitMQ实战解决方案之死信队列
1 消息中间件产生了消息堆积如何解决
课题内容
- RabbitMQ消息产生了堆积如何解决
- RabbitMQ如何保证消息不丢失
- RabbitMQ实战操作死信队列
- RabbitMQ异步如何获取消费结果
RabbitMQ如果产生了消息堆积如何处理?
产生的背景:如果没有及时的消费者消费消息,生产者一直不断往队列服务器存放消息,最终会导致消息堆积。
两种场景:
- 没有消费者消费的情况下:死信队列、设置消息有效期
相当于对消息设置有效期,在规定的时间内如果没有消费的话,自动过期,过期的时候会执行客户端回调监听的方法将消息存放到数据库记录,后期实现补偿。 - 有一个消费者消费的情况:应该提高消费能力,消费者实现集群
2 RabbitMQ如何保证消息不丢失
RabbitMQ如何彻底保证我们的消息不丢失?
1.MQ服务器端应该消息持久化到硬盘
2.生产者使用消息确认机制百分百能够将消息投递到MQ成功
3.消费者使用手动ack机制确认消息百分百消费成功
如果队列容量满了,再继续投递可能会丢失。解决:死信队列
死信队列:备胎队列,消息中间件队列因为某种原因拒绝存放该消息,可以转移到死信队列中存放。
死信队列产生的背景:
1.生产者投递消息到MQ中,消息过期了;
2.队列已经达到最大长度(队列存放消息满了)MQ拒绝接受存放该消息;
3.消费者多次消费该消息失败的情况,也会存放到死信队列;
3 RabbitMQ整合死信队列
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.0.RELEASE</version>
</parent>
<dependencies>
<!-- springboot-web组件 -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- 添加springboot对amqp的支持 -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</dependency>
<!--fastjson -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.49</version>
</dependency>
</dependencies>
application.yml
spring:
rabbitmq:
####连接地址
host: 127.0.0.1
####端口号
port: 5672
####账号
username: guest
####密码
password: guest
### 地址
virtual-host: /mayikt_rabbitmq
server:
port: 8080
###模拟演示死信队列
mayikt:
### 创建死信队列交换机和队列路由键
dlx:
exchange: mayikt_order_dlx_exchange
queue: mayikt_order_dlx_queue
routingKey: dlx
###创建订单队列交换机和队列路由键
order:
exchange: mayikt_order_exchange
queue: mayikt_order_queue
routingKey: mayikt.order
配置死信队列
@Component
public class DeadLetterMQConfig {
/**
* 订单交换机
*/
@Value("${mayikt.order.exchange}")
private String orderExchange;
/**
* 订单队列
*/
@Value("${mayikt.order.queue}")
private String orderQueue;
/**
* 订单路由key
*/
@Value("${mayikt.order.routingKey}")
private String orderRoutingKey;
/**
* 死信交换机
*/
@Value("${mayikt.dlx.exchange}")
private String dlxExchange;
/**
* 死信队列
*/
@Value("${mayikt.dlx.queue}")
private String dlxQueue;
/**
* 死信路由
*/
@Value("${mayikt.dlx.routingKey}")
private String dlxRoutingKey;
/**
* 声明死信交换机
*
* @return DirectExchange
*/
@Bean
public DirectExchange dlxExchange() {
return new DirectExchange(dlxExchange);
}
/**
* 声明死信队列
*
* @return Queue
*/
@Bean
public Queue dlxQueue() {
return new Queue(dlxQueue);
}
/**
* 声明订单业务交换机
*
* @return DirectExchange
*/
@Bean
public DirectExchange orderExchange() {
return new DirectExchange(orderExchange);
}
/**
* 声明订单队列
*
* @return Queue
*/
@Bean
public Queue orderQueue() {
Map<String, Object> arguments = new HashMap<>(2);
// 绑定死信交换机
arguments.put("x-dead-letter-exchange", dlxExchange);
// 绑定路由key
arguments.put("x-dead-letter-routing-key", dlxRoutingKey);
return new Queue(orderQueue, true, false, false, arguments);
}
/**
* 绑定订单队列到订单交换机
*
* @return Binding
*/
@Bean
public Binding orderBinding() {
return BindingBuilder.bind(orderQueue())
.to(orderExchange())
.with(orderRoutingKey);
}
/**
* 绑定死信队列到死信交换机
*
* @return Binding
*/
@Bean
public Binding binding() {
return BindingBuilder.bind(dlxQueue())
.to(dlxExchange())
.with(dlxRoutingKey);
}
}
注意:新项目此时只有这一个配置文件创建队列和交换机失败,一定要有一个监听消费者,使springboot能够连接到rabbitmq才能创建成功。
运行结果:
4 RabbitMQ死信队列效果演示
@RestController
public class DeadLetterController {
@Autowired
private RabbitTemplate rabbitTemplate;
/**
* 订单交换机
*/
@Value("${mayikt.order.exchange}")
private String orderExchange;
/**
* 订单路由key
*/
@Value("${mayikt.order.routingKey}")
private String orderRoutingKey;
@RequestMapping("/sendOrderMsg")
public String sendOrderMsg(){
rabbitTemplate.convertAndSend(orderExchange, orderRoutingKey, "mayikt", new MessagePostProcessor() {
@Override
public Message postProcessMessage(Message message) throws AmqpException {
message.getMessageProperties().setExpiration("10000");
return message;
}
});
return "success";
}
}
@Component
public class OrderDlxConsumer {
/**
* 监听死信队列
*
* @return
*/
@RabbitListener(queues = "mayikt_order_dlx_queue")
public void orderConsumer(String msg) {
System.out.println("死信队列获取消息:" + msg);
}
}
@Component
public class OrderConsumer {
/**
* 监听订单队列
*
* @return
*/
@RabbitListener(queues = "mayikt_order_queue")
public void orderConsumer(String msg) {
System.out.println("订单队列获取消息:" + msg);
}
}
运行结果:
死信队列不能够和正常队列存放在同一个服务器中,应该分开服务器存放。
5 RabbitMQ如何获取消费者结果
RabbitMQ异步如何获取消费结果?
多线程+主动查询。
消费的结果根据业务来定,假设业务场景是订单消费者消费成功将订单数据存入数据库,数据库能查到结果说明消费成功。
Maven依赖增加
<!--fastjson -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.49</version>
</dependency>
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
<version>1.1.1</version>
</dependency>
<!-- mysql 依赖 -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
application.yml增加
spring:
datasource:
url: jdbc:mysql://localhost:3306/test?useUnicode=true&characterEncoding=UTF-8&&useSSL=false
username: root
password: root
driver-class-name: com.mysql.jdbc.Driver
核心代码
@Data
public class OrderEntity {
private int id;
private String orderName;
private String orderId;
public OrderEntity(String orderName, String orderId) {
this.orderName = orderName;
this.orderId = orderId;
}
public OrderEntity() {
}
}
public interface OrderMapper {
@Insert("insert order_info values (null,#{orderName},#{orderId})")
int addOrder(OrderEntity orderEntity);
@Select("SELECT * from order_info where orderId=#{orderId} ")
OrderEntity getOrder(String orderId);
}
@RestController
public class DeadLetterController {
@Autowired
private RabbitTemplate rabbitTemplate;
/**
* 订单交换机
*/
@Value("${mayikt.order.exchange}")
private String orderExchange;
@Autowired
private OrderMapper orderMapper;
/**
* 订单路由key
*/
@Value("${mayikt.order.routingKey}")
private String orderRoutingKey;
@RequestMapping("/sendOrderMsg")
public String sendOrderMsg() {
// 1.生产订单id
String orderId = System.currentTimeMillis() + "";
String orderName = "蚂蚁课堂第六期报名";
OrderEntity orderEntity = new OrderEntity(orderName, orderId);
String msg = JSONObject.toJSONString(orderEntity);
sendMsg(msg);
return orderId;
// 后期客户端主动使用orderId调用服务器接口 查询该订单id是否在数据库中存在,存在说明消费成功,否则消费失败
}
// 如果用同步可以使用消息确认机制,必须投递成功再返回orderId,效率低
@Async
public void sendMsg(String msg) {
rabbitTemplate.convertAndSend(orderExchange, orderRoutingKey, msg,
new MessagePostProcessor() {
@Override
public Message postProcessMessage(Message message) throws AmqpException {
message.getMessageProperties().setExpiration("10000");
return message;
}
});
// 消息投递失败
}
/**
* 主动查询接口
* 先查询该订单的消息是否投递失败
* 再查询数据库
*/
@RequestMapping("/getOrder")
public Object getOrder(String orderId) {
OrderEntity orderEntity = orderMapper.getOrder(orderId);
if (orderEntity == null) {
return "消息正在异步的处理中";
}
return orderEntity;
}
}
@Component
public class OrderConsumer {
@Autowired
private OrderMapper orderMapper;
/**
* 监听订单队列
*
* @return
*/
@RabbitListener(queues = "mayikt_order_queue")
public void orderConsumer(String msg) {
System.out.println("订单队列获取消息:" + msg);
OrderEntity orderEntity = JSONObject.parseObject(msg, OrderEntity.class);
if(orderEntity == null){
return;
}
orderMapper.addOrder(orderEntity);
}
}
@SpringBootApplication
@EnableAsync
@MapperScan("com.mayikt.mapper")
public class SpringApp {
public static void main(String[] args) {
SpringApplication.run(SpringApp.class);
}
}
6 主动调用接口查询消费结果
运行结果: