RabbitMQ
1.1 引言
- 模块之间的耦合度多高,导致一个模块宕机后,全部功能都不能用了
- 同步通讯的成本问题。
1.2 RabbitMQ的介绍
市面上比较火爆的几款MQ:
ActiveMQ,RocketMQ,Kafka,RabbitMQ。
- 语言的支持:AcfiveMQ,RocketMQ只支持Java语言。Kafka可以支持多种语言,RabbitMQ支持多种语言
- 效率方面:ActiveMQ, RocketMQ,Kafka效率都是毫秒级别,RabbitMo是微秒级别的。
- 消息丢失,消息重复问题:RabbitMQ针对消息的持久化,和重复问题都有比较成熟的解决方案。
- 学习成本:RabbitMQ非常简单。
RabbitMo是由Rabbit公司去研发和维护的,最终是在Pivotal。
RabbitMO严格的遵循AMO协议,高级消息队列协议,帮助我们在进程之间传递异步消息。
二、安装RabbitMQ
version: "3.1"
services:
rabbitmq:
image: daocloud.io/library/rabbitmq:management
restart: always
container_name: rabbitmq
ports:
- 5672:5672
- 15672:15672
volumes:
- ./data:/var/lib/rabbitmq
启动容器后访问
http://192.168.56.100:15672
默认用户名密码都是 guest
三、RabbitMQ架构
3.1 官方的简单架构图
- Pubisher-生产者:发布消息到RabbitMQ中的Exchange
- Consumer-消费者:监听RabbitMQ中的Queue中的消息
- Exchange-交换机:和生产者建立连接并接受生产者的消息
- Queue-队列:Exchange会将消息分发到指定的Queue,Queue和消费者进行交互
- Routes-路由(交换机以什么样的策略将消息发布到Queue)
3.2 RabbitMQ完整架构图
3.3 查看图形化界面并创建一个Virtual Host
创建一个全新的用户和全新的Virtual Host,并且将test用户设置上可以操作/test的权限
四、RabbitMQ的使用
4.4 Java连接RabbitMQ
1.导入依赖
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.6.0</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
2.工具类
public static Connection getConnection() throws Exception {
//1.创建ConnectionFactory工厂
ConnectionFactory factory=new ConnectionFactory();
factory.setHost("192.168.56.100");
factory.setPort(5672);
factory.setUsername("test");
factory.setPassword("test");
factory.setVirtualHost("/test");
//2.创建Connection
Connection conn=null;
try {
conn=factory.newConnection();
}catch (Exception e){
e.printStackTrace();
}
//3.返回
return conn;
}
4.4.1 Hello-World
一个生产者,一个默认的交换机,一个队列,一个消费者
4.4.2 Work
一个生产者,一个默认的交换机,一个队列,两个个消费者
4.4.3 Publish/Subscribe
一个生产者,一个默认的交换机,凉个队列,两个个消费者
4.4.4 Routing
一个生产者,一个交换机,两个队列,两个消费者
4.4.5 Topic
一个生产者,一个交换机,两个队列,两个消费者
1、生产者创建Topic的exchange并且绑定到队列中,这次绑定可以通过*和#关键字,对指定RoutingKey内容,
编写时注意格式xox.xxx.xxx去编写,’>一个xx,而#->代表多个xxx.xxx,在发送消息时,指定具体的
RoutingKey到底是什么。
五、RabbitMQ整合SpringBoot
5.1 SpringBoot整合RabbitMQ
1.导入pom.xml
<!--RabbitMQ-->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
2.编写配置文件
spring:
rabbitmq:
host: 192.168.56.100
port: 5672
username: test
password: test
virtual-host: /test
3.编写配置类,声明exchange和queue,并且绑定到一起
@Configuration
public class RabbitMQConfig {
//1.创建exchange -topic
@Bean
public TopicExchange getTopicExchange(){
return new TopicExchange("boot-topic-exchange",true,false);
}
//2.创建queue
@Bean
public Queue getQueue(){
return new Queue("boot-queue",true,false,false,null);
}
//3.绑定在一起
@Bean
public Binding getBinding(TopicExchange topicExchange,Queue queue){
return BindingBuilder.bind(queue).to(topicExchange).with("*.red.*");
}
}
4.发布消息
@Autowired
private RabbitTemplate rabbitTemplate;
@Test
void contextLoads() {
rabbitTemplate.convertAndSend("boot-topic-exchange","slow.red.dog","红色大狼狗");
}
5.创建消费者监听消息
@Component
public class Consumer {
@RabbitListener(queues = "boot-queue")
public void getMassage(Object message){
System.out.println("接收到的消息:"+message);
}
}
5.2 手动ACK
1.添加配置文件
spring:
rabbitmq:
host: 192.168.56.100
port: 5672
username: test
password: test
virtual-host: /test
listener:
simple:
acknowledge-mode: manual #手动
2.在消费消息的位置,修改方法,再手动ack
@Component
public class Consumer {
@RabbitListener(queues = "boot-queue")
public void getMassage(String msg, Channel channel, Message message) throws IOException {
System.out.println("接收到的消息:"+message);
//手动ACK
channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);
}
}
六、RabbitMQ的其他操作
6.1 消息的可靠性
RabbitMQ的事务:事务可以保证消息100%传递,可以通过事务的回滚去记录日志,后面定时再次发送当前消
息。事务的操作,效率太低,加了事务操作后,比平时的操作效率的至少要慢100倍。
confrim机制
RabbitMQ除了事务,还提供了Confirm的确认机制,这个效率比事务高很多。
- 普通Confirm方式
- 异步Confirm方式
- 批量Confirm方式
普通Confirm方式
@Test
public void publish() throws Exception {
//1.获取Connection
Connection connection= RabbitMQClient.getConnection();
//2.创建Channel
Channel channel = connection.createChannel();
//3.创建exchange
//3.1开启confirm
channel.confirmSelect();
//3.2发送消息
String msg="Hello-World";
channel.basicPublish("","HelloWorld",null,msg.getBytes());
//3.3判断消息是否发送成功
if(channel.waitForConfirms()){
System.out.println("消息发送成功");
}else {
System.out.println("发送消息失败");
}
System.out.println("生产者发布消息成功");
//4.释放
channel.close();
connection.close();
}
批量Confirm方式
//开启confrim
channel.confirmSelect();
//发送消息
for (int i=0;i<1000;i++){
String msg="Hello-World";
channel.basicPublish("","HelloWorld",null,msg.getBytes());
}
//当你发送的全部消息,有一个失败的时候,就直接全部失败,抛出IOException
channel.waitForConfirms();
异步Confirm方式
public class Publisher3 {
@Test
public void publish() throws Exception {
//1.获取Connection
Connection connection= RabbitMQClient.getConnection();
//2.创建Channel
Channel channel = connection.createChannel();
//3.创建exchange
//3.1开启confrim
channel.confirmSelect();
//3.2发送消息
for (int i=0;i<1000;i++){
String msg="Hello-World";
channel.basicPublish("","HelloWorld",null,msg.getBytes());
}
channel.addConfirmListener(new ConfirmListener() {
@Override
public void handleAck(long l, boolean b) throws IOException {
System.out.println("消息发送成功,标识:"+l+"是否批量:"+b);
}
@Override
public void handleNack(long l, boolean b) throws IOException {
System.out.println("消息发送失败,标识:"+l+"是否批量:"+b);
}
});
System.in.read();
System.out.println("生产者发布Publisher2消息成功");
//4.释放
channel.close();
connection.close();
}
}
6.1.2 Return机制
Confirm只能保证消息到达exchange,无法保证消息可以被exchange分发到指定queue。
而且exchange是不能持久化消息的,queue是可以持久化消息采用Return机制来监听消息是否从exchange送到了指定的queue中
//开启Return机制
channel.addReturnListener(new ReturnListener() {
@Override
public void handleReturn(int i, String s, String s1, String s2, AMQP.BasicProperties basicProperties, byte[] body) throws IOException {
//当消息没有送达到队列中时才会执行
System.out.println(new String(body,"UTF-8")+"没有送达到Queue");
}
});
//发送消息时候,需要指定mandatory为true,发送消息调用以下构造方法
//void basicPublish(String exchange, String routingKey, boolean mandatory, BasicProperties props, byte[] body)
6.1.3 Springboot实现Confirm以及Return
1.添加配置文件,开启confrim以及return
spring:
rabbitmq:
publisher-confirm-type: simple
publisher-returns: true
2.指定RabbitTemplate对象,开启Confrim和Return
@Component
public class PublisherConfrimAndReturnConfig implements RabbitTemplate.ConfirmCallback,RabbitTemplate.ReturnCallback {
@Autowired
private RabbitTemplate rabbitTemplate;
@PostConstruct
public void initMethod(){
rabbitTemplate.setConfirmCallback(this);
}
@Override
public void confirm(CorrelationData correlationData, boolean ack, String cause) {
if(ack){
System.out.println("消息以及送到到ack");
}else{
System.out.println("消息没有送达exchange");
}
}
@Override
public void returnedMessage(Message message, int replyCode, String replyText, String exchange, String routingKey) {
System.out.println("消息没有送达queue");
}
}
6.2 消息重复消费
重复消费消息,会对非幂等行操作造成问题
重复消费消息的原因是,消费者没有给RabbitMQ一个ack
为了解决消息重复消费的问题,可以采用Redis,在消费者消费消息之前,现将消息的id放到Redis中
id-0 (正在执行业务)
id-1 (执行业务成功)
如果ack失败,在RabbitMQ将消息交给其他的消费者时,先执行setnx,如果key已经存在,获取他的值,如果是0,当前消费者就什么都不做,如果是1,直接ack。
极端情况:第一个消费者在执行业务时,出现了死锁,在setnx的基础上,再给key设置一个生存时
1.生产者,发送消息时,指定messageld
AMQP.BasicProperties properties=new AMQP.BasicProperties().builder()
.deliveryMode(1) //1:需要持久化 2:不需要
.messageId(UUID.randomUUID().toString())
.build();
String msg="Hello-World";
channel.basicPublish("","HelloWorld",true,properties,msg.getBytes());
2.消费者,在消费消息时,根据具体业务逻辑去操作redis
Jedis jedis=new Jedis("192.168.56.100",6379);
jedis.auth("xxxx");
String messageId = properties.getMessageId();
//1.setnx到Redis中,默认指定value=0
String result = jedis.set(messageId, "0", "NX", "EX", 10);
System.out.println("result"+result);
if(result!=null&&result.equalsIgnoreCase("OK")){
System.out.println("接收到消息:"+new String(body,"UTF-8"));
//2.消费成功,set messageId 1
jedis.set(messageId,"1");
channel.basicAck(envelope.getDeliveryTag(),false);
}else {
//3.如果1中的
String s = jedis.get(messageId);
if("1".equalsIgnoreCase(s)){
channel.basicAck(envelope.getDeliveryTag(),false);
}
}
6.2.2 SpringBoot如何实现
导入依赖
<!--redis-->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
编写配置文件
spring:
redis:
host: 192.168.56.100
port: 6379
修改生产者
@Test
void contextLoads() throws IOException {
CorrelationData messageId=new CorrelationData(UUID.randomUUID().toString());
rabbitTemplate.convertAndSend("boot-topic-exchange","slow.red.dog","红色大狼狗",messageId);
System.in.read();
}
修改消费者
@RabbitListener(queues = "boot-queue")
public void getMassage(String msg, Channel channel, Message message) throws IOException {
//0.获取messageId
String messageId = message.getMessageProperties().getHeader("spring_returned_message_correlation");
//1.设置key到Redis
if (redisTemplate.opsForValue().setIfAbsent(messageId,"0",10, TimeUnit.SECONDS)) {
//2.消费消息
System.out.println("接收到的消息"+msg);
//3.设置key的value为1
redisTemplate.opsForValue().set(messageId,"1",10,TimeUnit.SECONDS);
//4.手动ACK
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
}else {
//5.获取Redis中的value 即可,如果是1,手动ack
if("1".equalsIgnoreCase(redisTemplate.opsForValue().get(messageId))){
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
}
}
}
七、RabbitMQ应用
客户模块
1.导入依赖
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
2.编写配置文件
spring:
rabbitmq:
host: 192.168.56.100
port: 5672
username: test
password: test
virtual-host: /test
3.编写配置类
package com.it.openapi.customer.config;
import org.springframework.amqp.core.*;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class RabbitMQConfig {
@Bean
public TopicExchange topicExchange(){
return new TopicExchange("openapi-customer-exchange",true,false);
}
@Bean
public Queue queue(){
return new Queue("openapi-customer-queue");
}
@Bean
public Binding binding(Queue queue, TopicExchange topicExchange){
return BindingBuilder.bind(queue).to(topicExchange).with("openapi.customer.*");
}
}
4.修改Service (发送消息的方式(第三步))
@Transactional
public void saveCustomer(Customer customer) {
//1.调用mapper添加数据到mysql
Integer count = customerMapper.saveCustomer(customer);
//2.判断是否成功
if(count!=1){
log.error("【添加客户信息失败】customer={}",customer);
throw new RuntimeException("【添加客户信息失败】");
}
//3.发送消息
rabbitTemplate.convertAndSend("openapi-customer-exchange","openapi.customer.add",JSON.toJSON(customer));
}
搜索模块
org.springframework.boot spring-boot-starter-amqp1.导入依赖
2.编写配置文件
spring:
rabbitmq:
host: 192.168.56.100
port: 5672
username: test
password: test
virtual-host: /test
listener:
simple:
acknowledge-mode: manual
3.编写配置类
package com.it.openapi.search.config;
import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.Queue;
import org.springframework.amqp.core.TopicExchange;
import org.springframework.context.annotation.Bean;
@Configuration
public class RabbitMQConfig {
@Bean
public TopicExchange topicExchange(){
return new TopicExchange("openapi-customer-exchange",true,false);
}
@Bean
public Queue queue(){
return new Queue("openapi-customer-queue");
}
@Bean
public Binding binding(Queue queue,TopicExchange topicExchange){
return BindingBuilder.bind(queue).to(topicExchange).with("openapi.customer.*");
}
}
4.编写消费者
package com.it.openapi.search.listener;
import com.it.openapi.search.entity.Customer;
import com.it.openapi.search.service.CustomerService;
import com.it.openapi.search.utils.JSON;
import com.rabbitmq.client.Channel;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import java.io.IOException;
@Component
public class CustomerListener {
@Autowired
private CustomerService customerService;
@RabbitListener(queues = "openapi-customer-queue")
public void consume(String json, Channel channel, Message message) throws IOException {
//1.获取RoutingKey
String routingKey = message.getMessageProperties().getReceivedRoutingKey();
//2.使用switch
switch(routingKey){
case "openapi.customer.add":
//3.add操作调用Service操作
customerService.saveCustomer(JSON.parseJSON(json,Customer.class));
//4.手动ack
channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);
}
}
}
ponent
public class CustomerListener {
@Autowired
private CustomerService customerService;
@RabbitListener(queues = "openapi-customer-queue")
public void consume(String json, Channel channel, Message message) throws IOException {
//1.获取RoutingKey
String routingKey = message.getMessageProperties().getReceivedRoutingKey();
//2.使用switch
switch(routingKey){
case "openapi.customer.add":
//3.add操作调用Service操作
customerService.saveCustomer(JSON.parseJSON(json,Customer.class));
//4.手动ack
channel.basicAck(message.getMessageProperties().getDeliveryTag(),false);
}
}
}