MQ
- 消息可靠性
- 消息幂等性
- MQ的高可用
基本概念
MQ,Message Queue消息队列,是消息传输过程中保存消息的容器,多用于分布式系统之间进行通信。
- MQ,消息队列,存储消息的中间件
- 分布式系统通信方式:直接远程调用 、借助第三方完成间接通信
- 发送方为生产者、接收方为消费者
优劣
优势
- 应用解耦:提高系统容错性与可维护性
- 异步提速:提高用户体验与系统吞吐量
- 消费填谷:提高系统稳定性
劣势
- 系统复杂度提高
- 引入rabbitmq产生的新问题,如:网络通信、数据一致性、幂等性等
使用条件
- 生产者无需从消费者处得到反馈
- 允许数据短暂的不一致性
- 引入MQ效益明显高于不引入时的效益
常见产品
- RabbitMQ:延迟最低
- AcitveMQ:老牌MQ,性能最低
- RocketMQ:高吞吐量,高并发、分布式MQ、金融
- Kafka:高吞吐量,大数据方面
AMQP协议
RabbitMQ
-
Broker:接收和分发消息的应用,RabbitMQ Server就是Message Broker
-
Virtual host:处于多租户和安全因素设计,把AMQP的基本组件分到一个虚拟的分组中,类似与网络中的namespace概念(VCP,专有网络),当多个不用用户使用同一个RabbitMQ Server提供的服务时,可以划分出多个vhost,每个用户在自己的vhost创建exchange、queue等
-
Connection:publisher/consumer和broker之间的TCP连接,每个Connection中有多个channel(类似连接池,避免重复创建channel损耗性能)
-
Channel:Connection内部建立的逻辑连接,如果应用程序支持多线程,通常每个thread创建单独的channel进行通讯,QMQP method包含了channel id帮助客户端和message broker识别channel,所有channel之间是完全隔离的。Channel作为轻量级Connection极大减少了操作系统建立TCP connection的开销
-
Exchange:交换机,message到达broker的第一站,根据分发规则,匹配表中的routing key,分发消息到queue中去。常用的类型有:direct(point-to-point),topic(publish-subscribe),fanout(multicast)
-
Queue:消息最终被送到这里等待consumer取走
-
Binding:exchange和queue之间的虚拟连接,binding中可以用包含routing key。Binding信息被保存到exchange的查询表中,用于message的分发依据
JMS
- JMS,Java消息服务(JavaMessage Service)应用程序接口,是一个Java平台中关于面向消息中间件的API.
- JMS是JavaEE规范中的一种,类比JDBC
- 很多消息中间件都实现了JMS规范,例如:ActiveMQ. RabbitMQ官方没有提供JMS的实现包,但是开源社区有提供.
工作模式(Java客户端版)
简单模式
生产者
<dependencies>
<!--rabbitmq java客户端-->
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.9.0</version>
</dependency>
</dependencies>
/**
* 发送消息
*/
public class MessageProducer {
public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
// 1.创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
// 2.设置参数
factory.setHost("43.139.51.247"); // ip 默认值:localhost
factory.setPort(5672); // 端口 默认值:5672
factory.setVirtualHost("/zhd"); // 虚拟机 默认值:/
factory.setUsername("czk");
factory.setPassword("czk");
// 3.创建连接Connection
Connection connection = factory.newConnection();
// 4.创建Channel
Channel channel = connection.createChannel();
// 5.创建队列Queue
/**
* public DeclareOk queueDeclare(String queue, boolean durable, boolean exclusive, boolean autoDelete, Map<String, Object> arguments)
* queue:队列名称,队列名称存在则使用,不存在则创建一个该名称队列
* durable:是否持久化
* exclusive:是否独占,只能有一个消费者监听队列,当Connection关闭时是否删除队列
* autoDelete:是否自动删除,没有Consumer时自动删除
* arguments:参数
*/
channel.queueDeclare("hello_world",true,false,false,null);
// 6.发送消息
/**
* public void basicPublish(String exchange, String routingKey, BasicProperties props, byte[] body)
* exchange:交换机名称。简单模式下交换机默认为 ""
* routingKey:路由名称。注意:使用默认交换机,则routingKey要与队列名一致才能正常路由
* props:配置信息
* body:消息数据
*/
String body = "hello rabbitmq~~~";
channel.basicPublish("","hello_world",null,body.getBytes());
// 7.释放资源
Thread.sleep(30000);
System.out.println("end...");
channel.close();
connection.close();
}
}
消费者
<dependencies>
<!--rabbitmq java客户端-->
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.9.0</version>
</dependency>
</dependencies>
/**
* 接收/消费消息
*/
public class MessageConsumer {
public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
// 1.创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
// 2.设置参数
factory.setHost("43.139.51.247"); // ip 默认值:localhost
factory.setPort(5672); // 端口 默认值:5672
factory.setVirtualHost("/zhd"); // 虚拟机 默认值:/
factory.setUsername("czk");
factory.setPassword("czk");
// 3.创建连接Connection
Connection connection = factory.newConnection();
// 4.创建Channel
Channel channel = connection.createChannel();
// 5.创建队列Queue
/**
* public DeclareOk queueDeclare(String queue, boolean durable, boolean exclusive, boolean autoDelete, Map<String, Object> arguments)
* queue:队列名称,队列名称存在则使用,不存在则创建一个该名称队列
* durable:是否持久化
* exclusive:是否独占,只能有一个消费者监听队列,当Connection关闭时是否删除队列
* autoDelete:是否自动删除,没有Consumer时自动删除
* arguments:参数
*/
channel.queueDeclare("hello_world",true,false,false,null);
// 6.接收消息
/**
* public String basicConsume(String queue, boolean autoAck, Consumer callback)
* queue:队列名称
* autoAck:收到消息是否自动确认,消息丢失相关
* callback:回调对象
*/
Consumer consumer = new DefaultConsumer(channel){
/**
* 回调方法
* consumerTag:标识
* envelope:获取一些消息,交换机、路由key
* properties:配置消息
* body:数据
*/
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("consumerTag:"+consumerTag);
System.out.println("envelope:"+envelope);
System.out.println("Exchange:"+envelope.getExchange());
System.out.println("RoutingKey:"+envelope.getRoutingKey());
System.out.println("properties:"+properties);
System.out.println("body:"+new String(body));
System.out.println("------------------------------------");
}
};
channel.basicConsume("hello_world",true,consumer);
// 监听程序,无需关闭资源
}
}
工作队列模式
生产者
<dependencies>
<!--rabbitmq java客户端-->
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.9.0</version>
<exclusions>
<exclusion>
<artifactId>slf4j-scala-api</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
/**
* 发送消息
*/
public class MessageProducer_WorkQueue {
public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
// 1.创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
// 2.设置参数
factory.setHost("43.139.51.247"); // ip 默认值:localhost
factory.setPort(5672); // 端口 默认值:5672
factory.setVirtualHost("/zhd"); // 虚拟机 默认值:/
factory.setUsername("czk");
factory.setPassword("czk");
// 3.创建连接Connection
Connection connection = factory.newConnection();
// 4.创建Channel
Channel channel = connection.createChannel();
// 5.创建队列Queue
/**
* public DeclareOk queueDeclare(String queue, boolean durable, boolean exclusive, boolean autoDelete, Map<String, Object> arguments)
* queue:队列名称,队列名称存在则使用,不存在则创建一个该名称队列
* durable:是否持久化
* exclusive:是否独占,只能有一个消费者监听队列,当Connection关闭时是否删除队列
* autoDelete:是否自动删除,没有Consumer时自动删除
* arguments:参数
*/
channel.queueDeclare("work_queue",true,false,false,null);
// 6.发送消息
/**
* public void basicPublish(String exchange, String routingKey, BasicProperties props, byte[] body)
* exchange:交换机名称。简单模式下交换机默认为 ""
* routingKey:路由名称。注意:使用默认交换机,则routingKey要与队列名一致才能正常路由
* props:配置信息
* body:消息数据
*/
for (int i = 0;i<10;i++){
String body = "hello rabbitmq : "+i;
channel.basicPublish("","work_queue",null,body.getBytes());
}
// 7.释放资源
Thread.sleep(30000);
System.out.println("end...");
channel.close();
connection.close();
}
}
消费者
<dependencies>
<!--rabbitmq java客户端-->
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.9.0</version>
<exclusions>
<exclusion>
<artifactId>slf4j-scala-api</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
/**
* 接收/消费消息
*/
public class MessageConsumer_WorkQueue1 {
public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
// 1.创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
// 2.设置参数
factory.setHost("43.139.51.247"); // ip 默认值:localhost
factory.setPort(5672); // 端口 默认值:5672
factory.setVirtualHost("/zhd"); // 虚拟机 默认值:/
factory.setUsername("czk");
factory.setPassword("czk");
// 3.创建连接Connection
Connection connection = factory.newConnection();
// 4.创建Channel
Channel channel = connection.createChannel();
// 5.创建队列Queue
/**
* public DeclareOk queueDeclare(String queue, boolean durable, boolean exclusive, boolean autoDelete, Map<String, Object> arguments)
* queue:队列名称,队列名称存在则使用,不存在则创建一个该名称队列
* durable:是否持久化
* exclusive:是否独占,只能有一个消费者监听队列,当Connection关闭时是否删除队列
* autoDelete:是否自动删除,没有Consumer时自动删除
* arguments:参数
*/
channel.queueDeclare("work_queue",true,false,false,null);
// 6.接收消息
/**
* public String basicConsume(String queue, boolean autoAck, Consumer callback)
* queue:队列名称
* autoAck:收到消息是否自动确认,消息丢失相关
* callback:回调对象
*/
Consumer consumer = new DefaultConsumer(channel){
/**
* 回调方法
* consumerTag:标识
* envelope:获取一些消息,交换机、路由key
* properties:配置消息
* body:数据
*/
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("consumerTag:"+consumerTag);
System.out.println("envelope:"+envelope);
System.out.println("Exchange:"+envelope.getExchange());
System.out.println("RoutingKey:"+envelope.getRoutingKey());
System.out.println("properties:"+properties);
System.out.println("body:"+new String(body));
System.out.println("------------------------------------");
}
};
channel.basicConsume("work_queue",true,consumer);
// 监听程序,无需关闭资源
}
}
/**
* 接收/消费消息
*/
public class MessageConsumer_WorkQueue2 {
public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
// 1.创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
// 2.设置参数
factory.setHost("43.139.51.247"); // ip 默认值:localhost
factory.setPort(5672); // 端口 默认值:5672
factory.setVirtualHost("/zhd"); // 虚拟机 默认值:/
factory.setUsername("czk");
factory.setPassword("czk");
// 3.创建连接Connection
Connection connection = factory.newConnection();
// 4.创建Channel
Channel channel = connection.createChannel();
// 5.创建队列Queue
/**
* public DeclareOk queueDeclare(String queue, boolean durable, boolean exclusive, boolean autoDelete, Map<String, Object> arguments)
* queue:队列名称,队列名称存在则使用,不存在则创建一个该名称队列
* durable:是否持久化
* exclusive:是否独占,只能有一个消费者监听队列,当Connection关闭时是否删除队列
* autoDelete:是否自动删除,没有Consumer时自动删除
* arguments:参数
*/
channel.queueDeclare("work_queue",true,false,false,null);
// 6.接收消息
/**
* public String basicConsume(String queue, boolean autoAck, Consumer callback)
* queue:队列名称
* autoAck:收到消息是否自动确认,消息丢失相关
* callback:回调对象
*/
Consumer consumer = new DefaultConsumer(channel){
/**
* 回调方法
* consumerTag:标识
* envelope:获取一些消息,交换机、路由key
* properties:配置消息
* body:数据
*/
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("consumerTag:"+consumerTag);
System.out.println("envelope:"+envelope);
System.out.println("Exchange:"+envelope.getExchange());
System.out.println("RoutingKey:"+envelope.getRoutingKey());
System.out.println("properties:"+properties);
System.out.println("body:"+new String(body));
System.out.println("------------------------------------");
}
};
channel.basicConsume("work_queue",true,consumer);
// 监听程序,无需关闭资源
}
}
Pub/Sub订阅模式
生产者
<dependencies>
<!--rabbitmq java客户端-->
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.9.0</version>
</dependency>
</dependencies>
/**
* 发送消息
*/
public class MessageProducer_PubSub {
public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
// 1.创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
// 2.设置参数
factory.setHost("43.139.51.247"); // ip 默认值:localhost
factory.setPort(5672); // 端口 默认值:5672
factory.setVirtualHost("/zhd"); // 虚拟机 默认值:/
factory.setUsername("czk");
factory.setPassword("czk");
// 3.创建连接Connection
Connection connection = factory.newConnection();
// 4.创建Channel
Channel channel = connection.createChannel();
// 5.创建交换机
/**
* public DeclareOk exchangeDeclare(String exchange, BuiltinExchangeType type, boolean durable, boolean autoDelete, boolean internal, Map<String, Object> arguments)
* exchange:交换机名称
* type:交换机类型(4种) : direct(定向)、fanout(扇形、广播,消息发送给每个与之绑定队列)、topic(通配符方式)、header(参数匹配、少用)
* durable:是否持久化
* autoDelete:自动删除
* internal:内部使用,一般为false
* arguments:参数
*/
String exchangeName = "pub_sub_fanout_exchange";
channel.exchangeDeclare(exchangeName, BuiltinExchangeType.FANOUT,true,false,false,null);
// 6.创建队列Queue
/**
* public DeclareOk queueDeclare(String queue, boolean durable, boolean exclusive, boolean autoDelete, Map<String, Object> arguments)
* queue:队列名称,队列名称存在则使用,不存在则创建一个该名称队列
* durable:是否持久化
* exclusive:是否独占,只能有一个消费者监听队列,当Connection关闭时是否删除队列
* autoDelete:是否自动删除,没有Consumer时自动删除
* arguments:参数
*/
String queue1Name = "fanout_queue1";
String queue2Name = "fanout_queue2";
channel.queueDeclare(queue1Name,true,false,false,null);
channel.queueDeclare(queue2Name,true,false,false,null);
// 7.绑定队列与交换机
/**
* public com.rabbitmq.client.AMQP.Queue.BindOk queueBind(String queue, String exchange, String routingKey)
* queue:绑定的队列名
* exchange:交换机名称
*routingKey:路由键,绑定规则,如果使用的是fanout(广播),则routingKey为空
*/
channel.queueBind(queue1Name,exchangeName,"");
channel.queueBind(queue2Name,exchangeName,"");
// 8.发送消息
/**
* public void basicPublish(String exchange, String routingKey, BasicProperties props, byte[] body)
* exchange:交换机名称。简单模式下交换机默认为 ""
* routingKey:路由名称。注意:使用默认交换机,则routingKey要与队列名一致才能正常路由
* props:配置信息
*/
String body = "fanout~~~";
channel.basicPublish(exchangeName,"",null,body.getBytes()); // 广播模式,第二个参数routingKey为空
// 9.释放资源
System.out.println("end...");
channel.close();
connection.close();
}
}
消费者
<dependencies>
<!--rabbitmq java客户端-->
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.9.0</version>
</dependency>
</dependencies>
/**
* 接收/消费消息
*/
public class MessageConsumer_PubSub1 {
public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
// 1.创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
// 2.设置参数
factory.setHost("43.139.51.247"); // ip 默认值:localhost
factory.setPort(5672); // 端口 默认值:5672
factory.setVirtualHost("/zhd"); // 虚拟机 默认值:/
factory.setUsername("czk");
factory.setPassword("czk");
// 3.创建连接Connection
Connection connection = factory.newConnection();
// 4.创建Channel
Channel channel = connection.createChannel();
// 5.接收消息
/**
* public String basicConsume(String queue, boolean autoAck, Consumer callback)
* queue:队列名称
* autoAck:收到消息是否自动确认,消息丢失相关
* callback:回调对象
*/
Consumer consumer = new DefaultConsumer(channel){
/**
* 回调方法
* consumerTag:标识
* envelope:获取一些消息,交换机、路由key
* properties:配置消息
* body:数据
*/
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("body:"+new String(body));
System.out.println("PubSub1:日志打印");
System.out.println("------------------------------------");
}
};
channel.basicConsume("fanout_queue1",true,consumer);
// 监听程序,无需关闭资源
}
}
/**
* 接收/消费消息
*/
public class MessageConsumer_PubSub2 {
public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
// 1.创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
// 2.设置参数
factory.setHost("43.139.51.247"); // ip 默认值:localhost
factory.setPort(5672); // 端口 默认值:5672
factory.setVirtualHost("/zhd"); // 虚拟机 默认值:/
factory.setUsername("czk");
factory.setPassword("czk");
// 3.创建连接Connection
Connection connection = factory.newConnection();
// 4.创建Channel
Channel channel = connection.createChannel();
// 5.接收消息
/**
* public String basicConsume(String queue, boolean autoAck, Consumer callback)
* queue:队列名称
* autoAck:收到消息是否自动确认,消息丢失相关
* callback:回调对象
*/
Consumer consumer = new DefaultConsumer(channel){
/**
* 回调方法
* consumerTag:标识
* envelope:获取一些消息,交换机、路由key
* properties:配置消息
* body:数据
*/
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("body:"+new String(body));
System.out.println("PubSub2:数据保存");
System.out.println("------------------------------------");
}
};
channel.basicConsume("fanout_queue2",true,consumer);
// 监听程序,无需关闭资源
}
}
Routing路由模式
生产者
<dependencies>
<!--rabbitmq java客户端-->
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.9.0</version>
</dependency>
</dependencies>
/**
* 发送消息
*/
public class MessageProducer_Routing {
public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
// 1.创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
// 2.设置参数
factory.setHost("43.139.51.247"); // ip 默认值:localhost
factory.setPort(5672); // 端口 默认值:5672
factory.setVirtualHost("/zhd"); // 虚拟机 默认值:/
factory.setUsername("czk");
factory.setPassword("czk");
// 3.创建连接Connection
Connection connection = factory.newConnection();
// 4.创建Channel
Channel channel = connection.createChannel();
// 5.创建交换机
/**
* public DeclareOk exchangeDeclare(String exchange, BuiltinExchangeType type, boolean durable, boolean autoDelete, boolean internal, Map<String, Object> arguments)
* exchange:交换机名称
* type:交换机类型(4种) : direct(定向)、fanout(扇形、广播,消息发送给每个与之绑定队列)、topic(通配符方式)、header(参数匹配、少用)
* durable:是否持久化
* autoDelete:自动删除
* internal:内部使用,一般为false
* arguments:参数
*/
String exchangeName = "direct_exchange";
channel.exchangeDeclare(exchangeName, BuiltinExchangeType.DIRECT,true,false,false,null);
// 6.创建队列Queue
/**
* public DeclareOk queueDeclare(String queue, boolean durable, boolean exclusive, boolean autoDelete, Map<String, Object> arguments)
* queue:队列名称,队列名称存在则使用,不存在则创建一个该名称队列
* durable:是否持久化
* exclusive:是否独占,只能有一个消费者监听队列,当Connection关闭时是否删除队列
* autoDelete:是否自动删除,没有Consumer时自动删除
* arguments:参数
*/
String queue1Name = "routing_queue1";
String queue2Name = "routing_queue2";
channel.queueDeclare(queue1Name,true,false,false,null);
channel.queueDeclare(queue2Name,true,false,false,null);
// 7.绑定队列与交换机
/**
* public com.rabbitmq.client.AMQP.Queue.BindOk queueBind(String queue, String exchange, String routingKey)
* queue:绑定的队列名
* exchange:交换机名称
*routingKey:路由键,绑定规则,如果使用的是fanout(广播),则routingKey为空
*/
//队列1与error绑定
channel.queueBind(queue1Name,exchangeName,"info");
//队列2与info、warning、error绑定
channel.queueBind(queue1Name,exchangeName,"warning");
channel.queueBind(queue1Name,exchangeName,"error");
channel.queueBind(queue2Name,exchangeName,"error");
// 8.发送消息
/**
* public void basicPublish(String exchange, String routingKey, BasicProperties props, byte[] body)
* exchange:交换机名称。简单模式下交换机默认为 ""
* routingKey:路由名称。注意:使用默认交换机,则routingKey要与队列名一致才能正常路由
* props:配置信息
*/
String info = "【INFO】:fanout~~~";
channel.basicPublish(exchangeName,"info",null,info.getBytes());
String warning = "【WARNING】:fanout~~~";
channel.basicPublish(exchangeName,"warning",null,warning.getBytes());
String error = "【ERROR】:fanout~~~";
channel.basicPublish(exchangeName,"error",null,error.getBytes());
// 9.释放资源
System.out.println("end...");
channel.close();
connection.close();
}
}
消费者
<dependencies>
<!--rabbitmq java客户端-->
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.9.0</version>
</dependency>
</dependencies>
/**
* 接收/消费消息
*/
public class MessageConsumer_Routing1 {
public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
// 1.创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
// 2.设置参数
factory.setHost("43.139.51.247"); // ip 默认值:localhost
factory.setPort(5672); // 端口 默认值:5672
factory.setVirtualHost("/zhd"); // 虚拟机 默认值:/
factory.setUsername("czk");
factory.setPassword("czk");
// 3.创建连接Connection
Connection connection = factory.newConnection();
// 4.创建Channel
Channel channel = connection.createChannel();
// 5.接收消息
/**
* public String basicConsume(String queue, boolean autoAck, Consumer callback)
* queue:队列名称
* autoAck:收到消息是否自动确认,消息丢失相关
* callback:回调对象
*/
Consumer consumer = new DefaultConsumer(channel){
/**
* 回调方法
* consumerTag:标识
* envelope:获取一些消息,交换机、路由key
* properties:配置消息
* body:数据
*/
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("body:"+new String(body));
System.out.println("PubSub1:日志打印");
System.out.println("------------------------------------");
}
};
channel.basicConsume("routing_queue1",true,consumer);
// 监听程序,无需关闭资源
}
}
/**
* 接收/消费消息
*/
public class MessageConsumer_Routing2 {
public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
// 1.创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
// 2.设置参数
factory.setHost("43.139.51.247"); // ip 默认值:localhost
factory.setPort(5672); // 端口 默认值:5672
factory.setVirtualHost("/zhd"); // 虚拟机 默认值:/
factory.setUsername("czk");
factory.setPassword("czk");
// 3.创建连接Connection
Connection connection = factory.newConnection();
// 4.创建Channel
Channel channel = connection.createChannel();
// 5.接收消息
/**
* public String basicConsume(String queue, boolean autoAck, Consumer callback)
* queue:队列名称
* autoAck:收到消息是否自动确认,消息丢失相关
* callback:回调对象
*/
Consumer consumer = new DefaultConsumer(channel){
/**
* 回调方法
* consumerTag:标识
* envelope:获取一些消息,交换机、路由key
* properties:配置消息
* body:数据
*/
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("body:"+new String(body));
System.out.println("PubSub2:日志数据保存");
System.out.println("------------------------------------");
}
};
channel.basicConsume("routing_queue2",true,consumer);
// 监听程序,无需关闭资源
}
}
Topic通配符模式
- *:表示任意一个单词
- #:表示一个/多个单词
生产者
<dependencies>
<!--rabbitmq java客户端-->
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.9.0</version>
</dependency>
</dependencies>
/**
* 发送消息
*/
public class MessageProducer_Topics {
public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
// 1.创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
// 2.设置参数
factory.setHost("43.139.51.247"); // ip 默认值:localhost
factory.setPort(5672); // 端口 默认值:5672
factory.setVirtualHost("/zhd"); // 虚拟机 默认值:/
factory.setUsername("czk");
factory.setPassword("czk");
// 3.创建连接Connection
Connection connection = factory.newConnection();
// 4.创建Channel
Channel channel = connection.createChannel();
// 5.创建交换机
/**
* public DeclareOk exchangeDeclare(String exchange, BuiltinExchangeType type, boolean durable, boolean autoDelete, boolean internal, Map<String, Object> arguments)
* exchange:交换机名称
* type:交换机类型(4种) : direct(定向)、fanout(扇形、广播,消息发送给每个与之绑定队列)、topic(通配符方式)、header(参数匹配、少用)
* durable:是否持久化
* autoDelete:自动删除
* internal:内部使用,一般为false
* arguments:参数
*/
String exchangeName = "topic_exchange";
channel.exchangeDeclare(exchangeName, BuiltinExchangeType.TOPIC,true,false,false,null);
// 6.创建队列Queue
/**
* public DeclareOk queueDeclare(String queue, boolean durable, boolean exclusive, boolean autoDelete, Map<String, Object> arguments)
* queue:队列名称,队列名称存在则使用,不存在则创建一个该名称队列
* durable:是否持久化
* exclusive:是否独占,只能有一个消费者监听队列,当Connection关闭时是否删除队列
* autoDelete:是否自动删除,没有Consumer时自动删除
* arguments:参数
*/
String queue1Name = "topic_queue1";
String queue2Name = "topic_queue2";
channel.queueDeclare(queue1Name,true,false,false,null);
channel.queueDeclare(queue2Name,true,false,false,null);
// 7.绑定队列与交换机
/**
* public com.rabbitmq.client.AMQP.Queue.BindOk queueBind(String queue, String exchange, String routingKey)
* queue:绑定的队列名
* exchange:交换机名称
*routingKey:路由键,绑定规则,如果使用的是fanout(广播),则routingKey为空
*/
//所有异常日志保存数据库、所有的订单日志保存数据库
channel.queueBind(queue2Name,exchangeName,"#.error");
channel.queueBind(queue2Name,exchangeName,"order.*");
//所有的日志均打印
channel.queueBind(queue1Name,exchangeName,"*.*");
// 8.发送消息
/**
* public void basicPublish(String exchange, String routingKey, BasicProperties props, byte[] body)
* exchange:交换机名称。简单模式下交换机默认为 ""
* routingKey:路由名称。注意:使用默认交换机,则routingKey要与队列名一致才能正常路由
* props:配置信息
*/
String order1 = "【INFO】:ORDER1~~~";
channel.basicPublish(exchangeName,"order.*",null,order1.getBytes());
String order2 = "【WARNING】:ORDER2~~~";
channel.basicPublish(exchangeName,"order.*",null,order2.getBytes());
String order3 = "【ERROR】:ORDER3~~~";
channel.basicPublish(exchangeName,"order.*",null,order3.getBytes());
String other1 = "【INFO】:Other1~~~";
channel.basicPublish(exchangeName,"*.*",null,other1.getBytes());
String other2 = "【ERROR】:Other2~~~";
channel.basicPublish(exchangeName,"#.error",null,other2.getBytes());
// 9.释放资源
System.out.println("end...");
channel.close();
connection.close();
}
}
消费者
<dependencies>
<!--rabbitmq java客户端-->
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.9.0</version>
</dependency>
</dependencies>
/**
* 接收/消费消息
*/
public class MessageConsumer_Topic1 {
public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
// 1.创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
// 2.设置参数
factory.setHost("43.139.51.247"); // ip 默认值:localhost
factory.setPort(5672); // 端口 默认值:5672
factory.setVirtualHost("/zhd"); // 虚拟机 默认值:/
factory.setUsername("czk");
factory.setPassword("czk");
// 3.创建连接Connection
Connection connection = factory.newConnection();
// 4.创建Channel
Channel channel = connection.createChannel();
// 5.接收消息
/**
* public String basicConsume(String queue, boolean autoAck, Consumer callback)
* queue:队列名称
* autoAck:收到消息是否自动确认,消息丢失相关
* callback:回调对象
*/
Consumer consumer = new DefaultConsumer(channel){
/**
* 回调方法
* consumerTag:标识
* envelope:获取一些消息,交换机、路由key
* properties:配置消息
* body:数据
*/
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("body:"+new String(body));
System.out.println("Topic1:日志打印");
System.out.println("------------------------------------");
}
};
channel.basicConsume("topic_queue1",true,consumer);
// 监听程序,无需关闭资源
}
}
/**
* 接收/消费消息
*/
public class MessageConsumer_Topic2 {
public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
// 1.创建连接工厂
ConnectionFactory factory = new ConnectionFactory();
// 2.设置参数
factory.setHost("43.139.51.247"); // ip 默认值:localhost
factory.setPort(5672); // 端口 默认值:5672
factory.setVirtualHost("/zhd"); // 虚拟机 默认值:/
factory.setUsername("czk");
factory.setPassword("czk");
// 3.创建连接Connection
Connection connection = factory.newConnection();
// 4.创建Channel
Channel channel = connection.createChannel();
// 5.接收消息
/**
* public String basicConsume(String queue, boolean autoAck, Consumer callback)
* queue:队列名称
* autoAck:收到消息是否自动确认,消息丢失相关
* callback:回调对象
*/
Consumer consumer = new DefaultConsumer(channel){
/**
* 回调方法
* consumerTag:标识
* envelope:获取一些消息,交换机、路由key
* properties:配置消息
* body:数据
*/
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("body:"+new String(body));
System.out.println("Topic2:日志数据保存");
System.out.println("------------------------------------");
}
};
channel.basicConsume("topic_queue2",true,consumer);
// 监听程序,无需关闭资源
}
}
工作模式(SpringBoot版)
Topic通配符模式
生产者
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
</dependency>
</dependencies>
spring:
rabbitmq:
host: 43.139.51.247
port: 5672
username: czk
password: czk
virtual-host: /zhd
@Configuration
public class TopicRabbitMQConfig {
public static final String EXCHANGE_NAME = "boot_topic_exchange";
public static final String QUEUE_NAME_ALL = "topic_queue_all";
public static final String QUEUE_NAME_ORDER = "topic_queue_order";
public static final String QUEUE_NAME_ERROR = "topic_queue_error";
public static final String TOPIC_KEY_1 = "#.#";
public static final String TOPIC_KEY_2 = "order.*";
public static final String TOPIC_KEY_3 = "#.error";
// 1.交换机
@Bean("topic_exchange")
public Exchange topicExchange(){
return ExchangeBuilder.topicExchange(EXCHANGE_NAME).durable(true).build();
}
// 2.队列
@Bean("topic_queue_all")
public Queue topicQueue1(){
return QueueBuilder.durable(QUEUE_NAME_ALL).build();
}
@Bean("topic_queue_order")
public Queue topicQueue2(){
return QueueBuilder.durable(QUEUE_NAME_ORDER).build();
}
@Bean("topic_queue_error")
public Queue topicQueue3(){
return QueueBuilder.durable(QUEUE_NAME_ERROR).build();
}
// 3.绑定
@Bean
public Binding bindTopicExchangeQueue1(@Qualifier("topic_exchange")Exchange exchange,@Qualifier("topic_queue_all") Queue queue){
return BindingBuilder.bind(queue).to(exchange).with(TOPIC_KEY_1).noargs();
}
@Bean
public Binding bindTopicExchangeQueue2(@Qualifier("topic_exchange")Exchange exchange,@Qualifier("topic_queue_order") Queue queue){
return BindingBuilder.bind(queue).to(exchange).with(TOPIC_KEY_2).noargs();
}
@Bean
public Binding bindTopicExchangeQueue3(@Qualifier("topic_exchange")Exchange exchange,@Qualifier("topic_queue_error") Queue queue){
return BindingBuilder.bind(queue).to(exchange).with(TOPIC_KEY_3).noargs();
}
}
@SpringBootTest
@RunWith(SpringRunner.class)
public class RabbitMqProducerTest {
@Autowired
private RabbitTemplate template;
@Test
public void topic_all(){
template.convertAndSend(TopicRabbitMQConfig.EXCHANGE_NAME,"key-all","topic-queue-all");
}
@Test
public void topic_order(){
template.convertAndSend(TopicRabbitMQConfig.EXCHANGE_NAME,"order.key","topic-queue-order");
}
@Test
public void topic_error(){
template.convertAndSend(TopicRabbitMQConfig.EXCHANGE_NAME,"key.error","topic-queue-order");
}
@Test
public void topic_order_error(){
template.convertAndSend(TopicRabbitMQConfig.EXCHANGE_NAME,"order.error","topic-queue-order-error");
}
}
消费者
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
</dependency>
</dependencies>
spring:
rabbitmq:
host: 43.139.51.247
port: 5672
username: czk
password: czk
virtual-host: /zh
@Component
public class RabbitMQListener {
@RabbitListener(queues = "topic_queue_all")
public void listenerTopicAll(Message message){
System.out.println("TopicQueue All:"+new String(message.getBody()));
}
@RabbitListener(queues = "topic_queue_order")
public void listenerTopicOrder(Message message){
System.out.println("TopicQueue Order:"+new String(message.getBody()));
}
@RabbitListener(queues = "topic_queue_error")
public void listenerTopicError(Message message){
System.out.println("TopicQueue Error:"+new String(message.getBody()));
}
}
高级特性
消息可靠性
持久可靠性
- 交换机:持久
- 队列:持久
- 消息:持久(队列持久化后消息就持久化了)
@Bean("confirm_exchange")
public Exchange confirmExchange(){
return ExchangeBuilder.topicExchange(EXCHANGE_NAME).durable(true).build();
}
@Bean("confirm_queue")
public Queue confirmQueue(){
return QueueBuilder.durable(QUEUE_NAME).build();
}
Broker可靠性
RabbitMQ集群保证Broker的集群,保证MQ可用性,从而保证消息可靠性
传输可靠性
生产者=>中间件
Confirm确认模式
生产者=>交换机Exchange,成功-ack-true,失败-ack-false
return退回模式
交换机Exchange=>队列Queue,成功-ack-true,失败-ack-false
备份交换机方案(另一种回退模式)
交换机Exchange=>队列Queue,成功-不做任何操作,失败-将消息发送给备份交换机
confirm-return案例
生产者
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
</dependencies>
spring:
rabbitmq:
host: 43.139.51.247
port: 5672
username: czk
password: czk
virtual-host: /zhd
publisher-confirm-type: correlated # confirm模式(异步) simple(同步)
publisher-returns: true
@Configuration
public class ConfirmRabbitMQConfig {
public static final String EXCHANGE_NAME = "confirm_exchange";
public static final String QUEUE_NAME = "confirm_queue";
public static final String Confirm_KEY="confirm.#";
@Bean("confirm_exchange")
public Exchange confirmExchange(){
return ExchangeBuilder.topicExchange(EXCHANGE_NAME).durable(true).build();
}
@Bean("confirm_queue")
public Queue confirmQueue(){
return QueueBuilder.durable(QUEUE_NAME).build();
}
@Bean
public Binding bindConfirmExchangeQueue(@Qualifier("confirm_exchange")Exchange exchange,@Qualifier("confirm_queue") Queue queue){
return BindingBuilder.bind(queue).to(exchange).with(Confirm_KEY).noargs();
}
}
@Component
@Slf4j
public class MyMessageCallBack implements RabbitTemplate.ConfirmCallback, RabbitTemplate.ReturnCallback {
@Autowired
RabbitTemplate rabbitTemplate;
@PostConstruct
public void init(){
// 为RabbitTemplate设置confirm回调
rabbitTemplate.setConfirmCallback(this);
// 为RabbitTemplate设置return回调
rabbitTemplate.setReturnCallback(this);
}
/**
* confirm模式
* @param correlationData:相关配置信息,消息唯一标识
* @param ack:exchange交换机是否成功接收到消息,true-成功、false-失败
* @param cause:失败原因
*/
@Override
public void confirm(CorrelationData correlationData, boolean ack, String cause) {
String messageID = correlationData==null?"":correlationData.getId();
if (ack){
log.info("exchange成功接收消息,不代表消息成功进入队列=>ack:{},correlationData:{},cause:{}",ack,correlationData,cause);
}else {
log.info("exchange接收消息失败=>ack:{},correlationData:{},cause:{}",ack,correlationData,cause);
}
}
/**
* return模式
* @param message:消息对象
* @param replyCode:错误码
* @param replyText:错误消息
* @param exchange:交换机
* @param routingKey:路由键
*/
@Override
public void returnedMessage(Message message, int replyCode, String replyText, String exchange, String routingKey) {
log.warn("消息由Exchange传递到Queue失败!!!");
log.warn("错误码:{}",replyCode);
log.warn("错误信息:{}",replyText);
log.warn("交换机:{}",exchange);
log.warn("路由键:{}",routingKey);
log.warn("可以在方法内写补偿方法...");
}
}
@SpringBootApplication
public class RabbitMQAdvancedApp {
public static void main(String[] args) {
ConfigurableApplicationContext run = SpringApplication.run(RabbitMQAdvancedApp.class, args);
RabbitTemplate rabbitTemplate = run.getBean(RabbitTemplate.class);
// confirm(rabbitTemplate);
returnBack(rabbitTemplate);
}
public static void confirm(RabbitTemplate rabbitTemplate){
CorrelationData correlationData1 = new CorrelationData("1");
rabbitTemplate.convertAndSend(ConfirmRabbitMQConfig.EXCHANGE_NAME,"confirm.test","confirm-queue-message",correlationData1);
// 模拟消息传递不到交换机=>恶意写错交换机名
CorrelationData correlationData2 = new CorrelationData("2");
rabbitTemplate.convertAndSend(ConfirmRabbitMQConfig.EXCHANGE_NAME+"error","confirm.test","confirm-queue-message",correlationData2);
}
public static void returnBack(RabbitTemplate rabbitTemplate){
CorrelationData correlationData1 = new CorrelationData("1");
rabbitTemplate.convertAndSend(ConfirmRabbitMQConfig.EXCHANGE_NAME,"confirm.test","confirm-queue-message",correlationData1);
// 模拟消息从交换机传递不到队列=>恶意写错routingKey
CorrelationData correlationData2 = new CorrelationData("2");
rabbitTemplate.convertAndSend(ConfirmRabbitMQConfig.EXCHANGE_NAME,"confirmError.test","confirm-queue-message",correlationData2);
}
}
消费者
@Component
@Slf4j
public class MessageCallBackListener {
@RabbitListener( queues = "confirm_queue",ackMode = "MANUAL")
public void onMessage(Message message,Channel channel) throws Exception{
long deliveryTag = message.getMessageProperties().getDeliveryTag();
try {
// 1.接收消息
log.info("消息:{}",message);
// 2.业务处理
log.info("业务处理...");
int i = 1/0; // 模拟业务异常
// 3.手动签收
channel.basicAck(deliveryTag,true);
} catch (Exception e) {
// 4.出现异常:拒绝签收
// 第三个参数: requeue重回队列,如果设置为true,则消息重新回到queue,broker会重新发送该消息给消费端
channel.basicNack(deliveryTag,true,true);
}
}
}
中间件=>消费者(Consumer ACK机制)
3种确认方式
- acknowledge=“none” ,默认确认方式
- 当消息一旦被Consumer接收后,则自动确认收到,并将message从RabbitMQ的消息缓存中移除,但实际业务中消息接收后,可能处理业务时出现异常,那么该消息就丢失了。
- acknowledge=“manual”,手动确认方式,当处理消息逻辑出现问题时,可以回退消息,也可以编写补偿方法
- 设置手动确认,在业务处理完成后,调用channel.basicAck(),手动签收,如果出现异常,则调用channel.basicNack()方法,让其自动重新发送消息。
- acknowledge=“auto”,根据异常情况确认,更为复杂的操作
生产者
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
</dependencies>
spring:
rabbitmq:
host: 43.139.51.247
port: 5672
username: czk
password: czk
virtual-host: /zhd
@Component
@Slf4j
public class MessageCallBackListener implements ChannelAwareMessageListener {
@RabbitListener( queues = "confirm_queue",ackMode = "MANUAL")
public void onMessage(Message message,Channel channel) throws Exception{
long deliveryTag = message.getMessageProperties().getDeliveryTag();
try {
// 1.接收消息
log.info("消息:{}",message);
// 2.业务处理
log.info("业务处理...");
int i = 1/0; // 模拟业务异常
// 3.手动签收
channel.basicAck(deliveryTag,true);
} catch (Exception e) {
// 4.出现异常:拒绝签收
// 第三个参数: requeue重回队列,如果设置为true,则消息重新回到queue,broker会重新发送该消息给消费端
channel.basicNack(deliveryTag,true,true);
}
}
}
public static void sendAck(RabbitTemplate rabbitTemplate) {
CorrelationData correlationData3 = new CorrelationData("3");
rabbitTemplate.convertAndSend(MessageCallBackRabbitMQConfig.EXCHANGE_NAME, "confirm.ack", "confirm-queue-message-ack", correlationData3);
}
消费端限流(消费填谷)
前置条件:
- ACK手动确认
- 设置perfetch
生产者:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
</dependencies>
perfetch方式:
- spring.rabbitmq.listener.simple.prefetch:1 ,一次拿一条消息,消费确认后才能继续拿消息
- spring.rabbitmq.listener.direct.prefetch:1,一次拿一条消息,消息如果为未确认状态,则消息不会被消费,最终仍然留在队列中
spring:
rabbitmq:
host: 43.139.51.247
port: 5672
username: czk
password: czk
virtual-host: /zhd
listener:
simple:
prefetch: 2 # 每次最多取到2条消息,两条消息被确认后才会继续取消息
@Component
@Slf4j
public class QosMessageListener {
@RabbitListener( queues = "confirm_queue",ackMode = "MANUAL")
public void onMessage(Message message, Channel channel) throws Exception{
long deliveryTag = message.getMessageProperties().getDeliveryTag();
Thread.sleep(1000);
// 模拟业务逻辑
log.info("业务逻辑模拟...{}",new String(message.getBody()));
// 手动签收限流才能失效
channel.basicAck(deliveryTag,true);
}
}
public static void sendQos(RabbitTemplate rabbitTemplate) {
for (int i = 0; i < 10; i++) {
CorrelationData correlationData = new CorrelationData(i + "");
rabbitTemplate.convertAndSend(MessageCallBackRabbitMQConfig.EXCHANGE_NAME, "confirm.qos", "confirm-queue-message-qos", correlationData);
}
}
消费者:
@Component
@Slf4j
public class QosMessageListener {
@RabbitListener( queues = "confirm_queue",ackMode = "MANUAL")
public void onMessage(Message message, Channel channel) throws Exception{
long deliveryTag = message.getMessageProperties().getDeliveryTag();
Thread.sleep(1000);
// 模拟业务逻辑
log.info("业务逻辑模拟...{}",new String(message.getBody()));
// 手动签收限流才能失效
channel.basicAck(deliveryTag,true);
}
}
TTL(Time To LiVE)过期时间
过期设置方式
- 消息:对消息进行过期时间设置
- 队列:对这个队列的消息进行过期时间设置
消息移除方式
- 如果消息与队列同时设置了过期时间,以时间短的为主
- 队列消息过期时间一到,则移除队列消息
- 单独消息过期时间一到,判断该消息是否在队列首部,是则移除,否则不移除,等到该消息即将被消费(到了队列首部)时发现过期才移除
GUI演示
添加队列
添加交换机
绑定队列
发送消息
10s后,消息过期…
代码方式实现
@Configuration
public class TTLRabbitMQConfig {
public static final String EXCHANGE_NAME = "ttl_exchange";
public static final String QUEUE_NAME = "ttl_queue";
public static final String TTL_KEY = "ttlKey.#";
// 1.交换机
@Bean("ttl_exchange")
public Exchange ttlExchange(){
return ExchangeBuilder.topicExchange(EXCHANGE_NAME).durable(true).build();
}
// 2.队列
@Bean("ttl_queue")
public Queue ttlQueue(){
// 设置参数,x-message-ttl=10000,表示10s过期
return QueueBuilder.durable(QUEUE_NAME).withArgument("x-message-ttl", 100000).build();
}
// 3.绑定
@Bean
public Binding bindTtlExchangeQueue(@Qualifier("ttl_exchange") Exchange exchange,@Qualifier("ttl_queue") Queue queue){
return BindingBuilder.bind(queue).to(exchange).with(TTL_KEY).noargs();
}
}
public static void sendQueueTTL(RabbitTemplate rabbitTemplate) {
for (int i = 0; i < 10; i++) {
CorrelationData correlationData = new CorrelationData(i + "");
rabbitTemplate.convertAndSend(TTLRabbitMQConfig.EXCHANGE_NAME, "ttlKey.test", "confirm-queue-message-ttl", correlationData);
}
}
public static void sendMessageTTL(RabbitTemplate rabbitTemplate) {
CorrelationData correlationData = new CorrelationData("0");
MessagePostProcessor messagePostProcessor = new MessagePostProcessor() {
@Override
public Message postProcessMessage(Message message) throws AmqpException {
message.getMessageProperties().setExpiration("5000"); // 5s过期
return message;
}
};
rabbitTemplate.convertAndSend(TTLRabbitMQConfig.EXCHANGE_NAME, "ttlKey.test", "confirm-queue-message-ttl",messagePostProcessor,correlationData);
}
public static void sendMessageQueueTTL(RabbitTemplate rabbitTemplate){
CorrelationData correlationData1 = new CorrelationData("1");
rabbitTemplate.convertAndSend(TTLRabbitMQConfig.EXCHANGE_NAME, "ttlKey.test", "confirm-queue-message-ttl", correlationData1);
// 消息2,不在队列首部,即使过期了,在gui页面也看不到消息剩余1条,得等到队列消息过期才能看到消息数清零
CorrelationData correlationData2 = new CorrelationData("2");
MessagePostProcessor messagePostProcessor = new MessagePostProcessor() {
@Override
public Message postProcessMessage(Message message) throws AmqpException {
message.getMessageProperties().setExpiration("5000"); // 5s过期
return message;
}
};
rabbitTemplate.convertAndSend(TTLRabbitMQConfig.EXCHANGE_NAME, "ttlKey.test", "confirm-queue-message-ttl",messagePostProcessor,correlationData2);
}
DLX死信交换机/死信队列
消息死信前置条件
- 队列消息长度达到限制
- 消费者拒绝消费消息,baskNack/basicReject,并且不把消息重新放入原目标队列,requeue=false
- 原队列存在消息过期设置,消息达到超时时间未被消费
生产者:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
</dependencies>
spring:
rabbitmq:
host: 43.139.51.247
port: 5672
username: czk
password: czk
virtual-host: /zhd
publisher-confirm-type: correlated # confirm模式(异步) simple(同步,不建议)
publisher-returns: true
@Configuration
public class DLXRabbitMQConfig {
public static final String NORMAL_EXCHANGE_NAME = "normal_exchange";
public static final String DLX_EXCHANGE_NAME = "dlx_exchange";
public static final String NORMAL_QUEUE_NAME = "normal_queue";
public static final String DLX_QUEUE_NAME = "dlx_queue";
public static final String NORMAL_ROUTING_KEY = "normal.#";
public static final String DLX_ROUTING_KEY = "dlx.#";
@Bean("dlx_exchange")
public Exchange dlxExchange() {
return ExchangeBuilder.topicExchange(DLX_EXCHANGE_NAME).durable(true).build();
}
@Bean("normal_exchange")
public Exchange normalExchange() {
return ExchangeBuilder.topicExchange(NORMAL_EXCHANGE_NAME).durable(true).build();
}
@Bean("dlx_queue")
public Queue dlxQueue() {
return QueueBuilder.durable(DLX_QUEUE_NAME).build();
}
@Bean("normal_queue")
public Queue normalQueue() {
Map<String, Object> arguments = new HashMap<>();
arguments.put("x-message-ttl", 30000); // 队列设置过期时间,过期消息进入死信队列
arguments.put("x-max-length", 10); // 队列设置最大长度,超过长度的将进入死信队列
arguments.put("x-dead-letter-exchange", DLX_EXCHANGE_NAME); // 绑定私信交换机dlx_exchange
arguments.put("x-dead-letter-routing-key", "dlx.test"); // 绑定私信队列dlx_queue,routingKey未=为dlx.#,所有这里为dlx.test,test为任意值均可
return QueueBuilder.durable(NORMAL_QUEUE_NAME).withArguments(arguments).build();
}
@Bean
public Binding bindDlxExchangeQueue(@Qualifier("dlx_exchange") Exchange exchange, @Qualifier("dlx_queue") Queue queue) {
return BindingBuilder.bind(queue).to(exchange).with(DLX_ROUTING_KEY).noargs();
}
@Bean
public Binding bindNormalExchangeQueue(@Qualifier("normal_exchange") Exchange exchange, @Qualifier("normal_queue") Queue queue) {
return BindingBuilder.bind(queue).to(exchange).with(NORMAL_ROUTING_KEY).noargs();
}
}
// 模拟消息过期进入私信
public static void sendDlx(RabbitTemplate rabbitTemplate) {
// 5s后过期的消息将进入死信队列
CorrelationData correlationData1 = new CorrelationData("1");
MessagePostProcessor messagePostProcessor = new MessagePostProcessor() {
@Override
public Message postProcessMessage(Message message) throws AmqpException {
message.getMessageProperties().setExpiration("5000"); // 5s过期
return message;
}
};
// 10个消息后的消息将进入死信,未进入死信的消息,在队列过期时间30s后,消息未被消费将进入死信
rabbitTemplate.convertAndSend(DLXRabbitMQConfig.NORMAL_EXCHANGE_NAME, "normal.test", "confirm-queue-message-dlx", messagePostProcessor, correlationData1);
for (int i = 0 ;i<20;i++){
rabbitTemplate.convertAndSend(DLXRabbitMQConfig.NORMAL_EXCHANGE_NAME,"normal.test","dlx-dlx-dlx");
}
}
// 模拟消息接收方处理移除,消息进入死信
public static void sendDlxError(RabbitTemplate rabbitTemplate){
CorrelationData correlationData = new CorrelationData("500");
rabbitTemplate.convertAndSend(DLXRabbitMQConfig.NORMAL_EXCHANGE_NAME, "normal.test", "confirm-queue-message-dlx", correlationData);
}
消费者
模拟义务处理异常进入死信
@Component
@Slf4j
public class DLXMessageListener {
@RabbitListener(queues = "normal_queue",ackMode = "MANUAL")
public void dlxListener(Message message, Channel channel) throws IOException {
long deliveryTag = message.getMessageProperties().getDeliveryTag();
try {
log.info("逻辑处理,模拟移除进入死信");
int i = 1/0;
channel.basicAck(deliveryTag,true);
} catch (Exception e) {
channel.basicNack(deliveryTag,true,false);
}
};
}
延迟队列
RabbitMQ并没有提供延迟队列,但可以通过 TTL + 死信队列 来实现 延迟队列。
生产者:
@Configuration
public class DelayRabbitMQConfig {
public static final String DELAY_EXCHANGE_NAME = "delay_exchange";
public static final String GENERAL_EXCHANGE_NAME = "general_exchange";
public static final String DELAY_QUEUE_NAME = "delay_queue";
public static final String GENERAL_QUEUE_NAME="general_queue";
public static final String DELAY_ROUTING_KEY = "delay.#";
public static final String GENERAL_ROUTING_key="general.#";
@Bean("delay_exchange")
public Exchange delayExchange(){
return ExchangeBuilder.topicExchange(DELAY_EXCHANGE_NAME).durable(true).build();
}
@Bean("general_exchange")
public Exchange generalExchange(){
return ExchangeBuilder.topicExchange(GENERAL_EXCHANGE_NAME).durable(true).build();
}
@Bean("delay_queue")
public Queue delayQueue(){
return QueueBuilder.durable(DELAY_QUEUE_NAME).build();
}
@Bean("general_queue")
public Queue generalQueue(){
Map<String, Object> arguments = new HashMap<>();
arguments.put("x-message-ttl", 10000); // 队列设置过期时间,过期消息进入死信队列
arguments.put("x-dead-letter-exchange", DELAY_EXCHANGE_NAME); // 绑定私信交换机dlx_exchange
arguments.put("x-dead-letter-routing-key", "delay.test"); // 绑定死信队列delay_queue,routingKey未=为delay.#,所有这里为delay.test,test为任意值均可
return QueueBuilder.durable(GENERAL_QUEUE_NAME).withArguments(arguments).build();
}
@Bean
public Binding bindDelayExchangeQueue(@Qualifier("delay_exchange")Exchange exchange,@Qualifier("delay_queue")Queue queue){
return BindingBuilder.bind(queue).to(exchange).with(DELAY_ROUTING_KEY).noargs();
}
@Bean
public Binding bindGeneralExchangeQueue(@Qualifier("general_exchange")Exchange exchange,@Qualifier("general_queue")Queue queue){
return BindingBuilder.bind(queue).to(exchange).with(GENERAL_ROUTING_key).noargs();
}
}
public static void sendDelay(RabbitTemplate rabbitTemplate) {
rabbitTemplate.convertAndSend(DelayRabbitMQConfig.GENERAL_EXCHANGE_NAME, "general.test", "delay-delay-delay");
}
消费者:
@Component
@Slf4j
public class DelayMessageListener {
@RabbitListener(queues = "delay_queue")
public void delayListener(Message message) {
log.info("延迟队列消息:{}",new String(message.getBody()));
}
}
缺陷
队列消息过期时间检测,只检测队首,当队首消息没到过期时间,那么即使队中消息中存在过期了的消息,那这些消息也是不会被加入到死信队列的,会出现一定的问题。任何解决?插件!
延迟插件
docker安装rabbitmq中插件安装参考:rabbitmq插件在docker中的安装
装完插件记得重启服务:systemctl restart rabbitmq_server
基于交换机与基于TTL
基于TTL
基于交换机:
预期结果
实现
1.安装并开启延迟插件,成功示例
2.配置延迟队列
package com.czk.rabbitmq.config;
import org.springframework.amqp.core.*;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.HashMap;
import java.util.Map;
@Configuration
public class DelayByPluginRabbitMQConfig {
public static final String EXCHANGE_NAME = "delay_plugin_exchange";
private static final String EXCHANGE_TYPE = "x-delayed-message";
public static final String QUEUE_NAME = "delay_plugin_queue";
public static final String DELAY_PLUGIN_ROUTING_KEY = "pluginDelayKey";
@Bean("delay_plugin_exchange")
public Exchange delayPluginExchange(){
/**
* public CustomExchange(String name, String type, boolean durable, boolean autoDelete, Map<String, Object> arguments)
* name:交换机名
* type:交换机类型
* durable:是否持久化
* autoDelete:是否自动删除
* arguments:其他参数
*/
Map<String,Object> arguments = new HashMap<>();
arguments.put("x-delayed-type","direct");
return new CustomExchange(EXCHANGE_NAME,EXCHANGE_TYPE,true,false,arguments);
}
@Bean("delay_plugin_queue")
public Queue delayPluginQueue(){
return QueueBuilder.durable(QUEUE_NAME).build();
}
@Bean
public Binding bindDelayPluginExchangeQueue(@Qualifier("delay_plugin_exchange") Exchange exchange,@Qualifier("delay_plugin_queue")Queue queue){
return BindingBuilder.bind(queue).to(exchange).with(DELAY_PLUGIN_ROUTING_KEY).noargs();
}
}
3.测试
public static void sendDelayPlugin(RabbitTemplate rabbitTemplate,Integer timeout,String id){
CorrelationData correlationData = new CorrelationData(id);
MessagePostProcessor messagePostProcessor = new MessagePostProcessor() {
@Override
public Message postProcessMessage(Message message) throws AmqpException {
message.getMessageProperties().setDelay(timeout); //⭐:看这里
return message;
}
};
rabbitTemplate.convertAndSend(DelayByPluginRabbitMQConfig.EXCHANGE_NAME,
DelayByPluginRabbitMQConfig.DELAY_PLUGIN_ROUTING_KEY,
"delay-plugin-message",
messagePostProcessor,
correlationData
);
}
sendDelayPlugin(rabbitTemplate,30000,"1111");
sendDelayPlugin(rabbitTemplate,1000,"2222");
@Component
@Slf4j
public class DelayPluginMessageListener {
@RabbitListener(queues = "delay_plugin_queue")
public void delayPluginListener(Message message){
log.info("插件:接收消息:{}",message);
}
}
4.结果演示
惰性队列
-
消息存放
- 正常情况:内存
- 异常情况:磁盘
-
使用场景:消费者宕机 + 消息堆积
-
原因:消息存放内存,消息堆积过多,占用大量内存,此时存放入磁盘更为合适
-
弊端:性能较低
日志与监控
RabbitMQ默认日志存放路径:/var/log/rabbitmq/rabbit@xxx.log
消息可靠性分析与追踪
**慎用:**开启后影响一定的性能
firehose
firehose开启后,rabbitmq中存在一个trace交换机,会在你发送消息后,自身再发送一个消息,该消息包含了更为消息的消息信息。
rabbitmq-plugins
添加插件,tracing,启用后将再控制台上多出一个导航按钮Tracing,可以添加规则。这里的规则是记录所有的消息
应用问题
消息可靠性保障-思路
数据库保存消息与对比、定期检测
消息幂等性处理-思路
- 方案1:数据库乐观锁机制保证幂等性
- 方案2:利用redis执行setnx命令,天然具有幂等性,无需实现不重复消费
消息优先级
实现
优先级Maximum priority范围为0255,推荐范围010,范围过大将影响CPU执行效率,影响排序效率,性能降低。
RabbitMQ高可用集群
镜像队列
队列数据备份,如:节点1队列消息备份到节点2队列中,使得节点2可以有节点1的消息,保证消息不丢失,当然,一个节点的数据可以有多个节点备份
^mirrior:代表队列名以mirrior为前缀
- ha-params:2 代表始终2个节点保持有备份数据功能(如node1、node2、node3,以node1为主节点产生队列,其随机选定另一个节点备份镜像队列(这里以node2节点备份镜像队列),这时node1宕机了,rabbitmq服务器将自动选出一个node(这里是node3)来备份镜像队列,该工程中,始终有2个节点保持有备份数据功能)
负载均衡
Haproxy + Keepalive
数据同步
方案1:Federation Exchange
**背景:**距离较远,通讯延迟问题。类似于nacos中的cluster,如:北京用户访问北京机房、广州用户访问广州机房。
**问题:**机房数据同步问题怎么解决?Federation Exchange!
Federation Exchange是Rabbitmq中的插件,自带,默认不开启,使用时需要手动开启。
1.开启插件
2.在node1中设置交换机XX,在node2中配置XX为其上游交换机
3.设置策略
4.成果结果
方案2:Federation Queue
1.原理:
2.设置上油(同上upstream)
3.添加策略:
4.结果演示:
方案3:Shovel插件
1.开启插件
2.配置同步
3.结果演示