RaabbitMQ学习(中)
4. 发布确认
1. 原理
生产者将channel设置为confirm模式,一旦信道进入confirm模式,所有在该信道上发布的消息都会被指派一个唯一的ID(从1开始),一旦消息别投递到所有匹配的队列之后,broker会发生一个确认给生产者(包含消息的唯一ID),使得生产者知道消息已经正确达到目的队列。前提必须是消息和队列已经持久化了,那么确认消息会在消息写入磁盘后发出,另外这个确认也可以设置multiple域(是否需要批量确认)。
2. 策略
发布确认默认是不开启的,需要调用方法channel.confirmSelect()
,每当想使用发布确认,都需要在channel上调用该方法。
a. 单个确认发布
单个确认发布也就是同步确认发布,发布一个消息之后等它被确认发布,才能发送下一个消息。
channel.waitForConfirms()
方法只有在消息被确认时才返回,如果在指定时间范围内这个消息没有被确认会抛出异常
缺点:发布速度特别慢
导入依赖
<dependencies>
<!--rabbitmq的依赖客户端-->
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.15.0</version>
</dependency>
<!--操作IO流的依赖-->
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.6</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>8</source>
<target>8</target>
</configuration>
</plugin>
</plugins>
</build>
由于每次使用MQ都需要创建工厂,获取连接,获取信道,于是将这些操作封装在一个工具类中,方便使用
public class RabbitMQUtils {
public static Channel getChannel() throws Exception{
// 创建RabbitMQ连接工厂 设置工厂IP,用户名以及密码
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("linux1");
factory.setUsername("admin");
factory.setPassword("hz1234");
// 创建连接 获取信道channel 生成队列
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
return channel;
}
}
public static void publishMsgIndividually() throws Exception {
Channel channel = RabbitMQUtils.getChannel();
String queueName = UUID.randomUUID().toString().substring(0,8);
// 声明队列
channel.queueDeclare(queueName, false, false, false, null);
// 开启发布确认
channel.confirmSelect();
long begin = System.currentTimeMillis();
for (int i = 0; i < MSG_NUMBER; i++) {
String msg = "消息" + i;
channel.basicPublish("", queueName, null,msg.getBytes("UTF-8"));
// 单个发布确认
boolean flag = channel.waitForConfirms();
if(flag) {
System.out.println(msg + "发送成功");
}
}
long end = System.currentTimeMillis();
System.out.println("===============================================");
System.out.println("单个确认发布"+MSG_NUMBER+"条消息,花费 "+ (end - begin)+"ms");
}
b. 批量确认发布
发布一批消息然后确认可以极大的提高吞吐量
缺点:当发生故障时,不知道是哪个消息出现问题,必须将整个批处理保存在内存中,以记录重要的消息然后重新发布消息
public static void publishMsgBatch() throws Exception {
Channel channel = RabbitMQUtils.getChannel();
String queueName = UUID.randomUUID().toString().substring(0,8);
// 声明队列
channel.queueDeclare(queueName, false, false, false, null);
// 开启发布确认
channel.confirmSelect();
long begin = System.currentTimeMillis();
int batchSize = 100;
for (int i = 0; i < MSG_NUMBER; i++) {
String msg = "消息" + i;
channel.basicPublish("", queueName, null,msg.getBytes("UTF-8"));
if(i>98 && (i+1)%batchSize==0){
boolean flag = channel.waitForConfirms();
if(flag){
System.out.println("第" + (i+1)/100 + "个" + "batch消息发送成功");
}
}
}
long end = System.currentTimeMillis();
System.out.println("===============================================");
System.out.println("批量确认发布"+MSG_NUMBER+"条消息,花费 "+ (end - begin)+"ms");
}
c. 异步确认发布
利用回调函数来达到消息可靠性传递的,生产者不会将消息直接发送给queue,而是一个Map(key为消息序号,value为消息内容),这个中间件也是通过函数回调来保证是否投递成功:
确认收到消息回调ackCallback,反之回调nackCallback
public static void publishMsgAsync() throws Exception {
Channel channel = RabbitMQUtils.getChannel();
String queueName = UUID.randomUUID().toString().substring(0,8);
// 声明队列
channel.queueDeclare(queueName, false, false, false, null);
// 开启发布确认
channel.confirmSelect();
long begin = System.currentTimeMillis();
// 消息确认成功的回调函数(tag为消息序号 multiple是否批量)
ConfirmCallback ackCallback = (tag, multiple) -> {
System.out.println("已确认的消息序号为: " + tag);
};
// 消息确认失败的回调函数
ConfirmCallback nackCallback = (tag, multiple) -> {
System.out.println("未确认的消息序号为: " + tag);
};
// 准备消息的监听器,监听哪些消息发送成功 哪些消息发送失败
channel.addConfirmListener(ackCallback, nackCallback);
for (int i = 0; i < MSG_NUMBER; i++) {
String msg = "消息" + i;
channel.basicPublish("", queueName, null,msg.getBytes("UTF-8"));
}
long end = System.currentTimeMillis();
System.out.println("===============================================");
System.out.println("异步确认发布"+MSG_NUMBER+"条消息,花费"+ (end - begin)+"ms");
}
那么如何去处理异步确认中还未被确认的消息?
将未确认的消息放到一个基于内存的并且能被发布线程所访问的队列,这里使用ConcurrentLinkedQueue队列保证监听器线程与发消息线程之间的消息传递
public static void publishMsgAsync() throws Exception {
Channel channel = RabbitMQUtils.getChannel();
String queueName = UUID.randomUUID().toString().substring(0,8);
// 声明队列
channel.queueDeclare(queueName, false, false, false, null);
// 开启发布确认
channel.confirmSelect();
// 1.记录需要发送的全部消息: 使用线程安全且有序的hash表 发送消息时将key和value依次存入到hash表中
// 2.在消息确认成功的回调函数中 从hash表中删除掉已被确认的消息
ConcurrentSkipListMap<Long, String> map = new ConcurrentSkipListMap<>();
// 消息确认成功的回调函数(tag为消息序号 multiple是否批量)
ConfirmCallback ackCallback = (tag, multiple) -> {
if(multiple) {
ConcurrentNavigableMap<Long, String> headMap = map.headMap(tag);
headMap.clear();
}else{
map.remove(tag);
}
System.out.println("已确认的消息序号为: " + tag);
};
// 消息确认失败的回调函数
ConfirmCallback nackCallback = (tag, multiple) -> {
String msg = map.get(tag);
System.out.println("未确认的消息: " + msg + "未确认的消息序号为: " + tag);
};
// 准备消息的监听器(异步),监听哪些消息发送成功 哪些消息发送失败
channel.addConfirmListener(ackCallback, nackCallback);
long begin = System.currentTimeMillis();
for (int i = 0; i < MSG_NUMBER; i++) {
String msg = "消息" + i;
channel.basicPublish("", queueName, null,msg.getBytes("UTF-8"));
map.put(channel.getNextPublishSeqNo(), msg);
}
long end = System.currentTimeMillis();
System.out.println("===============================================");
System.out.println("异步确认发布"+MSG_NUMBER+"条消息,花费"+ (end - begin)+"ms");
}
3种发布确认速度对比
- 单独确认:同步等待确认,简单,单吞吐量有限
- 批量确认:简单合理的吞吐量,一旦发送出现问题很难判断出是哪一条消息出现问题
- 异步确认:最佳性能和资源使用,在出现错误时可以很好的控制
5. 交换机
发布的消息传达给多个消费者,称之为“发布/订阅”模式
生产者只能将消息发送到交换机,交换机必须知道如何处理接收到的消息,由交换机的类型来决定
- 直接(direct,路由类型)
- 主题(topic)
- 标题(header)
- 扇出(fanout即发布订阅模式)
- 无名交换机,在发布消息时交换机参数设置为“ ”
1. 绑定(bindings)
交换机和队列的桥梁,告诉交换机和哪个队列进行了绑定关系(使用RoutingKey进行消息的指定发送)
2. 发布订阅模式(fanout)
交换机将接收到的消息==广播==到它知道的所有队列中,MQ存在几种默认的交换机
/**
* 发布订阅模式-消息接收方
*/
public class ReceiveLogs1 {
public static final String EXCHANGE_NAME = "logs";
public static void main(String[] args) throws Exception {
Channel channel = RabbitMQUtils.getChannel();
// 声明交换机, 交换机类型为fanout
channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.FANOUT);
// 声明临时队列 当消费者断开与队列的连接时 队列就会自动删除
String queueName = channel.queueDeclare().getQueue();
// 交换机绑定队列: 队列名 交换机名 routingKey(可为空)
channel.queueBind(queueName, EXCHANGE_NAME, "");
System.out.println("ReceiveLogs1等待接受消息...");
channel.basicConsume(queueName, true, (tag, message) -> {
System.out.println("ReceiveLogs1接受到消息: " + message.getBody().toString());
}, (tag, message) -> {});
}
}
/**
* 发布订阅模式-消息接收方
*/
public class ReceiveLogs2 {
public static final String EXCHANGE_NAME = "logs";
public static void main(String[] args) throws Exception {
Channel channel = RabbitMQUtils.getChannel();
// 声明交换机, 交换机类型为fanout
channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.FANOUT);
// 声明临时队列 当消费者断开与队列的连接时 队列就会自动删除
String queueName = channel.queueDeclare().getQueue();
// 交换机绑定队列: 队列名 交换机名 routingKey(可为空)
channel.queueBind(queueName, EXCHANGE_NAME, "");
System.out.println("ReceiveLogs2等待接受消息...");
channel.basicConsume(queueName, true, (tag, message) -> {
System.out.println("ReceiveLogs2接受到消息: " + message.getBody());
}, (tag, message) -> {});
}
}
/**
* 发布订阅模式-消息发送方
*/
public class Logs {
public static final String EXCHANGE_NAME = "logs";
public static void main(String[] args) throws Exception {
Channel channel = RabbitMQUtils.getChannel();
// channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.FANOUT);
Scanner sc = new Scanner(System.in);
while(sc.hasNext()){
String message = sc.next();
channel.basicPublish(EXCHANGE_NAME, "", null, message.getBytes(StandardCharsets.UTF_8));
System.out.println("发送消息: " + message);
}
}
}
3. 路由模式(direct)
Fanout交换类型并不能带来很大的灵活性,只是无脑的进行广播,而使用direct交换类型,消息只会去它所绑定的routingKey队列
如果某个交换机类型是direct,但是绑定的每个队列的key都相同,等价于fanout交换类型
交换机direct_logs绑定两个队列,并且对其中一个队列进行多重绑定,在两个队列之间轮流发送消息
public class ReceiveDirect1 {
public static final String EXCHANGE_NAME = "direct_logs";
public static void main(String[] args) throws Exception {
Channel channel = RabbitMQUtils.getChannel();
// 声明交换机, 交换机类型为fanout
channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.DIRECT);
// 声明队列
String queueName = channel.queueDeclare("console", false, false, false, null).getQueue();
// 交换机多重绑定某个队列: 队列名 交换机名 routingKey不同
channel.queueBind(queueName, EXCHANGE_NAME, "prod");
channel.queueBind(queueName, EXCHANGE_NAME, "test");
System.out.println("ReceiveDirect1等待接受消息...");
DeliverCallback deliverCallback = (tag, message) -> {
System.out.println("ReceiveDirect1接收到消息: " + new String(message.getBody(), "UTF-8"));
};
channel.basicConsume(queueName, true, deliverCallback, tag -> {});
}
}
public class ReceiveDirect2 {
public static final String EXCHANGE_NAME = "direct_logs";
public static void main(String[] args) throws Exception {
Channel channel = RabbitMQUtils.getChannel();
// 声明交换机, 交换机类型为fanout
channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.DIRECT);
// 声明队列
String queueName = channel.queueDeclare("disk", false, false, false, null).getQueue();
// 交换机绑定某个队列: 队列名 交换机名 routingKey
channel.queueBind(queueName, EXCHANGE_NAME, "error");
System.out.println("ReceiveDirect2等待接受消息...");
DeliverCallback deliverCallback = (tag, message) -> {
System.out.println("ReceiveDirect2接收到消息: " + new String(message.getBody(), "UTF-8"));
};
channel.basicConsume(queueName, true, deliverCallback, tag -> {});
}
}
public class DirectLogs {
public static final String EXCHANGE_NAME = "direct_logs";
public static void main(String[] args) throws Exception {
Channel channel = RabbitMQUtils.getChannel();
Scanner sc = new Scanner(System.in);
int num = sc.nextInt();
String routingKey = null;
for (int i = 0; i < num; i++) {
if(i%3 == 0){
routingKey = "test";
}else if(i%3 == 1){
routingKey = "prod";
}else{
routingKey = "error";
}
String message = sc.next();
channel.basicPublish(EXCHANGE_NAME, routingKey, null, message.getBytes("UTF-8"));
System.out.println("发送消息: " + message);
}
}
}
4. 主题模式(Topic)
使用direct交换机虽然可以实现选择性的接受日志,但是对于日志类型有info.data和info.runtime,某个队列只想获取info.data的消息,那direct模式就无法做到。Topic模式不仅像fanout可以同时给某几个队列发送消息,也可以像direct模式一样只给某一个队列发送消息。
Topic交换机的routingKey不能随意写。必须是一个单词列表,并且以点号分隔开,如“spring.data.rabbitmq”。另外,*可以代替一个单词,#可代替0或者多个单词
Q1对应的routingKey为 (*.data.*)
,中间单词为data的3个单词的routingKey可路由到Q1
Q2对应的routingKey为 (*.*.rabbitmq)
,最后一个单词为rabbitmq的3个单词的routingKey可路由到Q2
Q3对应的routingKey为 (spring.#)
,第一个单词为spring的任意单词数的routingKey可路由到Q3
如果某一个队列的绑定键为#,那么这个队列将接受所有消息,类似于fanout。如果队列中的绑定键中没有*和#,那么就类似于direct
public class ReceiveTopic1 {
public static final String EXCHANGE_NAME = "topic_logs";
public static void main(String[] args) throws Exception {
Channel channel = RabbitMQUtils.getChannel();
channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.TOPIC);
String queueName = channel.queueDeclare("Q1", false, false, false, null).getQueue();
channel.queueBind(queueName, EXCHANGE_NAME, "*.data.*");
System.out.println("ReceiveTopic1等待接受消息...");
DeliverCallback deliverCallback = (tag, message) -> {
System.out.println("ReceiveTopic1接受到消息: " + new String(message.getBody(), "UTF-8"));
System.out.println("接受队列:" + queueName + "\t绑定键为:" + message.getEnvelope().getRoutingKey());
};
channel.basicConsume(queueName, true, deliverCallback, tag -> {});
}
}
public class ReceiveTopic2 {
public static final String EXCHANGE_NAME = "topic_logs";
public static void main(String[] args) throws Exception {
Channel channel = RabbitMQUtils.getChannel();
channel.exchangeDeclare(EXCHANGE_NAME, BuiltinExchangeType.TOPIC);
String queueName = channel.queueDeclare("Q2", false, false, false, null).getQueue();
channel.queueBind(queueName, EXCHANGE_NAME, "*.*.rabbitmq");
channel.queueBind(queueName, EXCHANGE_NAME, "spring.#");
System.out.println("ReceiveTopic2等待接受消息...");
DeliverCallback deliverCallback = (tag, message) -> {
System.out.println("ReceiveTopic2接受到消息: " + new String(message.getBody(), "UTF-8"));
System.out.println("接受队列:" + queueName + "\t绑定键为:" + message.getEnvelope().getRoutingKey());
};
channel.basicConsume(queueName, true, deliverCallback, tag -> {});
}
}
public class TopicLogs {
public static final String EXCHANGE_NAME = "topic_logs";
public static void main(String[] args) throws Exception {
Channel channel = RabbitMQUtils.getChannel();
Map<String, String> map = new HashMap<>();
map.put("aaa.data.rabbitmq", "Q1Q2接收");
map.put("spring.data.bbb", "Q1Q2接收");
map.put("aaa.data.bbb", "Q1接收");
map.put("spring.aaa.bbb", "Q2接收");
map.put("spring.aaa.rabbitmq", "Q2接收一次");
map.put("aaa.bbb.ccc", "丢弃");
map.put("aaa.bbb.ccc.rabbitmq", "丢弃");
map.put("spring.aaa.bbb.rabbitmq", "Q2接收");
Set<String> set = map.keySet();
for(String key : set){
String value = map.get(key);
channel.basicPublish(EXCHANGE_NAME, key, null, value.getBytes("UTF-8"));
System.out.println("发送消息: " + value);
}
}
}
查看RabbitMQ所绑定的路由信息
验证代码中所发送的8种消息,队列ReceiveTopic1和队列ReceiveTopic2是否接收正确