一、消息如何保证100%的投递成功
- 什么是生产端的可靠性投递？
保证消息的成功发出。
保证MQj节点的成功接受。
发送端收到MQ节点（Broker）确认应答。
完善的消息进行补偿机制。
- 生产端---可靠性投递。
消息落库，对消息状态进行打标。
消息的延迟投递，做二次确认、回调检查。
二、幂等性
- 幂等性是什么？
我们可以借鉴数据库的乐观锁机制：
比如我们执行一条更新库存的sql语句
UPDATE T_REPS SET COUNT=COUNT-1,VERSION=VERSION+1 WHERE VERSION=1
- 消费端---幂等性的保障？
在海量订单产生的业务高峰期，如何避免消息的重复消费的问题？
- 消费端实现幂等性，就意味着，我们永远不会消费多次，即使我们收到了多条一样的消息。
- 解决方案
- 唯一ID+指纹吗机制，利用数据据库主键去重。
唯一ID+指纹码机制，利用数据库主键去重
SELECT COUNT(1) FROM T_ORDER WHERE ID=唯一ID+指纹码
好处：实现简单
坏处：高并发下有数据库写入性的性能瓶颈
解决方案：跟进ID进行分库分表进行算法路由。
2. 利用redis的原子性去实现。
使用redis进行幂等性，需要考虑的问题。
三、 Cinfirm 确认消息
理解Confirm 消息确认机制
- 消息确认，是指生产者投递消息后，如果Broker收到消息 ，则会给我们生产者一个应答
- 生产者进行接受应答，用来确定这条消息是否正常的发送 到Broker,这种方式也是消息可可靠性投递的核心保障。
如何实现Confirm确认消息：
- 第一步：在channel 上开启确认模式：channel.confirmSelect()
- 在channel上添加监听： addConfirmListenner,监听成功和失败的返回结果，根据具体的结果对消息进行重新发送、或者记录日志等后续处理。
- 实现代码
-
package wang.chunsen.api.confirm; import com.rabbitmq.client.Channel; import com.rabbitmq.client.ConfirmListener; import com.rabbitmq.client.Connection; import com.rabbitmq.client.ConnectionFactory; import java.io.IOException; /** * @ClassName: Producer * @Author: wcs * @Description: * @Date: Created in 20:31 2018/12/22 * @Package: wang.chunsen.api.confirm * @project: rabbitmq-api * @Modified By: */ public class Producer { public static void main(String[] args) throws Exception { //创建一个来连接工厂 ConnectionFactory connectionFactory = new ConnectionFactory(); connectionFactory.setHost("192.168.28.128"); connectionFactory.setPort(5672); connectionFactory.setVirtualHost("/"); //获取connection Connection connection = connectionFactory.newConnection(); //同过connection 创建一个Channel Channel channel = connection.createChannel(); //指定我们的消息投递模式 channel.confirmSelect(); String exChangeName = "test_confirm_exchange"; String routingkey = "confirm.save"; String message = "hello RabbitMQ Send confirm message!"; channel.basicPublish(exChangeName, routingkey, null, message.getBytes("UTF-8")); //添加一个确认监听 channel.addConfirmListener(new ConfirmListener() { @Override public void handleAck(long deliveryTag, boolean multiple) throws IOException { System.err.println("-----------ack!--------------"); } @Override public void handleNack(long deliveryTag, boolean multiple) throws IOException { System.out.println("------------no ack----------------"); } }); } }package wang.chunsen.api.confirm; import com.rabbitmq.client.*; import java.io.IOException; /** * @ClassName: Consumer * @Author: wcs * @Description: * @Date: Created in 20:31 2018/12/22 * @Package: wang.chunsen.api.confirm * @project: rabbitmq-api * @Modified By: */ public class Consumer { public static void main(String[] args) throws Exception { //创建一个来连接工厂 ConnectionFactory connectionFactory = new ConnectionFactory(); connectionFactory.setHost("192.168.28.128"); connectionFactory.setPort(5672); connectionFactory.setVirtualHost("/"); //获取connection Connection connection = connectionFactory.newConnection(); //同过connection 创建一个Channel Channel channel = connection.createChannel(); String exChangeName = "test_confirm_exchange"; String routingkey = "confirm.#"; String queueName = "test_confirm_quque"; //声明一个exchannge channel.exchangeDeclare(exChangeName, "topic", true); //创建一个队列 channel.queueDeclare(queueName, true, false, false, null); //进行绑定 channel.queueBind(queueName, exChangeName, routingkey); DefaultConsumer defaultConsumer = new DefaultConsumer(channel); while (true) { channel.basicConsume(queueName, true, new DefaultConsumer(channel) { @Override public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException { String message = new String(body, "UTF-8"); System.out.println(message); } }); } } }
三、 Return 消息机制
- return listenner 用于处理一些不可路由的消息
- 我们消息的生产者，通过指定一个Exchange 和RoutingKey ，把消息发送到某一个队列中去，然后我们的消费者监听对队列，进行消息的消费处理。
- 但是在某些情况下，如果我们在发送消息的时候，当前exchange不存在或者指定的路由key路由不到，这个时候如果我们需要监听这种不可达的消息，就要使用Return Listener!
在基础API中有一个关键的配置项：
Mandatory: 如果为ture,则监听器会接受不可达到的消息，然后进行处理，如果为false,那么broker端自动删除该消息！
package wang.chunsen.api.returnlistener;
import com.rabbitmq.client.*;
import java.io.IOException;
/**
* @ClassName: Consumer
* @Author: wcs
* @Description:
* @Date: Created in 21:33 2018/12/22
* @Package: wang.chunsen.api.returnlistener
* @project: rabbitmq-api
* @Modified By:
*/
public class Consumer {
public static void main(String[] args) throws Exception {
//创建一个来连接工厂
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("192.168.28.128");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("/");
//获取connection
Connection connection = connectionFactory.newConnection();
//同过connection 创建一个Channel
Channel channel = connection.createChannel();
String exChangeName = "test_return_exchange";
String routingkey = "return.#";
String queueName = "test_returnl_quque";
//声明一个exchannge
channel.exchangeDeclare(exChangeName, "topic", true);
//创建一个队列
channel.queueDeclare(queueName, true, false, false, null);
//进行绑定
channel.queueBind(queueName, exChangeName, routingkey);
while (true) {
channel.basicConsume(queueName, true, new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag,
Envelope envelope,
AMQP.BasicProperties properties,
byte[] body) throws IOException {
String message = new String(body, "UTF-8");
System.out.println(message);
}
});
}
}
}
package wang.chunsen.api.returnlistener;
import com.rabbitmq.client.*;
import java.io.IOException;
/**
* @ClassName: Producer
* @Author: wcs
* @Description:
* @Date: Created in 21:32 2018/12/22
* @Package: wang.chunsen.api.returnlistener
* @project: rabbitmq-api
* @Modified By:
*/
public class Producer {
public static void main(String[] args) throws Exception {
//创建一个来连接工厂
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("192.168.28.128");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("/");
//获取connection
Connection connection = connectionFactory.newConnection();
//同过connection 创建一个Channel
Channel channel = connection.createChannel();
String exChangeName = "test_return_exchange";
String routingkey = "return.save";
String routingkeyError = "abc.save";
String message = "hello RabbitMQ Send return message!";
channel.addReturnListener(new ReturnListener() {
@Override
public void handleReturn(int replyCode, String replyText, String exchange, String routingKey, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.out.println("-------handle return-----");
System.out.println("replyCode" + replyCode);
System.out.println("exchange" + exchange);
System.out.println("routingKey" + routingKey);
System.out.println("properties" + properties);
System.out.println("body" + new String(body));
}
});
channel.basicPublish(exChangeName, routingkey, true, null, message.getBytes("UTF-8"));
}
}
四、自定义消费者
package wang.chunsen.api.consumer;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
/**
* @ClassName: Producer
* @Author: wcs
* @Description:
* @Date: Created in 21:32 2018/12/22
* @Package: wang.chunsen.api.returnlistener
* @project: rabbitmq-api
* @Modified By:
*/
public class Producer {
public static void main(String[] args) throws Exception {
//创建一个来连接工厂
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("192.168.28.128");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("/");
//获取connection
Connection connection = connectionFactory.newConnection();
//同过connection 创建一个Channel
Channel channel = connection.createChannel();
String exChangeName = "test_consumer_exchange";
String routingkey = "consumer.save";
String message = "hello RabbitMQ Send return message!";
for (int i = 0; i < 10; i++) {
channel.basicPublish(exChangeName, routingkey, true, null, message.getBytes("UTF-8"));
}
}
}
package wang.chunsen.api.consumer;
import com.rabbitmq.client.*;
import java.io.IOException;
/**
* @ClassName: Consumer
* @Author: wcs
* @Description:
* @Date: Created in 21:33 2018/12/22
* @Package: wang.chunsen.api.returnlistener
* @project: rabbitmq-api
* @Modified By:
*/
public class Consumer {
public static void main(String[] args) throws Exception {
//创建一个来连接工厂
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("192.168.28.128");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("/");
//获取connection
Connection connection = connectionFactory.newConnection();
//同过connection 创建一个Channel
Channel channel = connection.createChannel();
String exChangeName = "test_consumer_exchange";
String routingkey = "consumer .#";
String queueName = "test_consumer_quque";
//声明一个exchannge
channel.exchangeDeclare(exChangeName, "topic", true);
//创建一个队列
channel.queueDeclare(queueName, true, false, false, null);
//进行绑定
channel.queueBind(queueName, exChangeName, routingkey);
while (true) {
channel.basicConsume(queueName, true, new MyConsumer(channel) {
@Override
public void handleDelivery(String consumerTag,
Envelope envelope,
AMQP.BasicProperties properties,
byte[] body) throws IOException {
String message = new String(body, "UTF-8");
System.out.println(message);
}
});
}
}
}
package wang.chunsen.api.consumer;
import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DefaultConsumer;
import com.rabbitmq.client.Envelope;
import java.io.IOException;
/**
* @ClassName: MyConsumer
* @Author: wcs
* @Description:
* @Date: Created in 10:53 2018/12/23
* @Package: wang.chunsen.api.consumer
* @project: rabbitmq-api
* @Modified By:
*/
public class MyConsumer extends DefaultConsumer {
public MyConsumer(Channel channel) {
super(channel);
}
/**
* No-op implementation of {@link Consumer#handleDelivery}.
*
* @param consumerTag
* @param envelope
* @param properties
* @param body
*/
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
super.handleDelivery(consumerTag, envelope, properties, body);
}
}
五、消费端限流
什么是消费端限流？
巨量的消息瞬间去拿不推送过来，但是我们单个客户端无法同时处理这么多的消息。
- RabbitMQ提供了一种qos(服务质量保证)功能，即在非自动确认消息的前提下，如果一定数目的消息（通过基于consume或者channel设置QOS的值）未被确认之前，不进行消费新的消息。
- void BasicQos(uint prefecchsize,ushort prefetchCount, bool glbal);
prefetchSize: 0
prefetchCount:会告诉RabbitMQ不要同时给一个消费者推送多于N个消息，即一旦有N个消息还没有被（ack）,则该consume将block掉，直到有消息ack
global:true\false 是将上面设置应用于channel 简单点来说，就是上面限制channel级别的还是consumer级别的。
prefecthSize 和global 这两项，rabbitmq没有实现，暂且不研究prefetch_count在no_ask=false的情况下生效，及在自动应答的情况下这两个值是不生效的。
package wang.chunsen.api.limit;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
/**
* @ClassName: Producer
* @Author: wcs
* @Description:
* @Date: Created in 21:32 2018/12/22
* @Package: wang.chunsen.api.returnlistener
* @project: rabbitmq-api
* @Modified By:
*/
public class Producer {
public static void main(String[] args) throws Exception {
//创建一个来连接工厂
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("192.168.28.128");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("/");
//获取connection
Connection connection = connectionFactory.newConnection();
//同过connection 创建一个Channel
Channel channel = connection.createChannel();
String exChangeName = "test_qos_exchange";
String routingkey = "qos.save";
String message = "hello RabbitMQ Send return message!";
for (int i = 0; i < 10; i++) {
channel.basicPublish(exChangeName, routingkey, true, null, message.getBytes("UTF-8"));
}
}
}
package wang.chunsen.api.limit;
import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DefaultConsumer;
import com.rabbitmq.client.Envelope;
import java.io.IOException;
/**
* @ClassName: MyConsumer
* @Author: wcs
* @Description:
* @Date: Created in 10:53 2018/12/23
* @Package: wang.chunsen.api.consumer
* @project: rabbitmq-api
* @Modified By:
*/
public class MyConsumer extends DefaultConsumer {
private Channel channel;
public MyConsumer(Channel channel) {
super(channel);
this.channel = channel;
}
/**
* No-op implementation of {@link Consumer#handleDelivery}.
*
* @param consumerTag
* @param envelope
* @param properties
* @param body
*/
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
super.handleDelivery(consumerTag, envelope, properties, body);
channel.basicQos((int) envelope.getDeliveryTag(), false);
}
}
package wang.chunsen.api.limit;
import com.rabbitmq.client.*;
import java.io.IOException;
/**
* @ClassName: Consumer
* @Author: wcs
* @Description:
* @Date: Created in 21:33 2018/12/22
* @Package: wang.chunsen.api.returnlistener
* @project: rabbitmq-api
* @Modified By:
*/
public class Consumer {
public static void main(String[] args) throws Exception {
//创建一个来连接工厂
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("192.168.28.128");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("/");
//获取connection
Connection connection = connectionFactory.newConnection();
//同过connection 创建一个Channel
Channel channel = connection.createChannel();
String exChangeName = "test_qos_exchange";
String routingkey = "qos.#";
String queueName = "test_qos_quque";
//声明一个exchannge
channel.exchangeDeclare(exChangeName, "topic", true);
//创建一个队列
channel.queueDeclare(queueName, true, false, false, null);
//进行绑定
channel.queueBind(queueName, exChangeName, routingkey);
//auto ACK 设置为false
channel.basicQos(0, 1, false);
while (true) {
channel.basicConsume(queueName, false, new MyConsumer(channel) {
@Override
public void handleDelivery(String consumerTag,
Envelope envelope,
AMQP.BasicProperties properties,
byte[] body) throws IOException {
String message = new String(body, "UTF-8");
System.out.println(message);
}
});
}
}
}
六、消息的ACK与重回队列
消费端的手工ACK和NACK(自动签收，手工签收)。
- 消费端进行消费的时候，如果由于业务异常我们可以进行日志记录，然后进行补偿。
- 如果由于服务器宕机等严重问题，那我们就需要手工Ack保证消费端消费成功。
消费端的重回队列
- 消费端重回队列是为了对没有处理成功的消息，把消息重新投递给Broker
- 一般我们在实际应用中，都会关闭 重回队列，也就是设置为false
CODING
七、TTL消息
- TTL是Time To Live 的缩写，业务就是生存时间
- RabbitMq 支持消息的过期时间，在消息发送时可以进行指定
- RabbitMQ 支持队列的过期时间，从消息发送入队列开始计算，只要超过队列超时时间配置，那么消息会自动清除。
八、死信队列
- DLX也是一个正常的Exchange，和一般的Exchange没有区别，他能在任何的队列上被指定，实际上就是设置某个死信队列的属性。
- 当这个队列中有死信时，RabbitMQ就会自动的将这个消息重新发布到设置的Exchange上去，进而被路由到另一个队列中。
- 可以监听这个队列中消息做相应的处理，这个特性可以弥补RabbitMq3.0以前支持的immediate 参数功能。
死信队列：DLX , Dead-Letter-Exchange
- 利用DLX,当消息子一个队列中变成死信之后，他能被重新publish到另一个Exchange,这个Exchange就是DL
消息变成死信队列有以下集中情况
消息被拒绝（basic.reject/basic.nack）并且requeue=false
消息TTL过期
队列达到最大长度
死信队列的设置：
首先需要设置死信队列的exchange 和queue，然后进行绑定：
- Exchange :dlx.exchange
- Queue: dlx.queue
- RoutingKey:#
然后我们进行正常声明交换机、队列、绑定、只不过我们需要在队列上加一个参数即可：arguments.put("x-dead-letter-exchange","dlx.exchange).
这样消息在过期，requeue，队列在达到最大的长度时，消息可以之浩劫路由到死信队列。
package com.bfxy.rabbitmq.api.ack;
import java.util.HashMap;
import java.util.Map;
import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
public class Producer {
public static void main(String[] args) throws Exception {
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("192.168.11.76");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("/");
Connection connection = connectionFactory.newConnection();
Channel channel = connection.createChannel();
String exchange = "test_ack_exchange";
String routingKey = "ack.save";
for(int i =0; i<5; i ++){
Map<String, Object> headers = new HashMap<String, Object>();
headers.put("num", i);
AMQP.BasicProperties properties = new AMQP.BasicProperties.Builder()
.deliveryMode(2)
.contentEncoding("UTF-8")
.headers(headers)
.build();
String msg = "Hello RabbitMQ ACK Message " + i;
channel.basicPublish(exchange, routingKey, true, properties, msg.getBytes());
}
}
}
package com.bfxy.rabbitmq.api.ack;
import java.io.IOException;
import com.rabbitmq.client.AMQP;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.DefaultConsumer;
import com.rabbitmq.client.Envelope;
public class MyConsumer extends DefaultConsumer {
private Channel channel ;
public MyConsumer(Channel channel) {
super(channel);
this.channel = channel;
}
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
System.err.println("-----------consume message----------");
System.err.println("body: " + new String(body));
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
if((Integer)properties.getHeaders().get("num") == 0) {
channel.basicNack(envelope.getDeliveryTag(), false, true);
} else {
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
}
package com.bfxy.rabbitmq.api.ack;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.QueueingConsumer;
import com.rabbitmq.client.QueueingConsumer.Delivery;
public class Consumer {
public static void main(String[] args) throws Exception {
ConnectionFactory connectionFactory = new ConnectionFactory();
connectionFactory.setHost("192.168.11.76");
connectionFactory.setPort(5672);
connectionFactory.setVirtualHost("/");
Connection connection = connectionFactory.newConnection();
Channel channel = connection.createChannel();
String exchangeName = "test_ack_exchange";
String queueName = "test_ack_queue";
String routingKey = "ack.#";
channel.exchangeDeclare(exchangeName, "topic", true, false, null);
channel.queueDeclare(queueName, true, false, false, null);
channel.queueBind(queueName, exchangeName, routingKey);
// 手工签收 必须要关闭 autoAck = false
channel.basicConsume(queueName, false, new MyConsumer(channel));
}
}
644
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
