RabbitMQ Tutorials
相关说明文档:www.rabbitmq.com/getstarted.html
rabbitmq-plugins enable rabbitmq_management
链接rabbitMQ 服务器的公共类
public class ConnectionUtils {
public static Connection getConnection() {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("127.0.0.1");
factory.setPort(5672);
factory.setVirtualHost("/vhost_mmr");
factory.setUsername("tealala");
factory.setPassword("111111");
try {
Connection connection = factory.newConnection();
return connection;
} catch (IOException e) {
e.printStackTrace();
} catch (TimeoutException e) {
e.printStackTrace();
}
return null;
}
}
一、work Queue 工作队列
1、轮询分发(round-robin) 结果就是不管谁忙或者不忙,消息都是平均分发的
public class Sender {
public static final String QUEUE_NAME = "test_work_queue";
public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
for (int i = 0; i < 50; i++) {
String msg = "hello "+i;
System.out.println("[WQ] send: "+msg);
channel.basicPublish("",QUEUE_NAME,null,msg.getBytes());
Thread.sleep(1*20);
}
channel.close();
connection.close();
}
}
运行后会在rabbitMQ服务器上看到生成的队列:
编写队列消息的消费者:
public class Recv1 {
public static void main(String[] args) throws IOException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(Sender.QUEUE_NAME,false,false,false,null);
Consumer consumer = new DefaultConsumer(channel){
//消息到达触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties,
byte[] body) throws IOException {
String msg = new String(body,"utf-8");
System.out.println("[1] Rece msg:"+msg);
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
}finally {
System.out.println("[1] done ");
}
}
};
boolean autoAck = true;
channel.basicConsume(Sender.QUEUE_NAME,autoAck,consumer);
}
}
运行之后可以看到的是队列中的消息是平均分发到两个消费者的,一个是奇数一个是偶数,并没有根据消息的处理时长来分配消息。
2、公平分发
必须关闭自动应答ack,改为手动;
每个消费者发送确认消息之前,消息队列不发送下一个消息到消费者,一次之处理限定的消息
使用basicQos(perfetch=1)
消息生产者:
public class Sender {
public static final String QUEUE_NAME = "test_work_queue";
public static final Integer prefetchCount = 1;
public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
boolean durable = true;//持久化操作
channel.queueDeclare(QUEUE_NAME, durable, false, false, null);
channel.basicQos(prefetchCount);
for (int i = 0; i < 50; i++) {
String msg = "hello "+i;
System.out.println("[WQ] send: "+msg);
channel.basicPublish("",QUEUE_NAME,null,msg.getBytes());
Thread.sleep(1*20);
}
channel.close();
connection.close();
}
}
消费者:
public class Recv1 {
public static void main(String[] args) throws IOException {
Connection connection = ConnectionUtils.getConnection();
final Channel channel = connection.createChannel();
channel.queueDeclare(Sender.QUEUE_NAME,false,false,false,null);
channel.basicQos(Sender.prefetchCount);
Consumer consumer = new DefaultConsumer(channel){
//消息到达触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties,
byte[] body) throws IOException {
String msg = new String(body,"utf-8");
System.out.println("[1] Rece msg:"+msg);
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}finally {
System.out.println("[1] done ");
channel.basicAck(envelope.getDeliveryTag(),false);
}
}
};
//自动应答给位false
boolean autoAck = false;
channel.basicConsume(Sender.QUEUE_NAME,autoAck,consumer);
}
}
public class Recv2 {
public static void main(String[] args) throws IOException {
Connection connection = ConnectionUtils.getConnection();
final Channel channel = connection.createChannel();
channel.queueDeclare(Sender.QUEUE_NAME,false,false,false,null);
channel.basicQos(Sender.prefetchCount);
Consumer consumer = new DefaultConsumer(channel){
//消息到达触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties,
byte[] body) throws IOException {
String msg = new String(body,"utf-8");
System.out.println("[2] Rece msg:"+msg);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}finally {
System.out.println("[2] done ");
channel.basicAck(envelope.getDeliveryTag(),false);
}
}
};
boolean autoAck = false;
channel.basicConsume(Sender.QUEUE_NAME,autoAck,consumer);
}
}
消费者1与消费者2的区别在于处理的时长,消费者1处理时长为2秒,而消费者2的处理时长为100毫秒。因此在队列分配消息的时候,消费者2分配到的消息数量要比消费者1多。
注意:公平分发的要点在于通道的属性basicQos的配置,以及要将消费者的自动应答改为手动应答ack的属性值true--->false。
消息应答与消息持久化
boolean autoAck = true
自动确认模式:一旦rabbitMQ将消息分发给消费者,就会从内存中删除,这种情况下,如果杀死正在执行的消费者,就会丢失正在处理的消息
boolean autoAck = false
手动模式,如果有一个消费者挂掉了,就会交付给其他的消费者,rabbitMQ支持消息应答,消费者发送一个消息应答,告诉rabbitmq 这个消息我已经处理完成了,你可以删除了,然后rabbitmq就删除内存中的消息
消息应答默认是打开的
Message acknowkedgment
如果rabbitmq挂了,消息也会丢失
消息的持久化
boolean durable = true;
声明好的队列就不能进行修改(如修改durable = true ,因为原来的队列是非持久化的),rabbitmq中不允许定义 (不同的参数)一个已经存在的队列
channel.queueDeclare(QUEUE_NAME, durable, false, false, null);
二、订阅模式(publish/subscribe)
模型
1、一个生产者多个消费者
2、每个消费者都有自己的队列
3、生产者没有直接吧消息发送到消息队列,而是发送到了交换机,转发器
4、每个队列都要绑定到交换机上
5、生产者发送发的消息经过交换机到达队列,就能实现一个消息被多个消费者消费
因为交换机没有存储能力,在rabbitmq 里面只有队列有存储能力,因为这个时候队列还没有绑定到交换机,所以数据会丢失
Exchange(交换机 / 转发器):一方面接收生产者的消息,一方面向队列推送消息,交换机的配置可以分为:匿名转发、fanout、direct三种类型
匿名转发 :
channel.exchangeDeclare(EXCHANGE_NAME, "");
Fanout(不处理路由键):每一个绑定到该交换机上的队列都会收到相同的消息。
channel.exchangeDeclare(EXCHANGE_NAME, "fanout");
Direct(处理路由键):根据配置的路由键,将消息分发到不同的队列上。【下文的路由模式中交换机的配置就是direct需要配置处理路由键】
channel.exchangeDeclare(EXCHANGE_NAME, "direct");
生产者:
注意通道与交换机绑定时候的配置:
channel.exchangeDeclare(EXCHANGE_NAME, "direct");
channel.basicPublish(EXCHANGE_NAME, "", null, msg.getBytes()); 不处理路由键,第二个参数为""
public class PsSender {
public static final String EXCHANGE_NAME = "test_exchange_fanout_1";
public static final String QUEUE_NAME = "test_exchange_queue";
public static final Integer prefetchCount = 1;
public static void main(String[] args) throws IOException, InterruptedException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
channel.exchangeDeclare(EXCHANGE_NAME, "fanout");
channel.basicQos(prefetchCount);
for (int i = 0; i < 50; i++) {
String msg = "hello ps " + i;
System.out.println("send: " + msg);
channel.basicPublish(EXCHANGE_NAME, "", null, msg.getBytes());
Thread.sleep(1 * 20);
}
channel.close();
connection.close();
}
}
消费者:
public class PsRecv1 {
public static final String QUEUE_NAME = "test_queue_fanout_wechat";
public static void main(String[] args) throws IOException {
Connection connection = ConnectionUtils.getConnection();
final Channel channel = connection.createChannel();
//队列声明
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
//绑定队列到交换机
channel.queueBind(QUEUE_NAME, PsSender.EXCHANGE_NAME, "");
channel.basicQos(PsSender.prefetchCount);
Consumer consumer = new DefaultConsumer(channel) {
//消息到达触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties,
byte[] body) throws IOException {
String msg = new String(body, "utf-8");
System.out.println("[1] Rece msg:" + msg);
try {
Thread.sleep(200);
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
System.out.println("[1] done ");
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
};
//自动应答给位false
boolean autoAck = false;
channel.basicConsume(QUEUE_NAME, autoAck, consumer);
}
}
public class PsRecv2 {
public static final String QUEUE_NAME = "test_queue_fanout_sms";
public static void main(String[] args) {
Connection connection = ConnectionUtils.getConnection();
final Channel channel;
try {
channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.queueBind(QUEUE_NAME, PsSender.EXCHANGE_NAME, "");
channel.basicQos(PsSender.prefetchCount);
Consumer consumer = new DefaultConsumer(channel) {
//消息到达触发这个方法
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties,
byte[] body) throws IOException {
String msg = new String(body, "utf-8");
System.out.println("[2] Rece msg:" + msg);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
System.out.println("[2] done ");
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
};
boolean autoAck = false;
channel.basicConsume(QUEUE_NAME, autoAck, consumer);
} catch (IOException e) {
e.printStackTrace();
}
}
}
四、路由模式
1、type=direct 注意名称不要弄错了
2、routingKey = "info"
sender中
channel.exchangeDeclare(EXCHANGE_NAME, "direct");
channel.basicPublish(EXCHANGE_NAME,routingKey,null,msg.getBytes());
recv中
channel.queueBind(QUEUE_NAME, RouteSender.EXCHANGE_NAME, routingKey );
路由类型必须明确,需要定义routingKey
生产者:
public class RouteSender {
public static final String EXCHANGE_NAME = "test_exchange_routing_direct";
public static void main(String[] args) throws IOException, TimeoutException, InterruptedException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
channel.exchangeDeclare(EXCHANGE_NAME, "direct");
for (int i = 0; i < 50; i++) {
String msg = "hello direct: " + i;
System.out.println(msg);
String routingKey = "info";
channel.basicPublish(EXCHANGE_NAME, routingKey, null, msg.getBytes());
Thread.sleep(1 * 20);
}
channel.close();
connection.close();
}
}
消费者:
public class RouteRecv1 {
private static final String QUEUE_NAME = "test_queue_direct1";
public static void main(String[] args) throws IOException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.queueBind(QUEUE_NAME, RouteSender.EXCHANGE_NAME, "error");
channel.basicQos(1);
Consumer consumer = new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties,
byte[] body) throws IOException {
String msg = new String(body, "utf-8");
System.out.println("[1] recv msg:" + msg);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
System.out.println("[1] done");
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
};
boolean autoAck = false;
channel.basicConsume(QUEUE_NAME, autoAck, consumer);
}
}
public class RouteRecv2 {
private static final String QUEUE_NAME = "test_queue_direct2";
public static void main(String[] args) throws IOException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.queueBind(QUEUE_NAME, RouteSender.EXCHANGE_NAME, "error");
channel.queueBind(QUEUE_NAME, RouteSender.EXCHANGE_NAME, "info");
channel.queueBind(QUEUE_NAME, RouteSender.EXCHANGE_NAME, "warning");
channel.basicQos(1);
Consumer consumer = new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties,
byte[] body) throws IOException {
String msg = new String(body, "utf-8");
System.out.println("[2] recv msg:" + msg);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
System.out.println("[2] done");
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
};
boolean autoAck = false;
channel.basicConsume(QUEUE_NAME, autoAck, consumer);
}
}
因此消费者1只能接收到路由键为error的消息,消费者2可以接收到error、info、warning三种路由的消息。
五、Topic Exchange主题模式
将路由和某个模式进行匹配
# 匹配一个或多个
* 匹配一个
生产者:
public class TopicSender {
public static final String EXCHANGE_NAME = "test_exchange_topic";
public static final String TYPE = "topic";
public static void main(String[] args) throws IOException, TimeoutException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
channel.exchangeDeclare(EXCHANGE_NAME, "topic");
String operateType = "user.insert";
String msg = "send topic msg"+operateType;
System.out.println(msg);
channel.basicPublish(EXCHANGE_NAME, operateType, null, msg.getBytes());
channel.close();
connection.close();
}
}
消费者:
public class TopicRecv1 {
private static final String QUEUE_NAME = "test_queue_topic_user1";
public static void main(String[] args) throws IOException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.queueBind(QUEUE_NAME, TopicSender.EXCHANGE_NAME, "user.#");
// channel.queueBind(QUEUE_NAME, TopicSender.EXCHANGE_NAME, "user.info");
Consumer consumer = new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties,
byte[] body) throws IOException {
String msg = new String(body, "utf-8");
System.out.println("[1] topicrecv msg:" + msg);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
System.out.println("[1] done");
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
};
boolean autoAck = false;
channel.basicConsume(QUEUE_NAME, autoAck, consumer);
}
}
public class TopicRecv2 {
private static final String QUEUE_NAME = "test_queue_topic_user2";
public static void main(String[] args) throws IOException {
Connection connection = ConnectionUtils.getConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
channel.queueBind(QUEUE_NAME, TopicSender.EXCHANGE_NAME, "user.delete");
channel.queueBind(QUEUE_NAME, TopicSender.EXCHANGE_NAME, "user.update");
Consumer consumer = new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties,
byte[] body) throws IOException {
String msg = new String(body, "utf-8");
System.out.println("[1] topicrecv msg:" + msg);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
System.out.println("[1] done");
channel.basicAck(envelope.getDeliveryTag(), false);
}
}
};
boolean autoAck = false;
channel.basicConsume(QUEUE_NAME, autoAck, consumer);
}
}
Rabbitmq的消息确认机制(事务+confirm)
在rabbitmq 中可以通过持久化数据解决rabbitmq服务器异常的数据丢失问题
问题:生产者将消息发送出去之后,消息到底有没有到达rabbitmq服务器,默认的情况是不知道的
两种方式:AMQP实现了事务机制,confirm模式
事务机制
txSelect txCommit txRollback
txSelect:用户将当前channel设置成transation模式
异常信息必须放于select 与 commit之间才能被回滚,否则仍然会被压入队列
txCommit:用于提交事务
txRollback:回滚事务
缺点:降低吞吐量
六、Confirm模式
生产者端confirm模式的实现原理
生产者将信道设置成confirm模式,一旦信道进入confirm模式,所有在该信道上发布的消息都会被指派一个唯一的ID(从1开始),一旦消息被投递到了队列之后,broker就会发送一个确认给成产者(包含消息的唯一ID),这就使得生产者指定消息已经争取到达了目的队列,如果消息和队列是可持久化的,那么确认消息会将消息写入到磁盘之后发出,broker回传给生产者的确认消息中deliver-tag域包含了消息的序列号,此外broker也可以设置basic.ack 的multiple域,表示到这个序列号之前的所有消息都已经得到了处理。
Confirm模式的最大的好处在于他是异步
Nack
开启confirm模式
channel.confirmSelect();
普通的 waitForConfirms();
批量 waitForConfirms
异步confirm模式,提供一个回调方法
异步模式:
Channel 对象提供的ConfirmListener() 回调方法只包含deliveryTag (当前Chanel发出的消息序列号),我们需要自己为每一个Channel维护一个unconfirm的消息序号集合,每publish一条数据,集合中的元素就加1,每回调一次就handleAck方法,unconfirm集合删掉相应的一条(multiple = false)或多条(multiple = true)记录,从程序运行的效率上看,这个unconfirm集合最好采用有序集合SortedSet存储结构。