前言:
有些浮躁的时候,我就跑来写博客了。之所以不先写Spring 消息概念再写RabbitMq具体使用,这个问题我也纠结了好一会儿。关键工作后在企业级应用中都是先会用,然后才去理解。反向理解虽然会遇到很多坑,但也是成长最快。
正文:
一、RabbitMq的安装和启用:
1、安装 :
安装目录cd /usr/local/Cellar/
brew install rabbitmq
2、启动:
1、启动 sudo ./rabbitmq-server start
------------------------------------------------------------------------
Password:
## ##
## ## RabbitMQ 3.7.7. Copyright (C) 2007-2018 Pivotal Software, Inc.
########## Licensed under the MPL. See http://www.rabbitmq.com/
###### ##
########## Logs: /usr/local/var/log/rabbitmq/rabbit@localhost.log
/usr/local/var/log/rabbitmq/rabbit@localhost_upgrade.log
Starting broker...
completed with 6 plugins.
------------------------------------------------------------------------
尝试访问:http://localhost:15672/#/ 用户名密码均为guest,可判断是否启用成功。
3、常用命令如下:
启动监控管理器:rabbitmq-plugins enable rabbitmq_management
关闭监控管理器:rabbitmq-plugins disable rabbitmq_management
启动rabbitmq:rabbitmq-service start
关闭rabbitmq:rabbitmq-service stop
查看所有的队列:rabbitmqctl list_queues
清除所有的队列:rabbitmqctl reset
关闭应用:rabbitmqctl stop_app
启动应用:rabbitmqctl start_app
二、RabbitMq实现一个简单的生产者消费者:(下面大部分来自官方英文文档)
《一》一个生产者一个消费者
0、依赖:
<!-- rabbitmq -->
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>4.2.2</version>
</dependency>
1、生产者:
package com.haibo.future.web.rabbitmq;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
public class RabbitProduct {
private final static String QUEUE_NAME = "hello";
public static void main(String[] argv) throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
String message = "Hello World!";
channel.basicPublish("", QUEUE_NAME, null, message.getBytes("UTF-8"));
System.out.println(" [x] Sent '" + message + "'");
channel.close();
connection.close();
}
}
2、消费者:
package com.haibo.future.web.rabbitmq;
import com.rabbitmq.client.*;
import java.io.IOException;
import java.util.concurrent.TimeoutException;
public class RabbitConsumer {
private final static String QUEUE_NAME = "hello";
public static void main(String[] argv)
throws java.io.IOException,
java.lang.InterruptedException {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = null;
try {
connection = factory.newConnection();
} catch (TimeoutException e) {
e.printStackTrace();
}
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
System.out.println(" [*] Waiting for messages. To exit press CTRL+C");
Consumer consumer = new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
AMQP.BasicProperties properties, byte[] body)
throws IOException {
String message = new String(body, "UTF-8");
System.out.println(" [x] Received '" + message + "'");
}
};
channel.basicConsume(QUEUE_NAME, true, consumer);
}
}
执行测试:
1、先执行消费者:(消费者先注册监听,每次执行生产者都会在消费者收到一条消息)
[*] Waiting for messages. To exit press CTRL+C
[x] Received 'Hello World!'
[x] Received 'Hello World!'
[x] Received 'Hello World!'
2、再执行生产者:
[x] Sent 'Hello World!'
Process finished with exit code 0
《二》一个生产者多个消费者
1、生产者:(一次发送多个消息)
package com.haibo.future.web.rabbitmq.demo2;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
public class RabbitProduct {
private final static String QUEUE_NAME = "hello2";
public static void main(String[] argv) throws Exception {
for (int i = 0; i < 20; i++) {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
String message = "第"+i+"条"+"Hello World!";
channel.basicPublish("", QUEUE_NAME, null, message.getBytes("UTF-8"));
System.out.println(" [x] Sent '" + message + "'");
channel.close();
connection.close();
}
}
}
2、消费者:(新增消费延时一秒处理逻辑)
package com.haibo.future.web.rabbitmq.demo2;
import com.rabbitmq.client.*;
import java.io.IOException;
import java.util.concurrent.TimeoutException;
public class RabbitConsumer {
private final static String QUEUE_NAME = "hello2";
public static void main(String[] argv)
throws IOException,
InterruptedException {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = null;
try {
connection = factory.newConnection();
} catch (TimeoutException e) {
e.printStackTrace();
}
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
System.out.println(" [*] Waiting for messages. To exit press CTRL+C");
Consumer consumer = new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
AMQP.BasicProperties properties, byte[] body)
throws IOException {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
String message = new String(body, "UTF-8");
System.out.println(" [x] Received '" + message + "'");
}
};
channel.basicConsume(QUEUE_NAME, true, consumer);
}
}
执行测试:
1、启动2个(或多个)消费者:
结果见3。
2、启动生产者:
[x] Sent '第0条Hello World!'
[x] Sent '第1条Hello World!'
[x] Sent '第2条Hello World!'
[x] Sent '第3条Hello World!'
[x] Sent '第4条Hello World!'
[x] Sent '第5条Hello World!'
[x] Sent '第6条Hello World!'
[x] Sent '第7条Hello World!'
[x] Sent '第8条Hello World!'
[x] Sent '第9条Hello World!'
[x] Sent '第10条Hello World!'
[x] Sent '第11条Hello World!'
[x] Sent '第12条Hello World!'
[x] Sent '第13条Hello World!'
[x] Sent '第14条Hello World!'
[x] Sent '第15条Hello World!'
[x] Sent '第16条Hello World!'
[x] Sent '第17条Hello World!'
[x] Sent '第18条Hello World!'
[x] Sent '第19条Hello World!'
Process finished with exit code 0
3、消费者1:
[*] Waiting for messages. To exit press CTRL+C
[x] Received '第0条Hello World!'
[x] Received '第2条Hello World!'
[x] Received '第4条Hello World!'
[x] Received '第6条Hello World!'
[x] Received '第8条Hello World!'
[x] Received '第10条Hello World!'
[x] Received '第12条Hello World!'
[x] Received '第14条Hello World!'
[x] Received '第16条Hello World!'
[x] Received '第18条Hello World!'
4、消费者2:
[*] Waiting for messages. To exit press CTRL+C
[x] Received '第1条Hello World!'
[x] Received '第3条Hello World!'
[x] Received '第5条Hello World!'
[x] Received '第7条Hello World!'
[x] Received '第9条Hello World!'
[x] Received '第11条Hello World!'
[x] Received '第13条Hello World!'
[x] Received '第15条Hello World!'
[x] Received '第17条Hello World!'
[x] Received '第19条Hello World!'
结论:
一对多,无需其它任何配置,多个消费者,会轮训(round-robin)分发消息。
《三》、异常情况:多个消费者,消费时服务宕机,消息丢失处理。
异常情况一、消费者2消费的时候(比如才执行3秒)服务挂了,会造成消息丢失。也许有人会理解成,消费者2挂了,那么这些没处理的消息就会投递给消费者1呢?日志如下:
消费者2:
[*] Waiting for messages. To exit press CTRL+C
[x] Received '第1条Hello World!'
[x] Received '第3条Hello World!'
[x] Received '第5条Hello World!'
Process finished with exit code 130 (interrupted by signal 2: SIGINT)
消费者1:
[*] Waiting for messages. To exit press CTRL+C
[x] Received '第0条Hello World!'
[x] Received '第2条Hello World!'
[x] Received '第4条Hello World!'
[x] Received '第6条Hello World!'
[x] Received '第8条Hello World!'
[x] Received '第10条Hello World!'
[x] Received '第12条Hello World!'
[x] Received '第14条Hello World!'
[x] Received '第16条Hello World!'
[x] Received '第18条Hello World!'
异常处理:
实现已经投递到消费者2十条消息,消费者2中途挂了,消费者1(或者除消费者2外的其它更多消费者)继续处理其它消息。消息不丢失。
实现方式:消费者,消息成功处理后返回一个ack给RabbitMq。假如消费者2挂了,那么未返回ack的消息,将在RabbitMq中重新排队被其它消费者处理。
1、消费者:(生产者如上不变,消费修改,新增ack确认)
package com.haibo.future.web.rabbitmq.demo3;
import com.rabbitmq.client.*;
import java.io.IOException;
import java.util.concurrent.TimeoutException;
public class RabbitConsumer {
private final static String QUEUE_NAME = "hello3";
public static void main(String[] argv)
throws IOException,
InterruptedException {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = null;
try {
connection = factory.newConnection();
} catch (TimeoutException e) {
e.printStackTrace();
}
final Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
System.out.println(" [*] Waiting for messages. To exit press CTRL+C");
//本次添加:每次接受1个没ack的消息
channel.basicQos(1); // accept only one unack-ed message at a time (see below)
Consumer consumer = new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
AMQP.BasicProperties properties, byte[] body)
throws IOException {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
String message = new String(body, "UTF-8");
System.out.println(" [x] Received '" + message + "'");
//本次添加
channel.basicAck(envelope.getDeliveryTag(), false);
}
};
//本次添加
boolean autoAck = false;
channel.basicConsume(QUEUE_NAME, autoAck, consumer);
}
}
修改后,日志如下:
0、生产者:
[x] Sent '第0条Hello World!'
[x] Sent '第1条Hello World!'
[x] Sent '第2条Hello World!'
[x] Sent '第3条Hello World!'
[x] Sent '第4条Hello World!'
[x] Sent '第5条Hello World!'
[x] Sent '第6条Hello World!'
[x] Sent '第7条Hello World!'
[x] Sent '第8条Hello World!'
[x] Sent '第9条Hello World!'
[x] Sent '第10条Hello World!'
[x] Sent '第11条Hello World!'
[x] Sent '第12条Hello World!'
[x] Sent '第13条Hello World!'
[x] Sent '第14条Hello World!'
[x] Sent '第15条Hello World!'
[x] Sent '第16条Hello World!'
[x] Sent '第17条Hello World!'
[x] Sent '第18条Hello World!'
[x] Sent '第19条Hello World!'
Process finished with exit code 0
1、消费者1:(消费者2宕机后,处理顺序135,4,678....)
[*] Waiting for messages. To exit press CTRL+C
[x] Received '第1条Hello World!'
[x] Received '第3条Hello World!'
[x] Received '第5条Hello World!'
[x] Received '第4条Hello World!'
[x] Received '第6条Hello World!'
[x] Received '第7条Hello World!'
[x] Received '第8条Hello World!'
[x] Received '第9条Hello World!'
[x] Received '第10条Hello World!'
[x] Received '第11条Hello World!'
[x] Received '第12条Hello World!'
[x] Received '第13条Hello World!'
[x] Received '第14条Hello World!'
[x] Received '第15条Hello World!'
[x] Received '第16条Hello World!'
[x] Received '第17条Hello World!'
[x] Received '第18条Hello World!'
[x] Received '第19条Hello World!'
2、消费者2:(消费者2宕机,第4条记录重新排队见消费者1)
[*] Waiting for messages. To exit press CTRL+C
[x] Received '第0条Hello World!'
[x] Received '第2条Hello World!'
Process finished with exit code 130 (interrupted by signal 2: SIGINT)
《四》、异常情况:RabbitMq服务器宕机。丢失所有队列中未投递消息。
例如将基于上一版本代码,在生产者生产后,过两秒将RabbitMq服务器Ctrl+C,那么将会造成队列中消息丢失。重启服务无法找回。测试日志如下:
消费者1:
[*] Waiting for messages. To exit press CTRL+C
[x] Received '第1条Hello World!'
[x] Received '第3条Hello World!'
[x] Received '第5条Hello World!'
[x] Received '第7条Hello World!'
[x] Received '第9条Hello World!'
消费者2:
[*] Waiting for messages. To exit press CTRL+C
[x] Received '第0条Hello World!'
[x] Received '第2条Hello World!'
[x] Received '第4条Hello World!'
[x] Received '第6条Hello World!'
[x] Received '第8条Hello World!'
重启服务,剩下的10条队列和消息永远丢失消息。
解决异常,将队列、消息持久化到硬盘。生产者消费者代码如下:
1、生产者:
package com.haibo.future.web.rabbitmq.demo4;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.MessageProperties;
public class RabbitProduct {
private final static String QUEUE_NAME = "hello4";
public static void main(String[] argv) throws Exception {
for (int i = 0; i < 20; i++) {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
//1、队列持久化
boolean durable = true;
channel.queueDeclare(QUEUE_NAME, durable, false, false, null);
String message = "第"+i+"条"+"Hello World!";
//MessageProperties.PERSISTENT_TEXT_PLAIN 2、消息持久化
channel.basicPublish("", QUEUE_NAME, MessageProperties.PERSISTENT_TEXT_PLAIN, message.getBytes("UTF-8"));
System.out.println(" [x] Sent '" + message + "'");
channel.close();
connection.close();
}
}
}
2、消费者:
package com.haibo.future.web.rabbitmq.demo4;
import com.rabbitmq.client.*;
import java.io.IOException;
import java.util.concurrent.TimeoutException;
public class RabbitConsumer {
private final static String QUEUE_NAME = "hello4";
public static void main(String[] argv)
throws IOException,
InterruptedException {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = null;
try {
connection = factory.newConnection();
} catch (TimeoutException e) {
e.printStackTrace();
}
final Channel channel = connection.createChannel();
//队列持久化
boolean durable = true;
channel.queueDeclare(QUEUE_NAME, durable, false, false, null);
System.out.println(" [*] Waiting for messages. To exit press CTRL+C");
//本次添加:每次接受1个没ack的消息
channel.basicQos(1); // accept only one unack-ed message at a time (see below)
Consumer consumer = new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope,
AMQP.BasicProperties properties, byte[] body)
throws IOException {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
String message = new String(body, "UTF-8");
System.out.println(" [x] Received '" + message + "'");
//本次添加
channel.basicAck(envelope.getDeliveryTag(), false);
}
};
//本次添加
boolean autoAck = false;
channel.basicConsume(QUEUE_NAME, autoAck, consumer);
}
}
执行测试:测试日志如下:
消费者1:(可见原本是执行奇数,服务宕机后变成执行偶数条了)
[x] Received '第1条Hello World!'
[x] Received '第3条Hello World!'
[x] Received '第5条Hello World!'
[x] Received '第7条Hello World!'
[x] Received '第9条Hello World!'
[x] Received '第9条Hello World!'
[x] Received '第10条Hello World!'
[x] Received '第12条Hello World!'
[x] Received '第15条Hello World!'
[x] Received '第16条Hello World!'
[x] Received '第19条Hello World!'
消费者2:
[x] Received '第0条Hello World!'
[x] Received '第2条Hello World!'
[x] Received '第4条Hello World!'
[x] Received '第6条Hello World!'
[x] Received '第8条Hello World!'
[x] Received '第8条Hello World!'
[x] Received '第11条Hello World!'
[x] Received '第13条Hello World!'
[x] Received '第14条Hello World!'
[x] Received '第17条Hello World!'
[x] Received '第18条Hello World!'
《五》异常情况:公平分配:
假如上一版本代码赋值一份消费者,让消费者延时睡眠为5秒。(原消费者延时睡眠为1秒)那么假如不设置消费者每次只接受一个未ack的消息。这一行代码:
channel.basicQos(1);
RabbitMq会给消费者RR投递,比如消费者一1357,消费者二2468。不会去考虑消费者是否处理完。
添加之后,会让空闲消费者多处理,忙碌消费者(5秒延时消费者)少处理。日志如下:
消费者1(闲):
[*] Waiting for messages. To exit press CTRL+C
[x] Received '第0条Hello World!'
[x] Received '第2条Hello World!'
[x] Received '第3条Hello World!'
[x] Received '第4条Hello World!'
[x] Received '第5条Hello World!'
[x] Received '第7条Hello World!'
[x] Received '第8条Hello World!'
[x] Received '第9条Hello World!'
[x] Received '第10条Hello World!'
[x] Received '第11条Hello World!'
[x] Received '第13条Hello World!'
[x] Received '第14条Hello World!'
[x] Received '第15条Hello World!'
[x] Received '第16条Hello World!'
[x] Received '第17条Hello World!'
[x] Received '第19条Hello World!'
消费者2(忙):
[*] Waiting for messages. To exit press CTRL+C
[x] Received '第1条Hello World!'
[x] Received '第6条Hello World!'
[x] Received '第12条Hello World!'
[x] Received '第18条Hello World!'