目录
1.RocketMQ角色及概念
1.1、消息生产者:producer,负责生产消息,一般由业务系统负责生产消息。
1.2、消息消费者:consumer,负责消费消息,一般是后台系统负责异步消费。
1.3、消息服务器:broker,消息中转角色,负责存储消息、转发消息。消息服务器在RocketMQ系统中负责接收从生产者发送来的消息并存储、同时为消费者的拉取请求作准备。
1.4、名称服务器:
broker
将自己注册进NameServer;
producer
、consumer
通过其获取broker
信息然后发送、接收消息;
名称服务器NameServer
通过心跳检测确认producer
、consumer
、broker
上下线(三者向NameServer,30s/次发送心跳)。
1.5、主题:topic,表示一类消息的集合,每个主题包含若干条消息,每条消息只能属于一个主题。
1.6、标签:tag,为消息设置的标志,用于同一主题下区分不同类型的消息。
1.7、消息组成:消息体(body)、主题(Topic)、标签(tag子主题)。
2.RocketMQ技术架构图
3.快速入门
3.1新建maven管理的java项目,引入依赖
<dependency>
<groupId>org.apache.rocketmq</groupId>
<artifactId>rocketmq-client</artifactId>
<version>4.9.3</version>
</dependency>
4.发送消息的方式
4.1同步发送
及时性较强,可靠,有回执的消息;被广泛应用于各种场景,如重要的通知消息、短消息通知等
4.2异步发送
及时性较弱,可靠,有回执的消息;通常用于响应时间敏感的业务场景
4.3单向发送
不需要回执的消息;用于需要中等可靠性的情况,如日志收集
演示代码:
.生产者Producer
public class Producer {
public static void main(String[] args) throws Exception {
DefaultMQProducer producer = new DefaultMQProducer("group1");
producer.setNamesrvAddr("47.112.153.198:9876");
producer.start();
for (int i = 1; i <= 5; i++) {
//同步消息发送
Message msg = new Message("topic1", ("同步消息:hello rocketmq " + i).getBytes("UTF-8"));
SendResult result = producer.send(msg);
System.out.println("返回结果:" + result);
//异步消息发送
msg = new Message("topic1", ("异步消息:hello rocketmq " + i).getBytes("UTF-8"));
producer.send(msg, new SendCallback() {
//表示成功返回结果
public void onSuccess(SendResult sendResult) {
System.out.println(sendResult);
}
//表示发送消息失败
public void onException(Throwable t) {
System.out.println(t);
}
});
//单向消息
msg = new Message("topic1", ("单向消息:hello rocketmq " + i).getBytes("UTF-8"));
producer.sendOneway(msg);
}
//添加一个休眠操作,确保异步消息返回后能够输出
// 工作中生产环境生产者程序会一直运行,就不需要休眠了
TimeUnit.SECONDS.sleep(10);
producer.shutdown();
}
}
.消费者Consumer
public class Consumer {
public static void main(String[] args) throws MQClientException {
// 实例化消费者
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1");
// 设置NameServer的地址
consumer.setNamesrvAddr("47.112.153.198:9876");
// 订阅一个或者多个Topic,以及Tag来过滤需要消费的消息
consumer.subscribe("topic1", "*");
// 注册回调实现类来处理从broker拉取回来的消息
consumer.registerMessageListener(new MessageListenerConcurrently() {
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
System.out.printf("%s Receive New Messages: %s %n", Thread.currentThread().getName(), msgs);
// 标记该消息已经被成功消费
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
});
// 启动消费者实例
consumer.start();
System.out.printf("Consumer Started.%n");
}
}
5.延时消息
RocketMQ
不支持任意时间的延时,只支持固定时间的延时
可通过 msg.setDelayTimeLevel(index)
来设置延时级别,index为0~17,共18级,
对应的延时时间为:1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h
应用场景:
下单订单之后,就可以发送一个延时消息;一个小时后执行该延时消息,检查订单是否支付,如未支付,就取消订单,释放库存。
演示代码:
.生产者Producer
public class Producer {
public static void main(String[] args) throws Exception {
DefaultMQProducer producer = new DefaultMQProducer("group1");
producer.setNamesrvAddr("192.168.115.130:9876");
producer.start();
for (int i = 1; i <= 5; i++) {
Message msg = new Message("topic1",("延时消息:hello rocketmq "+i).getBytes("UTF-8"));
// 1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h
//设置当前消息的延时级别,level=2,表示延时10s
msg.setDelayTimeLevel(2);
SendResult result = producer.send(msg);
System.out.println("返回结果:"+result);
}
producer.shutdown();
}
}
.消费者Consumer
public class Consumer {
public static void main(String[] args) throws Exception {
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1");
consumer.setNamesrvAddr("47.112.153.198:9876");
consumer.subscribe("topic1","*");
consumer.registerMessageListener(new MessageListenerConcurrently() {
@Override
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
for(MessageExt msg : list){
System.out.println("收到延时消息:"+new String(msg.getBody()));
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
});
consumer.start();
System.out.println("Consumer start.....");
}
}
6.批量消息发送
如果有多个消息,可以一次性发送。创建多个消息,添加到list
对象中,一起发送。
批量发送消息时,每次发送的消息总量不能超过4M。批量发送消息能显著提高传递小消息的性能
代码演示:
.生产者Producer
public class Producer {
public static void main(String[] args) throws Exception {
DefaultMQProducer producer = new DefaultMQProducer("group1");
producer.setNamesrvAddr("47.112.153.198:9876");
producer.start();
//创建要发送的消息,指定主题topic和内容body
Message message1 = new Message("topic1", "hello world 1".getBytes("utf-8")),
message2 = new Message("topic1", "hello world 2".getBytes("utf-8")),
message3 = new Message("topic1", "hello world 3".getBytes("utf-8")),
message4 = new Message("topic1", "hello world 4".getBytes("utf-8")),
message5 = new Message("topic1", "hello world 5".getBytes("utf-8"));
List<Message> list = new ArrayList<Message>();
list.add(message1);
list.add(message2);
list.add(message3);
list.add(message4);
list.add(message5);
//发送批量消息(每次发送的消息总量不得超过4M)
SendResult result = producer.send(list);
System.out.println("发送消息,返回结果:" + result);
producer.shutdown();
}
}
.消费者Consumer
public class Consumer {
public static void main(String[] args) throws Exception {
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1");
consumer.setNamesrvAddr("47.112.153.198:9876");
consumer.subscribe("topic1","*");
consumer.registerMessageListener(new MessageListenerConcurrently() {
@Override
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
for(MessageExt msg : list){
System.out.println("批量消息:"+new String(msg.getBody()));
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
});
consumer.start();
System.out.println("Consumer start...");
}
}
6.消息过滤
消费端可以根据消费规则选择性的消费符合要求的消息,过滤规则如下:
6.1主题过滤
消费者按topic过滤,只消费指定topic的消息。
代码略过。
6.2标签过滤
6.2.1、需要生产者创建消息时,指定tag
6.2.2、消费者按照tag过滤,只消费指定topic下对应tag的消息
6.2.3、消费时,通过tag过滤,支持指定一个或多个tag,多个tag用||拼接。eg:tag1 || tag2
代码演示:
.生产者Producer
public class Producer {
public static void main(String[] args) throws Exception {
DefaultMQProducer producer = new DefaultMQProducer("group1");
producer.setNamesrvAddr("47.112.153.198:9876");
//启动发送服务
producer.start();
//创建要发送的消息,指定主题topic和内容body
Message message = new Message("topic1", "tag1", "hello world".getBytes("utf-8"));
SendResult result = producer.send(message);
System.out.println("发送消息,返回结果:" + result);
producer.shutdown();
}
}
.消费者Consumer
public class Consumer {
public static void main(String[] args) throws MQClientException {
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1");
consumer.setNamesrvAddr(nameSrvAddr);
//设置接收消息对应的topic,指定接收的tag,tag1||tag2表示多个tag
consumer.subscribe("topic3", "tag1||tag2");
//开启监听,用于接收消息
consumer.registerMessageListener(new MessageListenerConcurrently() {
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
for (MessageExt msg : list) {
System.out.println("收到消息:" + new String(msg.getBody()));
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
});
//启动接收消息的服务
consumer.start();
System.out.println("接收消息服务已经启动");
}
}
6.3SQL过滤
6.3.1、生产者在创建消息时,为消息添加属性
6.3.2、消费者按照属性过滤,只消费指定topic下指定属性(或属性值)的消息
6.3.3、消费时,通过属性过滤。语法类似SQL,支持=
、>=
、<=
、or
、and
、in
,不支持模糊查询like
6.3.4、在broker.conf配置文件中,添加enablePropertyFilter=true
true:支持属性过滤,false:不支持属性过滤
演示代码:
.生产者Producer
public class Producer {
public static void main(String[] args) throws Exception {
DefaultMQProducer producer = new DefaultMQProducer("group1");
producer.setNamesrvAddr("47.112.153.198:9876");
//启动发送服务
producer.start();
//创建要发送的消息,指定主题topic,tag和body以及属性
Message message = new Message("topic1", "tag1", "hello leon".getBytes("utf-8"));
message.putUserProperty("vip", "1");
message.putUserProperty("age", "18");
message.putUserProperty("userName", "leon");
SendResult result = producer.send(message);
System.out.println("发送消息,返回结果:" + result);
producer.shutdown();
}
}
.消费者Consumer
public class Consumer {
public static void main(String[] args) throws Exception {
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1");
consumer.setNamesrvAddr("47.112.153.198:9876");
/**
* 使用sql过滤器过滤,语法格式为类SQL语法
*/
consumer.subscribe("topic3", MessageSelector.bySql("age >= 18"));
consumer.subscribe("topic3", MessageSelector.bySql("userName='leon'"));
/**
* 并集
*/
consumer.subscribe("topic3", MessageSelector.bySql("age >=18 or userName='leon'"));
/**
* 交集
*/
consumer.subscribe("topic3", MessageSelector.bySql("age >=18 and userName='leon'"));
/**
* 枚举tag
*/
consumer.subscribe("topic3", MessageSelector.bySql("TAGS in ('tag1','tag2')"));
//开启监听,用于接收消息
consumer.registerMessageListener(new MessageListenerConcurrently() {
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
for (MessageExt msg : list) {
System.out.println("收到消息:" + new String(msg.getBody()));
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
});
//启动接收消息的服务
consumer.start();
System.out.println("接收消息服务已经启动");
}
}
7.顺序消息
消息有序指的是可以按照消息的发送顺序来消费(FIFO)。RocketMQ可以严格的保证消息有序,可以分为分区有序或者全局有序。
顺序消费的原理解析,在默认的情况下消息发送会采取Round Robin轮询方式把消息发送到不同的queue(分区队列);而消费消息的时候从多个queue上拉取消息,这种情况发送和消费是不能保证顺序。但是如果控制发送的顺序消息只依次发送到同一个queue中,消费的时候只从这个queue上依次拉取,则就保证了顺序。当消息发送和消费参与的queue只有一个,则是全局有序;如果多个queue参与,则为分区有序,即相对每个queue,消息都是有序的。
下面用订单进行分区有序的示例。一个订单的顺序流程是:创建、付款、推送、完成。订单号相同的消息会被先后发送到同一个队列中,消费时,同一个OrderId获取到的肯定是同一个队列。
演示代码:
.生产者Producer
#实体类
@Data
public class OrderMsg {
/**
* 为了便于区分,同一个主单的多个Order对象id相同
*/
private String orderId;
/**
* 为了便于区分,msg描述当前order对象是主单还是子单
*/
private String msg;
}
#生产者Producer
public class Producer {
public static void main(String[] args) throws Exception {
DefaultMQProducer producer = new DefaultMQProducer("orderGroup1");
producer.setNamesrvAddr("47.112.153.198:9876");
//启动发送服务
producer.start();
List<OrderMsg> orderList=getOrderList();
for (final OrderMsg order : orderList) {
Message message = new Message("orderTopic",order.toString().getBytes("utf-8"));
SendResult result = producer.send(message, new MessageQueueSelector() {
public MessageQueue select(List<MessageQueue> list, Message msg, Object o) {
// 一致性hash算法
//根据发送的信息不同,选择不同的消息队列
//根据id来选择一个消息队列的对象,先获取hashcode,再取模
int index = order.getOrderId().hashCode() % list.size();
return list.get(index);
}
}, null);
System.out.println("发送消息,返回结果:" + result);
}
producer.shutdown();
}
private static List<OrderMsg> getOrderList(){
List<OrderMsg> orderList = new ArrayList<OrderMsg>();
OrderMsg order11 = new OrderMsg();
order11.setOrderId("a");
order11.setMsg("主单-1");
OrderMsg order12 = new OrderMsg();
order12.setOrderId("a");
order12.setMsg("子单-2");
OrderMsg order13 = new OrderMsg();
order13.setOrderId("a");
order13.setMsg("子单-3");
OrderMsg order21 = new OrderMsg();
order21.setOrderId("b");
order21.setMsg("主单-1");
OrderMsg order22 = new OrderMsg();
order22.setOrderId("b");
order22.setMsg("子单-2");
OrderMsg order23 = new OrderMsg();
order23.setOrderId("b");
order23.setMsg("子单-3");
OrderMsg order31 = new OrderMsg();
order31.setOrderId("c");
order31.setMsg("主单-1");
OrderMsg order32 = new OrderMsg();
order32.setOrderId("c");
order32.setMsg("子单-2");
OrderMsg order33 = new OrderMsg();
order33.setOrderId("c");
order33.setMsg("子单-3");
orderList.add(order11);
orderList.add(order12);
orderList.add(order13);
orderList.add(order21);
orderList.add(order22);
orderList.add(order23);
orderList.add(order31);
orderList.add(order32);
orderList.add(order33);
return orderList;
}
}
.消费者Consumer
public class Consumer {
public static void main(String[] args) throws Exception {
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("orderGroup1");
consumer.setNamesrvAddr(nameSrvAddr);
consumer.subscribe("orderTopic", "*");
//开启监听,用于接收消息
consumer.registerMessageListener(new MessageListenerOrderly() {
public ConsumeOrderlyStatus consumeMessage(List<MessageExt> list, ConsumeOrderlyContext consumeOrderlyContext) {
for (MessageExt msg : list) {
System.out.println(Thread.currentThread().getName() + "----收到顺序消息:" + new String(msg.getBody()));
}
return ConsumeOrderlyStatus.SUCCESS;
}
});
//启动接收消息的服务
consumer.start();
System.out.println("接收消息服务已经启动");
}
}
顺序消息总结:
存入的时候有序:同一个业务的多个消息有序的存入同一个队列。实现:让业务id
和队列id
绑定
消费的时候有序:只能有一个确定的线程消费当前对列。
8.事务消息
8.1事务消息共有三种状态
提交状态、回滚状态、中间状态:
TransactionStatus.CommitTransaction: 提交事务,它允许消费者消费此消息。 TransactionStatus.RollbackTransaction: 回滚事务,它代表该消息将被删除,不允许被消费。 TransactionStatus.Unknown: 中间状态,它代表需要检查消息队列来确定状态。
8.2事务消息两个过程
-
正常事务过程。本地事务没有卡住,直接回滚或者提交了;继而直接发送通知给
broker
,让其处理消息。 -
事务补偿过程。事务回检过程。本地事务卡主了,
broker
等急了,所以不断的来问问。
8.3事务消息流转过程
演示代码:
.生产者Producer
public class Producer {
public static void main(String[] args) throws Exception {
TransactionMQProducer producer=new TransactionMQProducer("group1");
producer.setNamesrvAddr(nameSrvAddr);
/**
* 添加本地事务监听
*/
producer.setTransactionListener(new TransactionListener() {
//正常事务回调方法
public LocalTransactionState executeLocalTransaction(Message message, Object o) {
// return LocalTransactionState.UNKNOW;
//return LocalTransactionState.COMMIT_MESSAGE;
return LocalTransactionState.ROLLBACK_MESSAGE;
}
//事务补偿过程
public LocalTransactionState checkLocalTransaction(MessageExt messageExt) {
System.out.println("事务补偿过程执行");
//return LocalTransactionState.COMMIT_MESSAGE;
//return null;
return null;
}
});
producer.start();
Message message=new Message("topic1","事务消息:hello rocketMq".getBytes("UTF-8"));
//发送事务消息sendMessageInTransaction,而不是send
SendResult sendResult = producer.sendMessageInTransaction(message,null);
System.out.println("事务消息返回结果:"+sendResult);
//producer.shutdown();
}
}
.消费者Consumer
public class Consumer {
public static void main(String[] args) throws Exception {
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("group1");
consumer.setNamesrvAddr(nameSrvAddr);
consumer.subscribe("topic1", "*");
consumer.setMessageListener(new MessageListenerConcurrently() {
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> list, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
for (MessageExt msg : list) {
System.out.println("收到事务消息:" + new String(msg.getBody()));
}
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
});
consumer.start();
System.out.println("接收消息服务已经开启运行");
}
}