在订阅消息的时候,有时我们希望消息能按照一定业务顺序消费,比如一个订单创建,订单修改,订单完成。这时候是需要顺序消息。RocketMQ支持顺序消费,下面来研究一下实现逻辑。
样例
生产者
public class OrderedProducer {
public static void main(String[] args) throws Exception {
//Instantiate with a producer group name.
MQProducer producer = new DefaultMQProducer("example_group_name");
//Launch the instance.
producer.start();
String[] tags = new String[] {"TagA", "TagB", "TagC", "TagD", "TagE"};
for (int i = 0; i < 100; i++) {
int orderId = i % 10;
//Create a message instance, specifying topic, tag and message body.
Message msg = new Message("TopicTest", tags[i % tags.length], "KEY" + i,
("Hello RocketMQ " + i).getBytes());
SendResult sendResult = producer.send(msg, new MessageQueueSelector() {
@Override
public MessageQueue select(List<MessageQueue> mqs, Message msg, Object arg) {
Integer id = (Integer) arg;
int index = id % mqs.size();
return mqs.get(index);
}
}, orderId);
System.out.printf("%s%n", sendResult);
}
//server shutdown
producer.shutdown();
}
}
消费者
public class OrderedConsumer {
public static void main(String[] args) throws Exception {
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("example_group_name");
consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);
consumer.subscribe("TopicTest", "TagA || TagC || TagD");
consumer.registerMessageListener(new MessageListenerOrderly() {
AtomicLong consumeTimes = new AtomicLong(0);
@Override
public ConsumeOrderlyStatus consumeMessage(List<MessageExt> msgs,
ConsumeOrderlyContext context) {
Random random = new Random();
context.setAutoCommit(false);
System.out.print(Thread.currentThread().getName() + " Receive New Messages: " );
for (MessageExt msg: msgs) {
System.out.println("topic=" + msg.getTopic() + ",tags=" + msg.getTags() + ", content:" + new String(msg.getBody()));
}
return ConsumeOrderlyStatus.SUCCESS;
}
});
consumer.start();
System.out.printf("Consumer Started.%n");
}
}
输出如下,从结果来看,虽然消费端创建了多个线程消费,但是从不同tag来看都是有序的。
ConsumeMessageThread_16 Receive New Messages: topic=TopicTest,tags=TagA, content:Hello RocketMQ 40
ConsumeMessageThread_16 Receive New Messages: topic=TopicTest,tags=TagD, content:Hello RocketMQ 48
ConsumeMessageThread_16 Receive New Messages: topic=TopicTest,tags=TagA, content:Hello RocketMQ 50
ConsumeMessageThread_16 Receive New Messages: topic=TopicTest,tags=TagD, content:Hello RocketMQ 58
ConsumeMessageThread_16 Receive New Messages: topic=TopicTest,tags=TagA, content:Hello RocketMQ 60
ConsumeMessageThread_16 Receive New Messages: topic=TopicTest,tags=TagD, content:Hello RocketMQ 68
ConsumeMessageThread_16 Receive New Messages: topic=TopicTest,tags=TagA, content:Hello RocketMQ 70
ConsumeMessageThread_16 Receive New Messages: topic=TopicTest,tags=TagD, content:Hello RocketMQ 78
ConsumeMessageThread_16 Receive New Messages: topic=TopicTest,tags=TagA, content:Hello RocketMQ 80
ConsumeMessageThread_16 Receive New Messages: topic=TopicTest,tags=TagD, content:Hello RocketMQ 88
ConsumeMessageThread_16 Receive New Messages: topic=TopicTest,tags=TagA, content:Hello RocketMQ 90
ConsumeMessageThread_16 Receive New Messages: topic=TopicTest,tags=TagD, content:Hello RocketMQ 98
...
源码分析
生产者
生产者代码不能难看,在发送消息的时候,进行了orderId进行哈希,让同一个orderId每次发送到同一个队列,保证同一个队列单个线程消费肯定是有序的。
消费者
首先了解一些概念
MessageQueue是逻辑队列,包含topic和brokerName以及queueId。通过MessageQueue可以中broker定位到消息队列。
public class MessageQueue implements Comparable<MessageQueue>, Serializable {
private static final long serialVersionUID = 6191200464116433425L;
private String topic;
private String brokerName;
private int queueId;
PullRequest由MessageQueue和ProcessQueue。
public class PullRequest {
private String consumerGroup;
private MessageQueue messageQueue;
private ProcessQueue processQueue;
private long nextOffset;
ProcessQueue是消费端实际的消费载体。
/**
* Queue consumption snapshot
*
* @author shijia.wxr<vintage.wang@gmail.com>
* @since 2013-7-24
*/
public class ProcessQueue {
public final static long RebalanceLockMaxLiveTime = Long.parseLong(System.getProperty(
"rocketmq.client.rebalance.lockMaxLiveTime", "30000"));
public final static long RebalanceLockInterval = Long.parseLong(System.getProperty(
"rocketmq.client.rebalance.lockInterval", "20000"));
private final Logger log = ClientLogger.getLog();
private final ReadWriteLock lockTreeMap = new ReentrantReadWriteLock();
private final TreeMap<Long, MessageExt> msgTreeMap = new TreeMap<Long, MessageExt>();
private volatile long queueOffsetMax = 0L;
private final AtomicLong msgCount = new AtomicLong();
入口
消费者内部实现比较复杂,实现逻辑在于ConsumeMessageOrderlyService。这个在注册MessageListenerOrderly会生成。
if (this.getMessageListenerInner() instanceof MessageListenerOrderly) {
this.consumeOrderly = true;
this.consumeMessageService =
new ConsumeMessageOrderlyService(this,
(MessageListenerOrderly) this.getMessageListenerInner());
}
else if (this.getMessageListenerInner() instanceof MessageListenerConcurrently) {
this.consumeOrderly = false;
this.consumeMessageService =
new ConsumeMessageConcurrentlyService(this,
(MessageListenerConcurrently) this.getMessageListenerInner());
}
this.consumeMessageService.start();
在consumer.start()后会创建了定时任务,默认每隔20秒向broker锁住当前消费端的消费队列MessageQueue。保证其他消费端不能消费这些队列消息。
public void start() {
if (MessageModel.CLUSTERING.equals(ConsumeMessageOrderlyService.this.defaultMQPushConsumerImpl
.messageModel())) {
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
ConsumeMessageOrderlyService.this.lockMQPeriodically();
}
}, 1000 * 1, ProcessQueue.RebalanceLockInterval, TimeUnit.MILLISECONDS);
}
}
public void lockAll() {
HashMap<String, Set<MessageQueue>> brokerMqs = this.buildProcessQueueTableByBrokerName();
Iterator<Entry<String, Set<MessageQueue>>> it = brokerMqs.entrySet().iterator();
while (it.hasNext()) {
Entry<String, Set<MessageQueue>> entry = it.next();
final String brokerName = entry.getKey();
final Set<MessageQueue> mqs = entry.getValue();
if (mqs.isEmpty())
continue;
FindBrokerResult findBrokerResult =
this.mQClientFactory.findBrokerAddressInSubscribe(brokerName, MixAll.MASTER_ID, true);
if (findBrokerResult != null) {
LockBatchRequestBody requestBody = new LockBatchRequestBody();
requestBody.setConsumerGroup(this.consumerGroup);
requestBody.setClientId(this.mQClientFactory.getClientId());
requestBody.setMqSet(mqs);
try {
Set<MessageQueue> lockOKMQSet =
this.mQClientFactory.getMQClientAPIImpl().lockBatchMQ(
findBrokerResult.getBrokerAddr(), requestBody, 1000);
for (MessageQueue mq : lockOKMQSet) {
ProcessQueue processQueue = this.processQueueTable.get(mq);
if (processQueue != null) {
if (!processQueue.isLocked()) {
log.info("the message queue locked OK, Group: {} {}", this.consumerGroup, mq);
}
processQueue.setLocked(true);
processQueue.setLastLockTimestamp(System.currentTimeMillis());
}
}
for (MessageQueue mq : mqs) {
if (!lockOKMQSet.contains(mq)) {
ProcessQueue processQueue = this.processQueueTable.get(mq);
if (processQueue != null) {
processQueue.setLocked(false);
log.warn("the message queue locked Failed, Group: {} {}", this.consumerGroup,
mq);
}
}
}
} catch (Exception e) {
log.error("lockBatchMQ exception, " + mqs, e);
}
}
}
}
拉取消息
PullMessageService负责不断地把消息拉过来消费。简单从pullRequestQueue取出来PullRequest,执行拉取操作。PullRequest由DefaultMQPushConsumerImpl推过来放入pullRequestQueue中。
public void run() {
log.info(this.getServiceName() + " service started");
while (!this.isStoped()) {
try {
PullRequest pullRequest = this.pullRequestQueue.take();
if (pullRequest != null) {
this.pullMessage(pullRequest);
}
}
catch (InterruptedException e) {
}
catch (Exception e) {
log.error("Pull Message Service Run Method exception", e);
}
}
log.info(this.getServiceName() + " service end");
}
负载均衡
RebalanceImpl定时10秒钟负载权衡,发送PullRequest,向broker请求拉取消息。负载几个就创建几个PullRequest。
List<PullRequest> pullRequestList = new ArrayList<PullRequest>();
for (MessageQueue mq : mqSet) {
if (!this.processQueueTable.containsKey(mq)) {
PullRequest pullRequest = new PullRequest();
pullRequest.setConsumerGroup(consumerGroup);
pullRequest.setMessageQueue(mq);
pullRequest.setProcessQueue(new ProcessQueue());
long nextOffset = this.computePullFromWhere(mq);
if (nextOffset >= 0) {
pullRequest.setNextOffset(nextOffset);
pullRequestList.add(pullRequest);
changed = true;
this.processQueueTable.put(mq, pullRequest.getProcessQueue());
log.info("doRebalance, {}, add a new mq, {}", consumerGroup, mq);
} else {
log.warn("doRebalance, {}, add new mq failed, {}", consumerGroup, mq);
}
}
}
this.dispatchPullRequest(pullRequestList);
RebalancePushImpl分发
public void dispatchPullRequest(List<PullRequest> pullRequestList) {
for (PullRequest pullRequest : pullRequestList) {
this.defaultMQPushConsumerImpl.executePullRequestImmediately(pullRequest);
log.info("doRebalance, {}, add a new pull request {}", consumerGroup, pullRequest);
}
}
通过PullMessageService把PullRequest发送出去
public void run() {
log.info(this.getServiceName() + " service started");
while (!this.isStoped()) {
try {
PullRequest pullRequest = this.pullRequestQueue.take();
if (pullRequest != null) {
this.pullMessage(pullRequest);
}
}
catch (InterruptedException e) {
}
catch (Exception e) {
log.error("Pull Message Service Run Method exception", e);
}
}
log.info(this.getServiceName() + " service end");
}
拉取消息
broker在收到PullRequest后会,响应生成PullResult返回给消费端。通过PullCallback回调消费消息。在callback中,DefaultMQPushConsumerImpl.this.consumeMessageService.submitConsumeRequest提交任务异步消费。然后将pullRequest中队列偏移量offset设置为PullRequest回传后的偏移量,继续发送PullRequest拉取消息。
PullCallback pullCallback = new PullCallback() {
@Override
public void onSuccess(PullResult pullResult) {
if (pullResult != null) {
pullResult =
DefaultMQPushConsumerImpl.this.pullAPIWrapper.processPullResult(
pullRequest.getMessageQueue(), pullResult, subscriptionData);
switch (pullResult.getPullStatus()) {
case FOUND:
long prevRequestOffset = pullRequest.getNextOffset();
pullRequest.setNextOffset(pullResult.getNextBeginOffset());
long pullRT = System.currentTimeMillis() - beginTimestamp;
DefaultMQPushConsumerImpl.this.getConsumerStatsManager().incPullRT(
pullRequest.getConsumerGroup(), pullRequest.getMessageQueue().getTopic(), pullRT);
long firstMsgOffset = Long.MAX_VALUE;
if (pullResult.getMsgFoundList() == null || pullResult.getMsgFoundList().isEmpty()) {
DefaultMQPushConsumerImpl.this.executePullRequestImmediately(pullRequest);
}
else {
firstMsgOffset = pullResult.getMsgFoundList().get(0).getQueueOffset();
DefaultMQPushConsumerImpl.this.getConsumerStatsManager().incPullTPS(
pullRequest.getConsumerGroup(), pullRequest.getMessageQueue().getTopic(),
pullResult.getMsgFoundList().size());
boolean dispathToConsume = processQueue.putMessage(pullResult.getMsgFoundList());
DefaultMQPushConsumerImpl.this.consumeMessageService.submitConsumeRequest(//
pullResult.getMsgFoundList(), //
processQueue, //
pullRequest.getMessageQueue(), //
dispathToConsume);
if (DefaultMQPushConsumerImpl.this.defaultMQPushConsumer.getPullInterval() > 0) {
DefaultMQPushConsumerImpl.this.executePullRequestLater(pullRequest,
DefaultMQPushConsumerImpl.this.defaultMQPushConsumer.getPullInterval());
}
else {
DefaultMQPushConsumerImpl.this.executePullRequestImmediately(pullRequest);
}
}
if (pullResult.getNextBeginOffset() < prevRequestOffset//
|| firstMsgOffset < prevRequestOffset) {
log.warn(
"[BUG] pull message result maybe data wrong, nextBeginOffset: {} firstMsgOffset: {} prevRequestOffset: {}",//
pullResult.getNextBeginOffset(),//
firstMsgOffset,//
prevRequestOffset);
}
break;
消费消息
通过创建ConsumeRequest任务消费。每次都消费那些被锁定的队列。
每次获取锁,保证消费端只有一个线程消费。这样保证消费有序的。而且只有消息队列被锁定才能消费。
public void run() {
if (this.processQueue.isDropped()) {
log.warn("run, the message queue not be able to consume, because it's dropped. {}",
this.messageQueue);
return;
}
final Object objLock = messageQueueLock.fetchLockObject(this.messageQueue);
synchronized (objLock) {
if (MessageModel.BROADCASTING
.equals(ConsumeMessageOrderlyService.this.defaultMQPushConsumerImpl.messageModel())
|| (this.processQueue.isLocked() && !this.processQueue.isLockExpired()))
if (MessageModel.CLUSTERING
.equals(ConsumeMessageOrderlyService.this.defaultMQPushConsumerImpl
.messageModel())
&& !this.processQueue.isLocked()) {
log.warn("the message queue not locked, so consume later, {}", this.messageQueue);
ConsumeMessageOrderlyService.this.tryLockLaterAndReconsume(this.messageQueue,
this.processQueue, 10);
break;
}
if (MessageModel.CLUSTERING
.equals(ConsumeMessageOrderlyService.this.defaultMQPushConsumerImpl
.messageModel())
&& this.processQueue.isLockExpired()) {
log.warn("the message queue lock expired, so consume later, {}",
this.messageQueue);
ConsumeMessageOrderlyService.this.tryLockLaterAndReconsume(this.messageQueue,
this.processQueue, 10);
break;
}
...
}
调用processQueue.takeMessags拿出消息,调用用户端MessageListenerOrderly消费消息
List<MessageExt> msgs = this.processQueue.takeMessags(consumeBatchSize);
...
try {
this.processQueue.getLockConsume().lock();
if (this.processQueue.isDropped()) {
log.warn(
"consumeMessage, the message queue not be able to consume, because it's dropped. {}",
this.messageQueue);
break;
}
status =
messageListener.consumeMessage(Collections.unmodifiableList(msgs),
context);
}
总结
保证消息顺序消费有两方面:
- 1.通过ReblanceImp的lockAll方法每隔一段时间定时锁住当前消费端消费的队列,设置本地队列ProcessQueue的locked属性为true。保证broker中的每个消息队列只对应一个消费端。
-
- 在消费端也是通过锁,保证每个ProcessQueue只有一个线程消费。
这里保证有序是保证同一个队列是有序消费的,但是不同的队列消费顺序是不能保证的。