RocketMQ入门-顺序消息

Rocket顺序消息

本篇基于RocketMQ集群开发,如需了解集群的搭建,可参考之前的文章RocketMQ集群安装

摘录RocketMQ官方示例中顺序消息的解释信息:

1.顺序消息指按照消息发送的顺序(FIFO)消费。RocketMQ支持分区有序和全局有序。

2.在默认的情况下消息发送会采取Round Robin轮询方式把消息发送到不同的queue(分区队列);而消费消息的时候从多个queue上拉取消息,这种情况发送和消费是不能保证顺序。但是如果控制发送的顺序消息只依次发送到同一个queue中,消费的时候只从这个queue上依次拉取,则就保证了顺序。当发送和消费参与的queue只有一个,则是全局有序;如果多个queue参与,则为分区有序,即相对每个queue,消息都是有序的。

顺序队列的关键就是让同一个订单的消息,按顺序放入同一个队列。

下面用订单进行分区有序的示例。一个订单的顺序流程是:创建、付款、推送、完成。订单号相同的消息会被先后发送到同一个队列中,消费时,同一个OrderId获取到的肯定是同一个队列。

Producer生产者

生产者使用send(Message msg, MessageQueueSelector selector, Object arg)方法,控制订单消息分区有序需要MessageQueueSelector对订单号相同的返回同一个MessageQueue消息队列。

public class OrderProducer {

    public static void main(String[] args) throws Exception {
        // 1.指定生产者组
        DefaultMQProducer defaultMQProducer = new DefaultMQProducer(MyRocketMqConstant.OrderLearn.ORDER_LEARN_PRODUCER_GROUP);
        // 2.关联注册中心
        defaultMQProducer.setNamesrvAddr(MyRocketMqConstant.NAME_SRV);
        // 3.启动生产者
        defaultMQProducer.start();
        String[] tags = {MyRocketMqConstant.OrderLearn.ORDER_LEARN_TAG_A
                , MyRocketMqConstant.OrderLearn.ORDER_LEARN_TAG_B
                , MyRocketMqConstant.OrderLearn.ORDER_LEARN_TAG_C};


        // 模拟订单数据
        List<OrderStep> orderStepList = new OrderProducer().buildOrders();

        Date date = new Date();
        SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
        String dateStr = sdf.format(date);

        for (int i = 0; i < 10; i++) {
            // 构造消息数据
            String body = orderStepList.get(i) + "Hello RocketMQ" + dateStr;
            Message message = new Message(MyRocketMqConstant.OrderLearn.ORDER_LEARN_TOPIC
                    , tags[i % tags.length]
                    , "KEY" + i, body.getBytes(StandardCharsets.UTF_8));

            SendResult sendResult = defaultMQProducer.send(message, new MessageQueueSelector() {
                // 消息队列对象
                @Override
                public MessageQueue select(List<MessageQueue> mqs, Message msg, Object arg) {
                    // arg来自于send(Message msg, MessageQueueSelector selector, Object arg)的第三个参数arg
                    // 同一订单号使用同一消息队列
                    Long id = (Long) arg;
                    long index = id % mqs.size();
                    return mqs.get((int) index);
                }
            }, orderStepList.get(i).getOrderId());

            System.out.println(String.format("SendResult status:%s,queueId:%d,body:%s",
                    sendResult.getSendStatus(),
                    sendResult.getMessageQueue().getQueueId(),
                    body));
        }
        defaultMQProducer.shutdown();
    }

    /**
     * 订单的步骤
     */
    private static class OrderStep {
        private long orderId;
        private String desc;

        public long getOrderId() {
            return orderId;
        }

        public void setOrderId(long orderId) {
            this.orderId = orderId;
        }

        public String getDesc() {
            return desc;
        }

        public void setDesc(String desc) {
            this.desc = desc;
        }

        @Override
        public String toString() {
            return "OrderStep{" +
                    "orderId=" + orderId +
                    ", desc='" + desc + '\'' +
                    '}';
        }
    }

    /**
     * 生成模拟订单数据
     * 保证订单的生成顺序是:创建->付款->推送->完成
     */
    private List<OrderStep> buildOrders() {
        List<OrderStep> orderList = new ArrayList<OrderStep>();

        OrderStep orderDemo = new OrderStep();
        orderDemo.setOrderId(15103111039L);
        orderDemo.setDesc("创建");
        orderList.add(orderDemo);

        orderDemo = new OrderStep();
        orderDemo.setOrderId(15103111065L);
        orderDemo.setDesc("创建");
        orderList.add(orderDemo);

        orderDemo = new OrderStep();
        orderDemo.setOrderId(15103111039L);
        orderDemo.setDesc("付款");
        orderList.add(orderDemo);

        orderDemo = new OrderStep();
        orderDemo.setOrderId(15103117235L);
        orderDemo.setDesc("创建");
        orderList.add(orderDemo);

        orderDemo = new OrderStep();
        orderDemo.setOrderId(15103111065L);
        orderDemo.setDesc("付款");
        orderList.add(orderDemo);

        orderDemo = new OrderStep();
        orderDemo.setOrderId(15103117235L);
        orderDemo.setDesc("付款");
        orderList.add(orderDemo);

        orderDemo = new OrderStep();
        orderDemo.setOrderId(15103111065L);
        orderDemo.setDesc("完成");
        orderList.add(orderDemo);

        orderDemo = new OrderStep();
        orderDemo.setOrderId(15103111039L);
        orderDemo.setDesc("推送");
        orderList.add(orderDemo);

        orderDemo = new OrderStep();
        orderDemo.setOrderId(15103117235L);
        orderDemo.setDesc("完成");
        orderList.add(orderDemo);

        orderDemo = new OrderStep();
        orderDemo.setOrderId(15103111039L);
        orderDemo.setDesc("完成");
        orderList.add(orderDemo);

        return orderList;
    }
}

Consumer消费者

public class OrderConsumer {

    public static void main(String[] args) throws Exception {
        // 1.指定消费者组
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer(MyRocketMqConstant.OrderLearn.ORDER_LEARN_CONSUMER_GROUP);
        // 2.关联注册中心
        consumer.setNamesrvAddr(MyRocketMqConstant.NAME_SRV);
        // 3.设置Consumer第一次启动是从队列头部开始消费还是从队列尾部开始消费
        // 如果非第一次启动,那么按照上次消费的位置继续消费
        consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);

        String[] tags = {MyRocketMqConstant.OrderLearn.ORDER_LEARN_TAG_A
                , MyRocketMqConstant.OrderLearn.ORDER_LEARN_TAG_B
                , MyRocketMqConstant.OrderLearn.ORDER_LEARN_TAG_C};
        String join = String.join(" || ", tags);

        consumer.subscribe(MyRocketMqConstant.OrderLearn.ORDER_LEARN_TOPIC, join);

        consumer.registerMessageListener(new MessageListenerOrderly() {
            Random random = new Random();
            @Override
            public ConsumeOrderlyStatus consumeMessage(List<MessageExt> msgs, ConsumeOrderlyContext context) {
                context.setAutoCommit(true);
                for (MessageExt msg : msgs) {
                    // 每个queue不一定有唯一的consumer线程消费,订单对每个queue有序
                    System.out.println("[consumerThread="+Thread.currentThread().getName()+"] [queueId="+msg.getQueueId()+"] content"+new String(msg.getBody()));
                }

                try {
                    // 模拟业务逻辑处理...
                    TimeUnit.SECONDS.sleep(random.nextInt(10));
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
                return ConsumeOrderlyStatus.SUCCESS;
            }
        });

        consumer.start();
    }
}

tips:RocketMQ源码中示例说:每个queue有唯一的consumer线程消费,订单对每个queue有序,实践证明这个描述不正确。正确的是:每个queue不一定有唯一的consumer线程消费,订单对每个queue有序。

测试结果

总共测试了三次,输出结果如下(可以把输出内容放入notepad++中,双击订单号查看同订单号的输出顺序)。

发现消费者的输出顺序不一定会一致,但是针对同一个订单的输出顺序是一定相同的。同时消费者消费时,同一个订单号可能有不同的线程消费。

生产者输出

生产者
//队列3
SendResult status:SEND_OK,queueId:3,body:OrderStep{orderId=15103111039, desc='创建'}
SendResult status:SEND_OK,queueId:3,body:OrderStep{orderId=15103111039, desc='付款'}
SendResult status:SEND_OK,queueId:3,body:OrderStep{orderId=15103117235, desc='创建'}
SendResult status:SEND_OK,queueId:3,body:OrderStep{orderId=15103117235, desc='付款'}
SendResult status:SEND_OK,queueId:3,body:OrderStep{orderId=15103111039, desc='推送'}
SendResult status:SEND_OK,queueId:3,body:OrderStep{orderId=15103117235, desc='完成'}
SendResult status:SEND_OK,queueId:3,body:OrderStep{orderId=15103111039, desc='完成'}

//队列1
SendResult status:SEND_OK,queueId:1,body:OrderStep{orderId=15103111065, desc='创建'}
SendResult status:SEND_OK,queueId:1,body:OrderStep{orderId=15103111065, desc='付款'}
SendResult status:SEND_OK,queueId:1,body:OrderStep{orderId=15103111065, desc='完成'}

消费者输出

消费者
//队列3
[consumerThread=ConsumeMessageThread_1] [broker=broker-b/192.168.15.16:10911] [queueId=3] [content=OrderStep{orderId=15103111039, desc='创建'}]
[consumerThread=ConsumeMessageThread_3] [broker=broker-a/192.168.15.15:10911] [queueId=3] [content=OrderStep{orderId=15103117235, desc='创建'}]
[consumerThread=ConsumeMessageThread_1] [broker=broker-b/192.168.15.16:10911] [queueId=3] [content=OrderStep{orderId=15103111039, desc='付款'}]
[consumerThread=ConsumeMessageThread_1] [broker=broker-b/192.168.15.16:10911] [queueId=3] [content=OrderStep{orderId=15103111039, desc='推送'}]
[consumerThread=ConsumeMessageThread_1] [broker=broker-b/192.168.15.16:10911] [queueId=3] [content=OrderStep{orderId=15103111039, desc='完成'}]
[consumerThread=ConsumeMessageThread_3] [broker=broker-a/192.168.15.15:10911] [queueId=3] [content=OrderStep{orderId=15103117235, desc='付款'}]
[consumerThread=ConsumeMessageThread_3] [broker=broker-a/192.168.15.15:10911] [queueId=3] [content=OrderStep{orderId=15103117235, desc='完成'}]

//队列1
[consumerThread=ConsumeMessageThread_2] [broker=broker-a/192.168.15.15:10911] [queueId=1] [content=OrderStep{orderId=15103111065, desc='创建'}]
[consumerThread=ConsumeMessageThread_2] [broker=broker-a/192.168.15.15:10911] [queueId=1] [content=OrderStep{orderId=15103111065, desc='付款'}]
[consumerThread=ConsumeMessageThread_2] [broker=broker-a/192.168.15.15:10911] [queueId=1] [content=OrderStep{orderId=15103111065, desc='完成'}]


//==================================================测试分隔线===================================================================================

//队列3
[consumerThread=ConsumeMessageThread_4] [broker=broker-b/192.168.15.16:10911] [queueId=3] [content=OrderStep{orderId=15103111039, desc='创建'}]
[consumerThread=ConsumeMessageThread_6] [broker=broker-a/192.168.15.15:10911] [queueId=3] [content=OrderStep{orderId=15103117235, desc='创建'}]
[consumerThread=ConsumeMessageThread_4] [broker=broker-b/192.168.15.16:10911] [queueId=3] [content=OrderStep{orderId=15103111039, desc='付款'}]
[consumerThread=ConsumeMessageThread_6] [broker=broker-a/192.168.15.15:10911] [queueId=3] [content=OrderStep{orderId=15103117235, desc='付款'}]
[consumerThread=ConsumeMessageThread_6] [broker=broker-a/192.168.15.15:10911] [queueId=3] [content=OrderStep{orderId=15103117235, desc='完成'}]
[consumerThread=ConsumeMessageThread_4] [broker=broker-b/192.168.15.16:10911] [queueId=3] [content=OrderStep{orderId=15103111039, desc='推送'}]
[consumerThread=ConsumeMessageThread_4] [broker=broker-b/192.168.15.16:10911] [queueId=3] [content=OrderStep{orderId=15103111039, desc='完成'}]

//队列1
[consumerThread=ConsumeMessageThread_5] [broker=broker-a/192.168.15.15:10911] [queueId=1] [content=OrderStep{orderId=15103111065, desc='创建'}]
[consumerThread=ConsumeMessageThread_5] [broker=broker-a/192.168.15.15:10911] [queueId=1] [content=OrderStep{orderId=15103111065, desc='付款'}]
[consumerThread=ConsumeMessageThread_5] [broker=broker-a/192.168.15.15:10911] [queueId=1] [content=OrderStep{orderId=15103111065, desc='完成'}]


//==================================================测试分隔线===================================================================================

//队列3
[consumerThread=ConsumeMessageThread_7] [broker=broker-b/192.168.15.16:10911] [queueId=3] [content=OrderStep{orderId=15103111039, desc='创建'}]
[consumerThread=ConsumeMessageThread_9] [broker=broker-a/192.168.15.15:10911] [queueId=3] [content=OrderStep{orderId=15103117235, desc='创建'}]
[consumerThread=ConsumeMessageThread_10] [broker=broker-a/192.168.15.15:10911] [queueId=3] [content=OrderStep{orderId=15103117235, desc='付款'}]
[consumerThread=ConsumeMessageThread_10] [broker=broker-a/192.168.15.15:10911] [queueId=3] [content=OrderStep{orderId=15103117235, desc='完成'}]
[consumerThread=ConsumeMessageThread_7] [broker=broker-b/192.168.15.16:10911] [queueId=3] [content=OrderStep{orderId=15103111039, desc='付款'}]
[consumerThread=ConsumeMessageThread_7] [broker=broker-b/192.168.15.16:10911] [queueId=3] [content=OrderStep{orderId=15103111039, desc='推送'}]
[consumerThread=ConsumeMessageThread_7] [broker=broker-b/192.168.15.16:10911] [queueId=3] [content=OrderStep{orderId=15103111039, desc='完成'}]
//队列1
[consumerThread=ConsumeMessageThread_8] [broker=broker-a/192.168.15.15:10911] [queueId=1] [content=OrderStep{orderId=15103111065, desc='创建'}]
[consumerThread=ConsumeMessageThread_8] [broker=broker-a/192.168.15.15:10911] [queueId=1] [content=OrderStep{orderId=15103111065, desc='付款'}]
[consumerThread=ConsumeMessageThread_8] [broker=broker-a/192.168.15.15:10911] [queueId=1] [content=OrderStep{orderId=15103111065, desc='完成'}]

总结

消费时同一订单号的消息顺序是如何保证和生产者生成订单相同的?需要关注MessageQueueSelector,它的返回值就是消息所关联的队列。

MessageQueueSelector

创建的匿名内部类如下:

 new MessageQueueSelector() {
                // 消息队列对象
                @Override
                public MessageQueue select(List<MessageQueue> mqs, Message msg, Object arg) {
                    // arg来自于send(Message msg, MessageQueueSelector selector, Object arg)的第三个参数arg
                    // 同一订单号使用同一消息队列
                    Long id = (Long) arg;
                    long index = id % mqs.size();
                    return mqs.get((int) index);
                }

测试环境使用了集群,2个broker-a,2个broker-b;创建MessageQueueSelector对象时,参数List<MessageQueue> mqs内容如下,有8条记录。Long id = (Long) arg;获取到send方法传入的订单id,long index = id % mqs.size();取余会使同一订单id的不同消息关联同一个消息队列。

根据上面的消费者输出测试结果,三次输出的消费者信息顺序虽然不完全一致,但是同一个订单id的顺序一定是一致的,因为同一订单号的消息关联了同一个消息队列,这就是RocketMQ的分区顺序。

[
	{
		"brokerName": "broker-a","queueId": 0,"topic": "order-topic"
	},
	{
		"brokerName": "broker-a","queueId": 1,"topic": "order-topic"
	},
	{
		"brokerName": "broker-a","queueId": 2,"topic": "order-topic"
	},
	{
		"brokerName": "broker-a","queueId": 3,"topic": "order-topic"
	},
	{
		"brokerName": "broker-b","queueId": 0,"topic": "order-topic"
	},
	{
		"brokerName": "broker-b","queueId": 1,"topic": "order-topic"
	},
	{
		"brokerName": "broker-b","queueId": 2,"topic": "order-topic"
	},
	{
		"brokerName": "broker-b","queueId": 3,"topic": "order-topic"
	}
]

关闭slave02上的broker(启动了broker-b主节点和broker-a的从节点),启动生产者获取到的mqs对象内容如下:

[
	{
		"brokerName": "broker-a","queueId": 0,"topic": "order-topic"
	},
	{
		"brokerName": "broker-a","queueId": 1,"topic": "order-topic"
	},
	{
		"brokerName": "broker-a","queueId": 2,"topic": "order-topic"
	},
	{
		"brokerName": "broker-a","queueId": 3,"topic": "order-topic"
	}
]
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值