RocketMQ消费者配置消费场景

最新推荐文章于 2024-07-15 15:01:05 发布

跌撞江湖

最新推荐文章于 2024-07-15 15:01:05 发布

阅读量1.2k

点赞数

分类专栏：亲力亲为 MQ 文章标签：中间件

本文链接：https://blog.csdn.net/yanwendonge/article/details/103539473

版权

亲力亲为同时被 2 个专栏收录

46 篇文章 0 订阅

订阅专栏

9 篇文章 1 订阅

订阅专栏

先看一句话：”rocketmq客户端初始化DefaultMQPushConsumer依据topic的数量进行初始化“

场景一：
一个producer, 一个consumer1，一个consumer2

1.producer发送消息 3条

DefaultMQProducer producer = new DefaultMQProducer("group");
        producer.setRetryTimesWhenSendAsyncFailed(3);
        producer.setNamesrvAddr("127.0.0.1:9876");
        producer.start();
        String[] tags = new String[] {"tag"};
        for (int i = 0; i < 3; i++) {
            Message msg = new Message("topic", tags[i % tags.length],
                ("Hello RocketMQ" + i).getBytes(RemotingHelper.DEFAULT_CHARSET));
            producer.send(msg, new SendCallback() {
                @Override
                public void onSuccess(SendResult sendResult) {
                    System.out.println(sendResult);
                }
                @Override
                public void onException(Throwable e) {
                    e.printStackTrace();
                }
            });

2.启动consumer1，consumer2 订阅关系完全一致（groupName+topic+tag）

 DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("consumerGroup");
    consumer.setNamesrvAddr("localhost:9876");
    consumer.setMessageModel(MessageModel.CLUSTERING);
    try {
        consumer.subscribe("topic","tag");
        consumer.registerMessageListener(new MessageListenerConcurrently() {
            @Override
            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs,
                ConsumeConcurrentlyContext context) {
                for (MessageExt msg : msgs) {
                    System.out.println(new String(msg.getBody()) + " >> " + msg);
                }
                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            }
        });
        consumer.start();
        System.out.println("Consumer Started.");
    }
    catch (MQClientException e) {
        e.printStackTrace();
    }

结果：
此种场景下，consumer1与consumer2能正常消费到producer发送的所有消息，并遵循broker端的lb策略，consumer1与consumer2消费到的消息之和等于producer发送的消息数目

场景二：consumer1和consumer2配置相同的单个consumerGroup，多个topic和tags

1.producer发送消息 6条消息（各topic发送3条）

public static void main(String[] args)
        throws MQClientException, UnsupportedEncodingException, RemotingException, InterruptedException {
        DefaultMQProducer producer = new DefaultMQProducer("group");
        producer.setRetryTimesWhenSendAsyncFailed(3);
        producer.setNamesrvAddr("127.0.0.1:9876");
        producer.start();
        String[] tags = new String[] {"tag"};
        for (int i = 0; i < 3; i++) {
            Message msg = new Message("topic", tags[i % tags.length],
                ("Hello RocketMQ" + i).getBytes(RemotingHelper.DEFAULT_CHARSET));
            producer.send(msg, new SendCallback() {
                @Override
                public void onSuccess(SendResult sendResult) {


                    System.out.println(sendResult);
                }

                @Override
                public void onException(Throwable e) {


                    e.printStackTrace();
                }
            });
        }

        String[] tags2 = new String[] {"tag1"};
        for (int i = 0; i < 3; i++) {
            Message msg = new Message("topic1", tags2[i % tags2.length],
                ("Hello RocketMQ" + i).getBytes(RemotingHelper.DEFAULT_CHARSET));
            producer.send(msg, new SendCallback() {
                @Override
                public void onSuccess(SendResult sendResult) {

                    System.out.println(sendResult);
                }

                @Override
                public void onException(Throwable e)
                {
                    e.printStackTrace();
                }
            });
        }

    }

2.启动consumer1，consumer2 订阅关系不一致
Consumer1. “group” “topic1” “tag1”（groupName+topic+tag）

public static void main(String[] args) {
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("consumerGroup");
        consumer.setNamesrvAddr("localhost:9876");
        consumer.setMessageModel(MessageModel.CLUSTERING);
        try {
            consumer.subscribe("topic1","tag1");
            consumer.registerMessageListener(new MessageListenerConcurrently() {
                @Override
                public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs,
                    ConsumeConcurrentlyContext context) {
                    for (MessageExt msg : msgs) {
                        System.out.println(new String(msg.getBody()) + " >> " + msg);
                    }
                    return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
                }
            });
            consumer.start();
            System.out.println("Consumer Started.");
        }
        catch (MQClientException e) {
            e.printStackTrace();
        }
    }

Consumer2. “group” “topic” “tag”（groupName+topic+tag）

public static void main(String[] args) {
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("consumerGroup");
        consumer.setNamesrvAddr("localhost:9876");
        consumer.setMessageModel(MessageModel.CLUSTERING);
        try {
            consumer.subscribe("topic","tag");
            consumer.registerMessageListener(new MessageListenerConcurrently() {
                @Override
                public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs,
                    ConsumeConcurrentlyContext context) {
                    for (MessageExt msg : msgs) {
                        System.out.println(new String(msg.getBody()) + " >> " + msg);
                    }
                    return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
                }
            });
            consumer.start();
            System.out.println("Consumer Started.");
        }
        catch (MQClientException e) {
            e.printStackTrace();
        }
    }

结果：
此种场景下，若客户端初始化DefaultMQPushConsumer依据topic的数量进行初始化，则consumer1与consumer2与broker建立心跳链接时会相互覆盖相同consumerGroup下的消费topic信息，这样会导致部分消息消费不了
换句话就是：“同组下，DefaultMQPushConsumer根据topic的数量进行初始化，则consumer1与consumer2与broker建立心跳链接时会相互覆盖相同consumerGroup下的消费topic信息，这样会导致部分消息消费不了"
再换句话说：“同组，不同topic，consumer1和consumer2与broker建立心跳时相互覆盖同组下的消费topic信息，会导致部分消息消费不了”

场景三：
consumer1和consumer2配置不同的多个consumerGroup，相同的单个topic和tags

1.producer

public static void main(String[] args)
        throws MQClientException, UnsupportedEncodingException, RemotingException, InterruptedException {
        DefaultMQProducer producer = new DefaultMQProducer("group");
        producer.setRetryTimesWhenSendAsyncFailed(3);
        producer.setNamesrvAddr("127.0.0.1:9876");
        producer.start();
        String[] tags = new String[] {"tag"};
        for (int i = 0; i < 3; i++) {
            Message msg = new Message("topic", tags[i % tags.length],
                ("Hello RocketMQ" + i).getBytes(RemotingHelper.DEFAULT_CHARSET));
            producer.send(msg, new SendCallback() {
                @Override
                public void onSuccess(SendResult sendResult) {
                    System.out.println(sendResult);
                }
                @Override
                public void onException(Throwable e) {
                    e.printStackTrace();
                }
            });
        }
    }

2.consumer1

public static void main(String[] args) {
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("consumerGroup1");
        consumer.setNamesrvAddr("localhost:9876");
        consumer.setMessageModel(MessageModel.CLUSTERING);
        try {
            consumer.subscribe("topic","tag");
            consumer.registerMessageListener(new MessageListenerConcurrently() {
                @Override
                public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs,
                    ConsumeConcurrentlyContext context) {
                    for (MessageExt msg : msgs) {
                        System.out.println(new String(msg.getBody()) + " >> " + msg);
                    }
                    return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
                }
            });
            consumer.start();
            System.out.println("Consumer Started.");
        }
        catch (MQClientException e) {
            e.printStackTrace();
        }
    }

consumer2:

public static void main(String[] args) {
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("consumerGroup2");
        consumer.setNamesrvAddr("localhost:9876");
        consumer.setMessageModel(MessageModel.CLUSTERING);
        try {
            consumer.subscribe("topic","tag");
            consumer.registerMessageListener(new MessageListenerConcurrently() {
                @Override
                public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs,
                    ConsumeConcurrentlyContext context) {
                    for (MessageExt msg : msgs) {
                        System.out.println(new String(msg.getBody()) + " >> " + msg);
                    }
                    return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
                }
            });
            consumer.start();
            System.out.println("Consumer Started.");
        }
        catch (MQClientException e) {
            e.printStackTrace();
        }
    }

结果：
此种情况用于consumer1和consumer2需要同时消费所有producer发出的消息。consumer1和consumer2消费的消息之和等于producer生产的消息的2倍，所有发出的消息都会同时出现在consumer1和consumer2的消费队列中