RocketMQ同一Topic、消费组创建多个消费者失败问题

最新推荐文章于 2024-07-26 15:52:44 发布

昔我往矣丶

最新推荐文章于 2024-07-26 15:52:44 发布

阅读量8.9k

点赞数 4

分类专栏：消息队列

本文链接：https://blog.csdn.net/qq_43128724/article/details/117324847

版权

消息队列专栏收录该内容

3 篇文章 0 订阅

订阅专栏

文章目录

业务场景

rocketmq建议一个服务对应一个topic，但是一个服务下会有多个不同的业务消息，同时rocketmq建议不同的业务消息对应不同的tag，当SpringBoot整合RocketMQ时，设置多个消费者发生报错

问题复现

RocketMQ创建多个消费者(同一个消费组)消费同一Topic的不同tag的消息发生报错

2021-05-27 11:10:12.862 ERROR 7636 --- [           main] o.a.r.s.a.ListenerContainerConfiguration : Started container failed. DefaultRocketMQListenerContainer{consumerGroup='MyConsumerGroup', nameServer='10.90.175.160:9876', topic='TestTopic', consumeMode=CONCURRENTLY, selectorType=TAG, selectorExpression='tag0', messageModel=CLUSTERING}

java.lang.IllegalStateException: Failed to start RocketMQ push consumer
	at org.apache.rocketmq.spring.support.DefaultRocketMQListenerContainer.start(DefaultRocketMQListenerContainer.java:281) ~[rocketmq-spring-boot-2.1.1.jar:2.1.1]
	at org.apache.rocketmq.spring.autoconfigure.ListenerContainerConfiguration.registerContainer(ListenerContainerConfiguration.java:120) ~[rocketmq-spring-boot-2.1.1.jar:2.1.1]
	at java.util.HashMap.forEach(HashMap.java:1289) ~[na:1.8.0_222]
	at org.apache.rocketmq.spring.autoconfigure.ListenerContainerConfiguration.afterSingletonsInstantiated(ListenerContainerConfiguration.java:79) ~[rocketmq-spring-boot-2.1.1.jar:2.1.1]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:862) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:865) ~[spring-context-5.1.0.RELEASE.jar:5.1.0.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:548) ~[spring-context-5.1.0.RELEASE.jar:5.1.0.RELEASE]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:140) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:742) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:389) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:311) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1213) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1202) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
	at com.roy.rocketmq.RocketMQSBApplication.main(RocketMQSBApplication.java:16) ~[classes/:na]
Caused by: org.apache.rocketmq.client.exception.MQClientException: The consumer group[MyConsumerGroup] has been created before, specify another name please.
See http://rocketmq.apache.org/docs/faq/ for further details.
	at org.apache.rocketmq.client.impl.consumer.DefaultMQPushConsumerImpl.start(DefaultMQPushConsumerImpl.java:634) ~[rocketmq-client-4.7.1.jar:4.7.1]
	at org.apache.rocketmq.client.consumer.DefaultMQPushConsumer.start(DefaultMQPushConsumer.java:698) ~[rocketmq-client-4.7.1.jar:4.7.1]
	at org.apache.rocketmq.spring.support.DefaultRocketMQListenerContainer.start(DefaultRocketMQListenerContainer.java:279) ~[rocketmq-spring-boot-2.1.1.jar:2.1.1]
	... 13 common frames omitted

2021-05-27 11:10:12.866  INFO 7636 --- [           main] o.apache.catalina.core.StandardService   : Stopping service [Tomcat]
2021-05-27 11:10:12.881  INFO 7636 --- [           main] ConditionEvaluationReportLoggingListener : 

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2021-05-27 11:10:12.892 ERROR 7636 --- [           main] o.s.boot.SpringApplication               : Application run failed

java.lang.RuntimeException: java.lang.IllegalStateException: Failed to start RocketMQ push consumer
	at org.apache.rocketmq.spring.autoconfigure.ListenerContainerConfiguration.registerContainer(ListenerContainerConfiguration.java:123) ~[rocketmq-spring-boot-2.1.1.jar:2.1.1]
	at java.util.HashMap.forEach(HashMap.java:1289) ~[na:1.8.0_222]
	at org.apache.rocketmq.spring.autoconfigure.ListenerContainerConfiguration.afterSingletonsInstantiated(ListenerContainerConfiguration.java:79) ~[rocketmq-spring-boot-2.1.1.jar:2.1.1]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:862) ~[spring-beans-5.1.8.RELEASE.jar:5.1.8.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:865) ~[spring-context-5.1.0.RELEASE.jar:5.1.0.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:548) ~[spring-context-5.1.0.RELEASE.jar:5.1.0.RELEASE]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:140) ~[spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:742) [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:389) [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:311) [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1213) [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1202) [spring-boot-2.1.6.RELEASE.jar:2.1.6.RELEASE]
	at com.roy.rocketmq.RocketMQSBApplication.main(RocketMQSBApplication.java:16) [classes/:na]
Caused by: java.lang.IllegalStateException: Failed to start RocketMQ push consumer
	at org.apache.rocketmq.spring.support.DefaultRocketMQListenerContainer.start(DefaultRocketMQListenerContainer.java:281) ~[rocketmq-spring-boot-2.1.1.jar:2.1.1]
	at org.apache.rocketmq.spring.autoconfigure.ListenerContainerConfiguration.registerContainer(ListenerContainerConfiguration.java:120) ~[rocketmq-spring-boot-2.1.1.jar:2.1.1]
	... 12 common frames omitted
Caused by: org.apache.rocketmq.client.exception.MQClientException: The consumer group[MyConsumerGroup] has been created before, specify another name please.
See http://rocketmq.apache.org/docs/faq/ for further details.
	at org.apache.rocketmq.client.impl.consumer.DefaultMQPushConsumerImpl.start(DefaultMQPushConsumerImpl.java:634) ~[rocketmq-client-4.7.1.jar:4.7.1]
	at org.apache.rocketmq.client.consumer.DefaultMQPushConsumer.start(DefaultMQPushConsumer.java:698) ~[rocketmq-client-4.7.1.jar:4.7.1]
	at org.apache.rocketmq.spring.support.DefaultRocketMQListenerContainer.start(DefaultRocketMQListenerContainer.java:279) ~[rocketmq-spring-boot-2.1.1.jar:2.1.1]
	... 13 common frames omitted

解决方式

实现RocketMQPushConsumerLifecycleListener接口，重写prepareStart，consumer.setInstanceName(“testTopic-tag0”);设置唯一标识的instanceName即可

@Component
@RocketMQMessageListener(consumerGroup = "MyConsumerGroup",
        topic = "TestTopic",
        consumeMode= ConsumeMode.CONCURRENTLY,
        selectorType = SelectorType.TAG,
        selectorExpression = "tag0")
public class Tag0Consumer implements RocketMQListener<String>, RocketMQPushConsumerLifecycleListener  {
    @Override
    public void onMessage(String message) {
        System.out.println("Received tag0 message : "+ message);
    }


    @Override
    public void prepareStart(DefaultMQPushConsumer consumer) {
        //consumer.setInstanceName("testTopic-tag0");
    }
}

@Component
@RocketMQMessageListener(consumerGroup = "MyConsumerGroup",
        topic = "TestTopic",
        consumeMode= ConsumeMode.CONCURRENTLY,
        selectorType = SelectorType.TAG,
        selectorExpression = "tag1")
public class Tag0Consumer implements RocketMQListener<String>, RocketMQPushConsumerLifecycleListener  {
    @Override
    public void onMessage(String message) {
        System.out.println("Received tag1 message : "+ message);
    }


    @Override
    public void prepareStart(DefaultMQPushConsumer consumer) {
        consumer.setInstanceName("testTopic-tag1");
    }
}

问题跟踪

发现如果不给consumer指定instanceName，rocketmq就会给此consumer设置一个默认的instanceName，如果有多个消费者都在同一个消费组里，并且不指定instanceName，通过this.consumerTable.putIfAbsent(group, consumer)拿到的消费者对象就会是同一个，就会报错

log.warn(“the consumer group[” + group + “] exist already.”);

throw new MQClientException(“The consumer group[” + this.defaultMQPushConsumer.getConsumerGroup()+ “] has been created before, specify another name please.” + FAQUrl.suggestTodo(FAQUrl.GROUP_NAME_DUPLICATE_URL), null);

代码片段

DefaultMQPushConsumerImpl类

boolean registerOK = mQClientFactory.registerConsumer(this.defaultMQPushConsumer.getConsumerGroup(), this);
if (!registerOK) {
    this.serviceState = ServiceState.CREATE_JUST;
    this.consumeMessageService.shutdown(defaultMQPushConsumer.getAwaitTerminationMillisWhenShutdown());
    throw new MQClientException("The consumer group[" + this.defaultMQPushConsumer.getConsumerGroup()
                                + "] has been created before, specify another name please." + FAQUrl.suggestTodo(FAQUrl.GROUP_NAME_DUPLICATE_URL),
                                null);
}

MQClientInstance类

public boolean registerConsumer(final String group, final MQConsumerInner consumer) {
    if (null == group || null == consumer) {
    return false;
    }

    MQConsumerInner prev = this.consumerTable.putIfAbsent(group, consumer);
    if (prev != null) {
    log.warn("the consumer group[" + group + "] exist already.");
    return false;
    }

    return true;
}