RocketMQ进阶（二）

今天的博客主题

      MQ消息中间件 --》RocketMQ --》RocketMQ进阶（二）

---

本文主要讲解RocketMQ消费者实现及底层实现原理刨根问底

 

同样和生产者一样，对于使用普通Java项目来实现的例子就不多说了，官方文档都有

样例地址：https://github.com/apache/rocketmq/blob/master/docs/cn/RocketMQ_Example.md

主要看下springboot实现消费者。

 

springboot集成rocketmq消费者实现

pom.xml文件引入MQ依赖jar包

<!-- rocketmq依赖 -->
<dependency>
    <groupId>org.apache.rocketmq</groupId>
    <artifactId>rocketmq-spring-boot-starter</artifactId>
    <version>2.0.3</version>
</dependency>

application.yml配置文件添加配置

rocketmq: 
    #服务地址
    name-server: 10.10.16.20:9876
      #消费者配置
      consumer:
         #消费组名
         group: CG_DEMO
         #主题
         topic: TOPIC_DEMO_ONE
        #标签
        tag: TAG_ONE

具体代码实现

@Service
@RocketMQMessageListener(consumerGroup = "${rocketmq.consumer.group}", topic = "${rocketmq.consumer.topic}")
public class MsgListener implements RocketMQListener<MessageExt> {

    private org.slf4j.Logger logger = LoggerFactory.getLogger(getClass());

    @Override
    public void onMessage(MessageExt msg) {
        logger.debug("RECEIVE_MSG_BEGIN: " + msg.toString());
       logger.debug(String.format("消费消息，消息ID：%s，消息KEY：%s，消息体：%s ", msg.getMsgId(), msg.getKeys(), new String(msg.getBody())));
    }
}

测试

直接启动服务即可，会自动监听MQ服务，监听到消息就消费。

启动完就看到控制台输出：

2019-07-10 10:22:16.633  INFO 9504 --- [MessageThread_1] com.mq.listener.MsgListener              : RECEIVE_MSG_BEGIN: MessageExt [queueId=2, storeSize=282, queueOffset=0, sysFlag=0, bornTimestamp=1562577172354, bornHost=/10.10.51.219:57247, storeTimestamp=1562576637320, storeHost=/10.10.16.20:10911, msgId=0A0A101400002A9F000000000006982F, commitLogOffset=432175, bodyCRC=1703141525, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message{topic='TOPIC_DEMO_ONE', flag=0, properties={MIN_OFFSET=0, MAX_OFFSET=1, KEYS=123456, CONSUME_START_TIME=1562725336633, id=6b3efbac-03e4-0ced-4a9d-a722be0f775c, UNIQ_KEY=0A0A33DB31DC18B4AAC227BE22DD0000, WAIT=false, TAGS=TAG_ONE, timestamp=1562577172027}, body=[72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100, 32, 50, 48, 49, 57, 45, 48, 55, 45, 48, 56, 84, 49, 55, 58, 49, 50, 58, 53, 50, 46, 48, 50, 50], transactionId='null'}]
2019-07-10 10:22:16.633  INFO 9504 --- [MessageThread_1] com.mq.listener.MsgListener              : 消费消息，消息ID：0A0A33DB31DC18B4AAC227BE22DD0000，消息KEY：123456，消息体：Hello World 2019-07-08T17:12:52.022 

到这里就在springboot中集成了rocketmq消费者并成功消费了生产者发送的消息。

下面开始对消费者消费消息进行刨根问底。

 

@RocketMQMessageListener 注解

在这个注解类里面，只有一些字符串类型的方法，主要来管控一些参数。

看一下这个注解是怎么初始化它的。看下被引用的地方，因为是自动装配的，我们直接找 Configuration 的哪个

ListenerContainerConfiguration 类，会在启动时候被spring扫描到

@Configuration
public class ListenerContainerConfiguration implements ApplicationContextAware, SmartInitializingSingleton {
    // 实现了ApplicationContextAware 应用程序上下文感知  SmartInitializingSingleton  初始化单例
    ...省略部分代码
    
    // 因为是实现必须要重写接口方法，直接从这个两个方法作为入口
    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        this.applicationContext = (ConfigurableApplicationContext) applicationContext;
    }

    @Override
    public void afterSingletonsInstantiated() {
        // 获取带有 RocketMQMessageListener 的bean
        Map<String, Object> beans = this.applicationContext.getBeansWithAnnotation(RocketMQMessageListener.class);
    
        if (Objects.nonNull(beans)) {
            beans.forEach(this::registerContainer);
        }
    }
    
    // 初始化单利，将监听器注册到容器
    private void registerContainer(String beanName, Object bean) {
        Class<?> clazz = AopProxyUtils.ultimateTargetClass(bean);
        
        // 是否可分配的实例
        if (!RocketMQListener.class.isAssignableFrom(bean.getClass())) {
            throw new IllegalStateException(clazz + " is not instance of " + RocketMQListener.class.getName());
        }
        // 获得消息监听器，进行验证
        RocketMQMessageListener annotation = clazz.getAnnotation(RocketMQMessageListener.class);
        validate(annotation);
        // 容器bean名称
        String containerBeanName = String.format("%s_%s", DefaultRocketMQListenerContainer.class.getName(),
            counter.incrementAndGet());
        GenericApplicationContext genericApplicationContext = (GenericApplicationContext) applicationContext;
       // 将bean注册到通用上下文应用中
        genericApplicationContext.registerBean(containerBeanName, DefaultRocketMQListenerContainer.class,
            () -> createRocketMQListenerContainer(containerBeanName, bean, annotation));
        // 从这个上下文中在获的这个bean?
        DefaultRocketMQListenerContainer container = genericApplicationContext.getBean(containerBeanName,
            DefaultRocketMQListenerContainer.class);
        if (!container.isRunning()) {
            try {
                // 调用这个容器里的开始方法，就是启动consumer
                container.start();
            } catch (Exception e) {
                log.error("Started container failed. {}", container, e);
                throw new RuntimeException(e);
            }
        }
        log.info("Register the listener to container, listenerBeanName:{}, containerBeanName:{}", beanName, containerBeanName);
    }
}

通过 DefaultRocketMQListenerContainer 里的 initRocketMQPushConsumer 来初始化 DefaultMQPushConsumer

public class DefaultRocketMQListenerContainer implements InitializingBean, RocketMQListenerContainer, SmartLifecycle, ApplicationContextAware {
    省略部分代码...
    private void initRocketMQPushConsumer() throws MQClientException {
        Assert.notNull(rocketMQListener, "Property 'rocketMQListener' is required");
        Assert.notNull(consumerGroup, "Property 'consumerGroup' is required");
        Assert.notNull(nameServer, "Property 'nameServer' is required");
        Assert.notNull(topic, "Property 'topic' is required");
        // 权限钩子
        RPCHook rpcHook = RocketMQUtil.getRPCHookByAkSk(applicationContext.getEnvironment(),
            this.rocketMQMessageListener.accessKey(), this.rocketMQMessageListener.secretKey());
        boolean enableMsgTrace = rocketMQMessageListener.enableMsgTrace();
        // 实例化DefaultMQPushConsumer
        if (Objects.nonNull(rpcHook)) {
            consumer = new DefaultMQPushConsumer(consumerGroup, rpcHook, new AllocateMessageQueueAveragely(),
                enableMsgTrace, this.applicationContext.getEnvironment().
                resolveRequiredPlaceholders(this.rocketMQMessageListener.customizedTraceTopic()));
            consumer.setVipChannelEnabled(false);
            consumer.setInstanceName(RocketMQUtil.getInstanceName(rpcHook, consumerGroup));
        } else {
            log.debug("Access-key or secret-key not configure in " + this + ".");
            consumer = new DefaultMQPushConsumer(consumerGroup, enableMsgTrace,
                    this.applicationContext.getEnvironment().
                    resolveRequiredPlaceholders(this.rocketMQMessageListener.customizedTraceTopic()));
        }
        // 为DefaultMQPushConsumer设置上下文参数
        String customizedNameServer = this.applicationContext.getEnvironment().resolveRequiredPlaceholders(this.rocketMQMessageListener.nameServer());
        if (customizedNameServer != null) {
            consumer.setNamesrvAddr(customizedNameServer);
        } else {
            consumer.setNamesrvAddr(nameServer);
        }
        if (accessChannel != null) {
            consumer.setAccessChannel(accessChannel);
        }
        consumer.setConsumeThreadMax(consumeThreadMax);
        if (consumeThreadMax < consumer.getConsumeThreadMin()) {
            consumer.setConsumeThreadMin(consumeThreadMax);
        }
        consumer.setConsumeTimeout(consumeTimeout);
        consumer.setInstanceName(this.name);
        // 消费方式
        switch (messageModel) {
            case BROADCASTING:
                consumer.setMessageModel(org.apache.rocketmq.common.protocol.heartbeat.MessageModel.BROADCASTING);
                break;
            case CLUSTERING:
                consumer.setMessageModel(org.apache.rocketmq.common.protocol.heartbeat.MessageModel.CLUSTERING);
                break;
            default:
                throw new IllegalArgumentException("Property 'messageModel' was wrong.");
        }
        // 过滤方式
        switch (selectorType) {
            case TAG:
                consumer.subscribe(topic, selectorExpression);
                break;
            case SQL92:
                consumer.subscribe(topic, MessageSelector.bySql(selectorExpression));
                break;
            default:
                throw new IllegalArgumentException("Property 'selectorType' was wrong.");
        }
        // 通过不同消费姿势，来注册不同的监听器
        switch (consumeMode) {
            case ORDERLY:
                consumer.setMessageListener(new DefaultMessageListenerOrderly());
                break;
            case CONCURRENTLY:
                consumer.setMessageListener(new DefaultMessageListenerConcurrently());
                break;
            default:
                throw new IllegalArgumentException("Property 'consumeMode' was wrong.");
        }
    
        if (rocketMQListener instanceof RocketMQPushConsumerLifecycleListener) {
            ((RocketMQPushConsumerLifecycleListener) rocketMQListener).prepareStart(consumer);
        }
    }
}

DefaultMQPushConsumer

/**
 * 1.推荐使用的消息消费方式
 * 2.从技术上来说，这个push实现底层实际上是对pull方式的一个封装。从服务器定时拉取数据，来回调监听程序。
 * 3.它是线程安全的
 */
public class DefaultMQPus