【个人笔记】RocketMQ-cpp-client源码解读（持续更新）

充分小足够大

已于 2024-02-03 22:16:40 修改

阅读量938

点赞数 19

文章标签： java-rocketmq 笔记 rocketmq

于 2024-02-03 22:15:14 首次发布

本文链接：https://blog.csdn.net/SYN_ACK/article/details/136016261

版权

0.背景

项目需要使用RocketMQ-cpp-client触发Prometheus告警。因无法搜索到相关的C++SDK的具体解释文档，因此通过阅读源码进行学习。边学边干。

SDK版本：rocketmq-client-cpp-release-2.2.0

1.示例解读：PullConsumer.cpp代码解读

PullConsumer.cpp是RocketMQ C++ SDK中提供的Pull类型Consumer的示例代码，接下来将解读其中比较重要的代码。

  1 /*
  2  * Licensed to the Apache Software Foundation (ASF) under one or more
  3  * contributor license agreements.  See the NOTICE file distributed with
  4  * this work for additional information regarding copyright ownership.
  5  * The ASF licenses this file to You under the Apache License, Version 2.0
  6  * (the "License"); you may not use this file except in compliance with
  7  * the License.  You may obtain a copy of the License at
  8  *
  9  *     http://www.apache.org/licenses/LICENSE-2.0
 10  *
 11  * Unless required by applicable law or agreed to in writing, software
 12  * distributed under the License is distributed on an "AS IS" BASIS,
 13  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 14  * See the License for the specific language governing permissions and
 15  * limitations under the License.
 16  */
 17 
 18 #include <stdlib.h>
 19 
 20 #include <chrono>
 21 #include <iomanip>
 22 #include <iostream>
 23 #include <map>
 24 #include <vector>
 25 
 26 #include "common.h"
 27
 28 using namespace rocketmq;
 29 
 30 std::map<MQMessageQueue, uint64_t> g_offseTable;

第一行到第三行，是一系列的头文件，其中“common.h”不是SDK中的一部分，是示例代码中的一部分，其中include了一些其他RocketMQ SDK的头文件。具体内容可参见源码文件。

第30行的g_offsetTable是map类型的数据，将MQMessageQueue类和一个uint64_t类型的值建立映射关系，这一对映射关系将在后来的代码中用到，uint64_t将代表MQMessageQueue Offset从哪里开始。

MQMessageQueue是SDK中定义的一个类，可以在<MQMessageQueue.h>文件中找到该类的定义。这个类中包含了一系列的函数和变量，用来指明一个RockeMQ队列的基本信息。例如topic,brokername,queueID。

 31 
 32 void putMessageQueueOffset(MQMessageQueue mq, uint64_t offset) {
 33   g_offseTable[mq] = offset;
 34 }
 35 
 36 uint64_t getMessageQueueOffset(MQMessageQueue mq) {
 37   map<MQMessageQueue, uint64_t>::iterator it = g_offseTable.find(mq);
 38   if (it != g_offseTable.end()) {
 39     return it->second;
 40   }
 41   return 0;
 42 }

PullConsumer.cpp文件31行到42行定义了两个函数。

putMessageQueueOffset函数将MQMessageQueue类的实例和偏移量（offse）初始值建立映射关系。

getMessageQueueOffset函数则是获取MQMessageQueue类的实例的偏移量的初始值。需要设置和获取偏移量初始值的原因是在之后进行Pull操作时，需要这两个值作为相关API的参数。


 43 
 44 int main(int argc, char* argv[]) {
 45   RocketmqSendAndConsumerArgs info;
 46   if (!ParseArgs(argc, argv, &info)) {
 47     exit(-1);
 48   }
 49   PrintRocketmqSendAndConsumerArgs(info);
 50   DefaultMQPullConsumer consumer("please_rename_unique_group_name");
 51   consumer.setNamesrvAddr(info.namesrv);
 52   consumer.setNamesrvDomain(info.namesrv_domain);
 53   consumer.setGroupName(info.groupname);
 54   consumer.setInstanceName(info.groupname);
 55   consumer.registerMessageQueueListener(info.topic, NULL);
 56   consumer.start();

第44行开始进入main函数。

45到49行是对用户传参进行处理。暂不做解释。

第50行是创建了一个DefaultMQPullConsumer类的实例。该类定义在<DefaultPullConsumer.h>文件中，其中包括consumer相关的方法和变量。

第51行到第55行是设置consumer的相关参数，例如name server（第51行），消费者组名称（第53行）。完成相关参数设置后，consumer.start()运行consumer。

 57   std::vector<MQMessageQueue> mqs;
 58 
 59   try {
 60     consumer.fetchSubscribeMessageQueues(info.topic, mqs);
 61     auto iter = mqs.begin();
 62     for (; iter != mqs.end(); ++iter) {
 63       std::cout << "mq:" << (*iter).toString() << endl;
 64     }
 65   } catch (MQException& e) {
 66     std::cout << e << endl;
 67   }
 68

第57行到67行是获取一个topic所有相关队列。fetchSubScribeMessageQueues返回一个存储MQMessageQueue类型量的vector。这一段代码的作用是获取info.topic关联的所有RocketMQ队列的信息。最终输出样式如下图。

 69   auto start = std::chrono::system_clock::now();
 70   auto iter = mqs.begin();
 71   for (; iter != mqs.end(); ++iter) {
 72     MQMessageQueue mq = (*iter);
 73     // if cluster model
 74     // putMessageQueueOffset(mq, g_consumer.fetchConsumeOffset(mq,true));
 75     // if broadcast model
 76     // putMessageQueueOffset(mq, your last consume offset);
 77 
 78     bool noNewMsg = false;
 79     do {
 80       try {
 81         PullResult result = consumer.pull(mq, "*", getMessageQueueOffset(mq), 32);
 82         g_msgCount += result.msgFoundList.size();
 83         std::cout << result.msgFoundList.size() << std::endl;
 84         // if pull request timeout or received NULL response, pullStatus will be
 85         // setted to BROKER_TIMEOUT,
 86         // And nextBeginOffset/minOffset/MaxOffset will be setted to 0
 87         if (result.pullStatus != BROKER_TIMEOUT) {
 88           putMessageQueueOffset(mq, result.nextBeginOffset);
 89           PrintPullResult(&result);
 90         } else {
 91           cout << "broker timeout occur" << endl;
 92         }
 93         switch (result.pullStatus) {
 94           case FOUND:
 95           case NO_MATCHED_MSG:
 96           case OFFSET_ILLEGAL:
 97           case BROKER_TIMEOUT:
 98             break;
 99           case NO_NEW_MSG:
100             noNewMsg = true;
101             break;
102           default:
103             break;
104         }
105       } catch (MQClientException& e) {
106         std::cout << e << std::endl;
107       }
108     } while (!noNewMsg);
109   }

从第71行开始的循环遍历刚刚获取到的所有MQMessageQueue,并拉取每个Queue中的消息（第81行），并且返回一个PullResult类的实例。PullResult类定义在<PullResult.h>中，该类包含了pull函数获得的RocketMQ的返回信息。例如，pullStatus-返回状态，nextBeginOffset-下一个offset值，minOffset-最小位点，maxOffset-最大位点，msgFoundList-一个元素类型为MQMessageExt的vector。

MQMessageExt是真正包含拉取到的Message的信息的类。它被定义在<MQMessageExt.h>头文件中。包含了获取到的Message的相关信息。如下图代码：

  1 /*
  2  * Licensed to the Apache Software Foundation (ASF) under one or more
  3  * contributor license agreements.  See the NOTICE file distributed with
  4  * this work for additional information regarding copyright ownership.
  5  * The ASF licenses this file to You under the Apache License, Version 2.0
  6  * (the "License"); you may not use this file except in compliance with
  7  * the License.  You may obtain a copy of the License at
  8  *
  9  *     http://www.apache.org/licenses/LICENSE-2.0
 10  *
 11  * Unless required by applicable law or agreed to in writing, software
 12  * distributed under the License is distributed on an "AS IS" BASIS,
 13  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 14  * See the License for the specific language governing permissions and
 15  * limitations under the License.
 16  */
 17 #ifndef __MESSAGEEXT_H__
 18 #define __MESSAGEEXT_H__
 19 
 20 #ifdef WIN32
 21 #include <Windows.h>
 22 #include <Winsock2.h>
 23 #else
 24 #include <sys/socket.h>
 25 #endif
 26 
 27 #include "MQMessage.h"
 28 #include "RocketMQClient.h"
 29 
 30 namespace rocketmq {
 31 //<!message extend class, which was generated on broker;
 32 class ROCKETMQCLIENT_API MQMessageExt : public MQMessage {
 33  public:
 34   MQMessageExt();
 35   MQMessageExt(int queueId,
 36                int64 bornTimestamp,
 37                sockaddr bornHost,
 38                int64 storeTimestamp,
 39                sockaddr storeHost,
 40                std::string msgId);
 41 
 42   virtual ~MQMessageExt();
 43 
 44   static int parseTopicFilterType(int sysFlag);
 45 
 46   int getQueueId() const;
 47   void setQueueId(int queueId);
 48 
 49   int64 getBornTimestamp() const;
 50   void setBornTimestamp(int64 bornTimestamp);
 51 
 52   sockaddr getBornHost() const;
 53   std::string getBornHostString() const;
 54   std::string getBornHostNameString() const;
 55   void setBornHost(const sockaddr& bornHost);
 56 
 57   int64 getStoreTimestamp() const;
 58   void setStoreTimestamp(int64 storeTimestamp);
 59 
 60   sockaddr getStoreHost() const;
 61   std::string getStoreHostString() const;
 62   void setStoreHost(const sockaddr& storeHost);
 63 
 64   const std::string& getMsgId() const;
 65   void setMsgId(const std::string& msgId);
 66 
 67   const std::string& getOffsetMsgId() const;
 68   void setOffsetMsgId(const std::string& offsetMsgId);
 69 
 70   int getBodyCRC() const;
 71   void setBodyCRC(int bodyCRC);
 72 
 73   int64 getQueueOffset() const;
 74   void setQueueOffset(int64 queueOffset);
 75 
 76   int64 getCommitLogOffset() const;
 77   void setCommitLogOffset(int64 physicOffset);
 78 
 79   int getStoreSize() const;
 80   void setStoreSize(int storeSize);
 81 
 82   int getReconsumeTimes() const;
 83   void setReconsumeTimes(int reconsumeTimes);
 84 
 85   int64 getPreparedTransactionOffset() const;
 86   void setPreparedTransactionOffset(int64 preparedTransactionOffset);
 87 
 88   std::string toString() const {
 89     std::stringstream ss;
 90     ss << "MessageExt [queueId=" << m_queueId << ", storeSize=" << m_storeSize << ", queueOffset=" << m_queueOffset
 91        << ", sysFlag=" << m_sysFlag << ", bornTimestamp=" << m_bornTimestamp << ", bornHost=" << getBornHostString()
 92        << ", storeTimestamp=" << m_storeTimestamp << ", storeHost=" << getStoreHostString() << ", msgId=" << m_msgId
 93        << ", commitLogOffset=" << m_commitLogOffset << ", bodyCRC=" << m_bodyCRC
 94        << ", reconsumeTimes=" << m_reconsumeTimes << ", preparedTransactionOffset=" << m_preparedTransactionOffset
 95        << ",  " << MQMessage::toString() << "]";
 96     return ss.str();
 97   }
 98 
 99  private:
100   int64 m_queueOffset;
101   int64 m_commitLogOffset;
102   int64 m_bornTimestamp;
103   int64 m_storeTimestamp;
104   int64 m_preparedTransactionOffset;
105   int m_queueId;
106   int m_storeSize;
107   int m_bodyCRC;
108   int m_reconsumeTimes;
109   sockaddr m_bornHost;
110   sockaddr m_storeHost;
111   std::string m_msgId;
112   std::string m_offsetMsgId;
113 };
114 }  // namespace rocketmq
115 #endif

第111行到120行是代码的最后一部分，显示代码已经执行完成，并且配合第69行使用的system_clock来计算代码最终执行时间。

111   auto end = std::chrono::system_clock::now();
112   auto duration = std::chrono::duration_cast<std::chrono::milliseconds>(end - start);
113 
114   std::cout << "msg count: " << g_msgCount.load() << "\n";
115   std::cout << "per msg time: " << duration.count() / (double)g_msgCount.load() << "ms \n"
116             << "========================finished==============================\n";
117 
118   consumer.shutdown();
119   return 0;
120 }

以上便是PullConsumer.cpp中比较关键的地方的解读，想要了解更多，快去下载源码观摩学习吧~。欢迎各位帅哥美女评论区交流。

充分小足够大

关注

19
点赞
踩
12

收藏

觉得还不错? 一键收藏
1
评论
【个人笔记】RocketMQ-cpp-client源码解读（持续更新）

这一段代码的作用是获取info.topic关联的所有RocketMQ队列的信息。最终输出样式如下图。第30行的g_offsetTable是map类型的数据，将MQMessageQueue类和一个uint64_t类型的值建立映射关系，这一对映射关系将在后来的代码中用到，uint64_t将代表MQMessageQueue Offset从哪里开始。第一行到第三行，是一系列的头文件，其中“common.h”不是SDK中的一部分，是示例代码中的一部分，其中include了一些其他RocketMQ SDK的头文件。
复制链接

扫一扫