一、源码环境搭建
1.源码拉取
RocketMQ的官网源码地址：http://rocketmq.apache.org/dowloading/releases/
RocketMQ的官方Git仓库地址：https://github.com/apache/rocketmq/
本文以4.7.1版本为例进行分析，下载完成后，需要对其进行编译：
mvn clean install -Dmaven.test.skip=true
代码结构如下：
2.源码启动
2.0 修改配置文件
先在项目目录下创建一个conf目录，并从 distribution 拷贝 broker.conf 和 logback_broker.xml 和 logback_namesrv.xml
修改broker.conf
brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = 0
deleteWhen = 04
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
# 自动创建Topic
autoCreateTopicEnable=true
# nameServ地址
namesrvAddr=127.0.0.1:9876
# 存储路径
storePathRootDir=/..../data/rocketmq/dataDir
# commitLog路径
storePathCommitLog=/..../data/rocketmq/dataDir/commitlog
# 消息队列存储路径
storePathConsumeQueue=/..../data/rocketmq/dataDir/consumequeue
# 消息索引存储路径
storePathIndex=/..../data/rocketmq/dataDir/index
# checkpoint文件路径
storeCheckpoint=/..../data/rocketmq/dataDir/checkpoint
# abort文件存储路径
abortFile=/..../data/rocketmq/dataDir/abort
2.1 启动nameServer
namesrv模块中，运行NamesrvStartup类即可启动
启动时需要配置一个ROCKETMQ_HOME环境变量，这个环境变量我们可以在主机上配置，也可以在IDEA的运行环境中配置。
启动成功后，会有如下提示：
The Name Server boot success. serializeType=JSON
2.2 启动Broker
broker模块中，运行BrokerStartup类即可启动
启动Broker时，同样需要ROCETMQ_HOME环境变量，并且还需要配置一个-c 参数，指向broker.conf配置文件。
启动成功后，会有如下提示：
The broker[broker-a, 192.168.2.16:10911] boot success. serializeType=JSON and name server is 127.0.0.1:9876
2.3启动Consumer
启动example模块下的的org.apache.rocketmq.example.quickstart.Consumer类，需要指定NameServer地址，一种是配置一个NAMESRV_ADDR的环境变量，另一种是在源码中指定
consumer.setNamesrvAddr("127.0.0.1:9876");
2.4启动Producer
启动example模块下的org.apache.rocketmq.example.quickstart.Producer类，同样需要指定NameServer地址。启动成功后就可以发送消息，消费者端会接收到消息。
至此，源码环境基本搭建完成。
二、启动NameServer
主要包括两个功能：
- Broker管理
- 路由信息管理
从NameServer启动类NamesrvStartup入手：
public static void main(String[] args) {
main0(args);
}
public static NamesrvController main0(String[] args) {
try {
//NamesrvController是NameServer启动的核心组件
NamesrvController controller = createNamesrvController(args);
start(controller);
...... //省略非重点代码
}
public static NamesrvController createNamesrvController(String[] args) throws IOException, JoranException {
//检测命令行参数
......
//NameServer核心的配置类：
//NameSrvConfig 包含NameServer自身运行的参数
//NettyServerConfig 包含Netty服务端的配置参数，Netty服务端口默认指定了9876，可修改端口
final NamesrvConfig namesrvConfig = new NamesrvConfig();
final NettyServerConfig nettyServerConfig = new NettyServerConfig();
nettyServerConfig.setListenPort(9876);
//解析配置对象 -c:指定配置文件; -p：指定每一个属性; ROCKETMQ_HOME环境变量检测; 日志相关配置
......
final NamesrvController controller = new NamesrvController(namesrvConfig, nettyServerConfig);
// remember all configs to prevent discard
controller.getConfiguration().registerConfig(properties);
return controller;
}
public static NamesrvController start(final NamesrvController controller) throws Exception {
......//省略非重点代码
//NameController初始化
boolean initResult = controller.initialize();
......//省略非重点代码
//NameController启动
controller.start();
return controller;
}
NamesrvController，是NameServer的核心管理控制组件：
//NameController初始化
public boolean initialize() {
//加载KV配置
this.kvConfigManager.load();
//初始化NettyRemotingServer，NettyServer的网络处理对象，处理Netty请求，9876端口
this.remotingServer = new NettyRemotingServer(this.nettyServerConfig, this.brokerHousekeepingService);
//NettyServer的工作线程池remotingExecutor，并注入到remotingServer中
......
//开启定时任务:每隔10s扫描一次Broker,移除不活跃的Broker
//routeInfoManager用来管理Broker列表
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
NamesrvController.this.routeInfoManager.scanNotActiveBroker();
}
}, 5, 10, TimeUnit.SECONDS);
//开启定时任务:每隔10min打印一次KV配置
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
NamesrvController.this.kvConfigManager.printAllPeriodically();
}
}, 1, 10, TimeUnit.MINUTES);
//tls安全传输层协议
......
}
public void start() throws Exception {
//启动remotingServer
this.remotingServer.start();
if (this.fileWatchService != null) {
this.fileWatchService.start();
}
}
总结流程图
三、启动Broker
Broker是整个RocketMQ的业务核心，负责存储消息、转发消息。
从Broker启动类BrokerStartup入手：
public static void main(String[] args) {
//创建并启动Broker核心组件BrokerController
start(createBrokerController(args));
}
public static BrokerController start(BrokerController controller) {
......
//Controller启动
controller.start();
......
}
public static BrokerController createBrokerController(String[] args) {
......
//Broker的核心配置信息,既做Netty服务端也做客户端（事务消息请求Producer）
final BrokerConfig brokerConfig = new BrokerConfig();
final NettyServerConfig nettyServerConfig = new NettyServerConfig();
final NettyClientConfig nettyClientConfig = new NettyClientConfig();
......
//Netty服务端的监听端口10911
nettyServerConfig.setListenPort(10911);
//这个是Broker用来存储消息的一些配置信息。
final MessageStoreConfig messageStoreConfig = new MessageStoreConfig();
//如果是SLAVE，会设置一个参数
if (BrokerRole.SLAVE == messageStoreConfig.getBrokerRole()) {
int ratio = messageStoreConfig.getAccessMessageInMemoryMaxRatio() - 10;
messageStoreConfig.setAccessMessageInMemoryMaxRatio(ratio);
}
......
//判断集群角色，通过BrokerId判断主从
switch (messageStoreConfig.getBrokerRole()) {
case ASYNC_MASTER:
case SYNC_MASTER:
brokerConfig.setBrokerId(MixAll.MASTER_ID);
break;
case SLAVE:
if (brokerConfig.getBrokerId() <= 0) {
System.out.printf("Slave's brokerId must be > 0");
System.exit(-3);
}
break;
default:
break;
}
//判断是否基于Dledger技术，如果是就把brokerId设置为-1
if (messageStoreConfig.isEnableDLegerCommitLog()) {
brokerConfig.setBrokerId(-1);
}
......
//创建BrokerController
final BrokerController controller = new BrokerController(
brokerConfig,
nettyServerConfig,
nettyClientConfig,
messageStoreConfig);
// remember all configs to prevent discard
controller.getConfiguration().registerConfig(properties);
//初始化BrokerController
boolean initResult = controller.initialize();
......
}
BrokerController，Broker的核心管理控制组件：
//Broker初始化过程 public boolean initialize() throws CloneNotSupportedException { ...... //消息存储管理组件，管理磁盘上的消息的。此处为消息存储相关入口 this.messageStore = new DefaultMessageStore(this.messageStoreConfig, this.brokerStatsManager, this.messageArrivingListener,this.brokerConfig); ....... //加载磁盘文件 result = result && this.messageStore.load(); if (result) { //Netty网络组件 this.remotingServer = new NettyRemotingServer(this.nettyServerConfig, this.clientHousekeepingService); NettyServerConfig fastConfig = (NettyServerConfig) this.nettyServerConfig.clone(); fastConfig.setListenPort(nettyServerConfig.getListenPort() - 2); this.fastRemotingServer = new NettyRemotingServer(fastConfig, this.clientHousekeepingService); //初始化一堆线程池 ...... //后台定时任务 ...... //设置NameServer的地址列表。可以配置加载，也可以发远程请求加载。 if (this.brokerConfig.getNamesrvAddr() != null) {this.brokerOuterAPI.updateNameServerAddressList(this.brokerConfig.getNamesrvAddr()); log.info("Set user specified name server address: {}", this.brokerConfig.getNamesrvAddr()); } else if (this.brokerConfig.isFetchNamesrvAddrByAddressServer()) { this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() { @Override public void run() { try { BrokerController.this.brokerOuterAPI.fetchNameServerAddr(); } catch (Throwable e) { log.error("ScheduledTask fetchNameServerAddr exception", e); } } }, 1000 * 10, 1000 * 60 * 2, TimeUnit.MILLISECONDS); } //开启Dledger后的一些操作 ...... //tls跟安全文件相关 ...... //初始化相关 initialTransaction(); initialAcl(); initialRpcHooks(); } return result; } //BrokerController核心的启动方法 public void start() throws Exception { //先启动一些功能核心组件 ...... //Broker核心的心跳注册任务。 this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() { @Override public void run() { try { //此处为Broker注册相关入口 BrokerController.this.registerBrokerAll(true, false, brokerConfig.isForceRegister()); } catch (Throwable e) { log.error("registerBrokerAll Exception", e); } } //这个任务间隔时间默认是30秒 }, 1000 * 10, Math.max(10000, Math.min(brokerConfig.getRegisterNameServerPeriod(), 60000)), TimeUnit.MILLISECONDS); //继续启动一些功能核心组件 ...... }
总结流程图
四、注册Broker
Broker会在启动时向NameServer注册自己的服务信息，BrokerController.this.registerBrokerAll方法会向NameServer注册心跳，也会启动一个线程池，以10秒延迟，默认30秒的间隔持续向NameServer发送心跳。
在NameServer中也会启动一个定时任务，每隔10s扫描一次Broker，移除不活跃的Broker.
从Broker的启动方法入手：
//BrokerController核心的启动方法 public void start() throws Exception { ...... //Broker核心的心跳注册任务。 this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() { @Override public void run() { try { //此处为Broker注册相关入口 BrokerController.this.registerBrokerAll(true, false, brokerConfig.isForceRegister()); } catch (Throwable e) { log.error("registerBrokerAll Exception", e); } } //这个任务间隔时间默认是30秒 }, 1000 * 10, Math.max(10000, Math.min(brokerConfig.getRegisterNameServerPeriod(), 60000)), TimeUnit.MILLISECONDS); ...... } public synchronized void registerBrokerAll(final boolean checkOrderConfig, boolean oneway, boolean forceRegister) { //Topic配置相关 ...... //关键，先判断是否需要注册，然后调用doRegisterBrokerAll方法真正去注册。 if (forceRegister || needRegister(this.brokerConfig.getBrokerClusterName(), this.getBrokerAddr(), this.brokerConfig.getBrokerName(), this.brokerConfig.getBrokerId(), this.brokerConfig.getRegisterBrokerTimeoutMills())) { doRegisterBrokerAll(checkOrderConfig, oneway, topicConfigWrapper); } } //Broker注册核心 private void doRegisterBrokerAll(boolean checkOrderConfig, boolean oneway, TopicConfigSerializeWrapper topicConfigWrapper) { //返回的结果是个List是因为Broker是向所有的NameServer进行注册。 List<RegisterBrokerResult> registerBrokerResultList = this.brokerOuterAPI.registerBrokerAll( this.brokerConfig.getBrokerClusterName(), this.getBrokerAddr(), this.brokerConfig.getBrokerName(), this.brokerConfig.getBrokerId(), this.getHAServerAddr(), topicConfigWrapper, this.filterServerManager.buildNewFilterServerList(), oneway, this.brokerConfig.getRegisterBrokerTimeoutMills(), this.brokerConfig.isCompressedRegister()); //如果注册结果的数量大于0，那么就找其中一个结果进行处理 ...... }
进入到brokerOuterAPI中：
public List<RegisterBrokerResult> registerBrokerAll( final String clusterName, final String brokerAddr, final String brokerName, final long brokerId, final String haServerAddr, final TopicConfigSerializeWrapper topicConfigWrapper, final List<String> filterServerList, final boolean oneway, final int timeoutMills, final boolean compressed) { //初始化一个list，用来放向每个NameServer注册的结果。 final List<RegisterBrokerResult> registerBrokerResultList = Lists.newArrayList(); //NameSrv的地址列表 List<String> nameServerAddressList = this.remotingClient.getNameServerAddressList(); ...... for (final String namesrvAddr : nameServerAddressList) { brokerOuterExecutor.execute(new Runnable() { @Override public void run() { try { //Broker真正执行注册的方法 RegisterBrokerResult result = registerBroker(namesrvAddr,oneway, timeoutMills,requestHeader,body); //注册完在本地保存 if (result != null) { registerBrokerResultList.add(result); } ...... return registerBrokerResultList; } private RegisterBrokerResult registerBroker( final String namesrvAddr, final boolean oneway, final int timeoutMills, final RegisterBrokerRequestHeader requestHeader, final byte[] body ) throws RemotingCommandException, MQBrokerException, RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException, InterruptedException { //封装网络请求 RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.REGISTER_BROKER, requestHeader); request.setBody(body); //特殊请求 sendOneWay if (oneway) { try { this.remotingClient.invokeOneway(namesrvAddr, request, timeoutMills); } catch (RemotingTooMuchRequestException e) { // Ignore } return null; } //真正发送网络请求的地方。这个remotingClient就是一个NettyClient。 RemotingCommand response = this.remotingClient.invokeSync(namesrvAddr, request, timeoutMills); //封装网络请求结果。 ...... }
总结流程图
五、Producer发送消息
Producer有两种：一种是普通发送者DefaultMQProducer；另一种是事务消息发送者TransactionMQProducer。
以DefaultMQProducer为例，主要有启动start、发送消息send
@Override public void start() throws MQClientException { this.setProducerGroup(withNamespace(this.producerGroup)); this.defaultMQProducerImpl.start(); ...... } @Override public SendResult send(Message msg) throws MQClientException, RemotingException, MQBrokerException, InterruptedException { ...... return this.defaultMQProducerImpl.send(msg); }
DefaultMQProducerImpl实现类：
public void start() throws MQClientException { this.start(true); } //Producer启动方法 public void start(final boolean startFactory) throws MQClientException { switch (this.serviceState) { case CREATE_JUST: ...... //获取MQ实例 this.mQClientFactory = MQClientManager.getInstance().getOrCreateMQClientInstance(this.defaultMQProducer, rpcHook); //注册MQClientInstance实例，方便后续调用 boolean registerOK = mQClientFactory.registerProducer(this.defaultMQProducer.getProducerGroup(), this); ...... //启动实例 if (startFactory) { mQClientFactory.start(); } ...... //改变状态 this.serviceState = ServiceState.RUNNING; break; ...... } public SendResult send(Message msg) throws MQClientException, RemotingException, MQBrokerException, InterruptedException { return send(msg, this.defaultMQProducer.getSendMsgTimeout()); } public SendResult send(Message msg,long timeout) throws MQClientException, RemotingException, MQBrokerException, InterruptedException { return this.sendDefaultImpl(msg, CommunicationMode.SYNC, null, timeout); } //Producer发送消息的实现类 private SendResult sendDefaultImpl( Message msg, final CommunicationMode communicationMode, final SendCallback sendCallback, final long timeout ) throws MQClientException, RemotingException, MQBrokerException, InterruptedException { ...... //生产者获取Topic的信息 TopicPublishInfo topicPublishInfo = this.tryToFindTopicPublishInfo(msg.getTopic()); if (topicPublishInfo != null && topicPublishInfo.ok()) { ...... for (; times < timesTotal; times++) { String lastBrokerName = null == mq ? null : mq.getBrokerName(); //Producer计算把消息发到哪个MessageQueue中 MessageQueue mqSelected = this.selectOneMessageQueue(topicPublishInfo, lastBrokerName); if (mqSelected != null) { mq = mqSelected; //根据MessageQueue去获取目标节点的信息。 brokersSent[times] = mq.getBrokerName(); try { ...... //实际发送消息的方法 sendResult = this.sendKernelImpl(msg, mq, communicationMode, sendCallback, topicPublishInfo, timeout - costTime); endTimestamp = System.currentTimeMillis(); this.updateFaultItem(mq.getBrokerName(), endTimestamp - beginTimestampPrev, false); switch (communicationMode) { case ASYNC: return null; case ONEWAY: return null; case SYNC: if (sendResult.getSendStatus() != SendStatus.SEND_OK) { if (this.defaultMQProducer.isRetryAnotherBrokerWhenNotStoreOK()) { continue; } } return sendResult; default: break; } } ...... public MessageQueue selectOneMessageQueue(final TopicPublishInfo tpInfo, final String lastBrokerName) { return this.mqFaultStrategy.selectOneMessageQueue(tpInfo, lastBrokerName); } private SendResult sendKernelImpl(final Message msg, final MessageQueue mq, final CommunicationMode communicationMode, final SendCallback sendCallback, final TopicPublishInfo topicPublishInfo, final long timeout) throws MQClientException, RemotingException, MQBrokerException, InterruptedException { long beginStartTime = System.currentTimeMillis(); //寻找Broker地址。找不到就去NameServer上获取。 String brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName()); if (null == brokerAddr) { tryToFindTopicPublishInfo(mq.getTopic()); brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName()); } ......
MQClientInstance：
//核心启动过程 public void start() throws MQClientException { synchronized (this) { switch (this.serviceState) { case CREATE_JUST: this.serviceState = ServiceState.START_FAILED; // If not specified,looking address from name server if (null == this.clientConfig.getNamesrvAddr()) { this.mQClientAPIImpl.fetchNameServerAddr(); } // Start request-response channel this.mQClientAPIImpl.start(); // Start various schedule tasks this.startScheduledTask(); // Start pull service this.pullMessageService.start(); // Start rebalance service this.rebalanceService.start(); // Start push service this.defaultMQProducer.getDefaultMQProducerImpl().start(false); log.info("the client factory [{}] start OK", this.clientId); this.serviceState = ServiceState.RUNNING; break; ...... } } }
MQFaultStrategy：
//Producer选择MessageQueue的方法 public MessageQueue selectOneMessageQueue(final TopicPublishInfo tpInfo, final String lastBrokerName) { //这个sendLatencyFaultEnable默认是关闭的，Broker故障延迟机制，表示一种发送消息失败后一定时间内不在往同一个Queue重复发送的机制 if (this.sendLatencyFaultEnable) { try { //这里Producer选择MessageQueue的方法就是自增，然后取模。并且只有这一种方法。 int index = tpInfo.getSendWhichQueue().getAndIncrement(); for (int i = 0; i < tpInfo.getMessageQueueList().size(); i++) { int pos = Math.abs(index++) % tpInfo.getMessageQueueList().size(); if (pos < 0) pos = 0; MessageQueue mq = tpInfo.getMessageQueueList().get(pos); //Broker轮询。尽量将请求平均分配给不同的Broker if (latencyFaultTolerance.isAvailable(mq.getBrokerName())) { if (null == lastBrokerName || mq.getBrokerName().equals(lastBrokerName)) return mq; } } final String notBestBroker = latencyFaultTolerance.pickOneAtLeast(); int writeQueueNums = tpInfo.getQueueIdByBroker(notBestBroker); if (writeQueueNums > 0) { //这里计算也还是自增取模 final MessageQueue mq = tpInfo.selectOneMessageQueue(); if (notBestBroker != null) { mq.setBrokerName(notBestBroker); mq.setQueueId(tpInfo.getSendWhichQueue().getAndIncrement() % writeQueueNums); } return mq; } else { latencyFaultTolerance.remove(notBestBroker); } } catch (Exception e) { log.error("Error occurred when selecting message queue", e); } return tpInfo.selectOneMessageQueue(); } return tpInfo.selectOneMessageQueue(lastBrokerName); }
总结流程图
六、消息存储
Producer把消息发到了Broker，接下来Broker会对接收到的消息进行存储，最终存储的文件：
- commitLog：消息存储目录
- confifig：运行期间一些配置信息
- consumerqueue：消息消费队列存储目录
- index：消息索引文件存储目录
- abort：如果存在改文件寿命Broker非正常关闭
- checkpoint：文件检查点，存储CommitLog文件最后一次刷盘时间戳、consumerquueue最后一次刷盘时间，index索引文件最后一次刷盘时间戳。
commitLog写入
BrokerController在对Broker进行初始化时，创建了消息存储管理组件DefaultMessageStore，直接从DefaultMessageStore的putMessage入手：
//Broker存储消息的地方 @Override public PutMessageResult putMessage(MessageExtBrokerInner msg) { ...... //消息写入commitlog的方法 PutMessageResult result = this.commitLog.putMessage(msg); ...... }
CommitLog：
//CommitLog写入的过程
public PutMessageResult putMessage(final MessageExtBrokerInner msg) {
......
//如果消息是非事务类型或者事务提交类型进入
if (tranType == MessageSysFlag.TRANSACTION_NOT_TYPE
|| tranType == MessageSysFlag.TRANSACTION_COMMIT_TYPE) {
// Delay Delivery
//延迟消息写入
if (msg.getDelayTimeLevel() > 0) {
......
topic = TopicValidator.RMQ_SYS_SCHEDULE_TOPIC;
queueId = ScheduleMessageService.delayLevel2QueueId(msg.getDelayTimeLevel());
......
//如果是延迟消息，会偷偷修改topic、queueId
msg.setTopic(topic);
msg.setQueueId(queueId);
}
}
......
MappedFile unlockMappedFile = null;
//mappedFile 零拷贝实现
MappedFile mappedFile = this.mappedFileQueue.getLastMappedFile();
putMessageLock.lock(); //spin or ReentrantLock ,depending on store config
try {
long beginLockTimestamp = this.defaultMessageStore.getSystemClock().now();
this.beginTimeInLock = beginLockTimestamp;
// Here settings are stored timestamp, in order to ensure an orderly
// global
//设置消息存盘的时间戳
msg.setStoreTimestamp(beginLockTimestamp);
......
//直接以追加Append的方式写入文件，顺序写
result = mappedFile.appendMessage(msg, this.appendMessageCallback);
//文件写入的结果
switch (result.getStatus()) {
case PUT_OK:
break;
//文件写满了，就创建一个新文件，重写消息
case END_OF_FILE:
unlockMappedFile = mappedFile;
// Create a new file, re-write the message
mappedFile = this.mappedFileQueue.getLastMappedFile(0);
if (null == mappedFile) {
// XXX: warn and notify me
log.error("create mapped file2 error, topic: " + msg.getTopic() + " clientAddr: " + msg.getBornHostString());
beginTimeInLock = 0;
return new PutMessageResult(PutMessageStatus.CREATE_MAPEDFILE_FAILED, result);
}
result = mappedFile.appendMessage(msg, this.appendMessageCallback);
break;
......//写入时异常情况，抛出异常
}
......
//文件刷盘
handleDiskFlush(result, putMessageResult, msg);
//主从同步
handleHA(result, putMessageResult, msg);
return putMessageResult;
}
//实际写入CommitLog的方法。
public AppendMessageResult doAppend(final long fileFromOffset, final ByteBuffer byteBuffer, final int maxBlank,
final MessageExtBrokerInner msgInner) {
//一些列的计算，位置、长度等，
......
// Write messages to the queue buffer
byteBuffer.put(this.msgStoreItemMemory.array(), 0, msgLen);
......
}
//处理数据刷盘
public void handleDiskFlush(AppendMessageResult result, PutMessageResult putMessageResult, MessageExt messageExt) {
// Synchronization flush 同步刷盘
if (FlushDiskType.SYNC_FLUSH == this.defaultMessageStore.getMessageStoreConfig().getFlushDiskType()) {
final GroupCommitService service = (GroupCommitService) this.flushCommitLogService;
if (messageExt.isWaitStoreMsgOK()) {
//构建一个GroupCommitRequest，交给GroupCommitService处理。
GroupCommitRequest request = new GroupCommitRequest(result.getWroteOffset() + result.getWroteBytes());
service.putRequest(request);
CompletableFuture<PutMessageStatus> flushOkFuture = request.future();
PutMessageStatus flushStatus = null;
//同步等待文件刷新
try {
flushStatus = flushOkFuture.get(this.defaultMessageStore.getMessageStoreConfig().getSyncFlushTimeout(),
TimeUnit.MILLISECONDS);
} catch (InterruptedException | ExecutionException | TimeoutException e) {
//flushOK=false;
}
......
}
// Asynchronous flush 异步刷盘
//异步刷盘是把消息映射到MappedFile后，单独唤醒一个服务来进行刷盘
else {
if (!this.defaultMessageStore.getMessageStoreConfig().isTransientStorePoolEnable()) {
flushCommitLogService.wakeup();
} else {
commitLogService.wakeup();
}
}
}
一个MappedFile就对应一个CommitLog文件：
public AppendMessageResult appendMessage(final MessageExtBrokerInner msg, final AppendMessageCallback cb) {
return appendMessagesInner(msg, cb);
}
//追加CommitLog文件
public AppendMessageResult appendMessagesInner(final MessageExt messageExt, final AppendMessageCallback cb) {
//文件位置偏移量的判断
......
//写入
if (messageExt instanceof MessageExtBrokerInner) {//内部消息
result = cb.doAppend(this.getFileFromOffset(), byteBuffer, this.fileSize - currentPos, (MessageExtBrokerInner) messageExt);
} else if (messageExt instanceof MessageExtBatch) {//批量消息
result = cb.doAppend(this.getFileFromOffset(), byteBuffer, this.fileSize - currentPos, (MessageExtBatch) messageExt);
} else {
......
}
分发ConsumeQueue和IndexFile
BrokerController在对Broker进行启动时，启动了核心消息存储组件this.messageStore.start();
DefaultMessageStore的start方法：
public void start() throws Exception {
......
//DefaultMessageStore启动时会启动一个reputMessageService线程来做分发
this.reputMessageService.setReputFromOffset(maxPhysicalPosInLogicQueue);
//启动一个线程，那么就需要去看ReputMessageService的run方法
this.reputMessageService.start();
......
//Broker启动删除过期文件的定时任务
this.addScheduleTask();
this.shutdown = false;
}
//每隔1毫秒，会往ConsumeQueue和IndexFile中转发一次CommitLog写入的消息。
@Override
public void run() {
......
Thread.sleep(1);
this.doReput();
......
}
//消息转发
private void doReput() {
......
for (int readSize = 0; readSize < result.getSize() && doNext; ) {
//从CommitLog中获取一个DispatchRequest,拿到一份需要进行转发的消息，也就是从commitlog中读取的。
DispatchRequest dispatchRequest = DefaultMessageStore.this.commitLog.checkMessageAndReturnSize(result.getByteBuffer(), false, false);
int size = dispatchRequest.getBufferSize() == -1 ? dispatchRequest.getMsgSize() : dispatchRequest.getBufferSize();
if (dispatchRequest.isSuccess()) {
if (size > 0) {
//分发CommitLog写入消息
DefaultMessageStore.this.doDispatch(dispatchRequest);
//长轮询： 如果有消息到了主节点，并且开启了长轮询。
if (BrokerRole.SLAVE != DefaultMessageStore.this.getMessageStoreConfig().getBrokerRole()
&& DefaultMessageStore.this.brokerConfig.isLongPollingEnable()) {
//唤醒NotifyMessageArrivingListener的arriving方法，进行一次请求线程的检查
DefaultMessageStore.this.messageArrivingListener.arriving(dispatchRequest.getTopic(),
dispatchRequest.getQueueId(), dispatchRequest.getConsumeQueueOffset() + 1,
dispatchRequest.getTagsCode(), dispatchRequest.getStoreTimestamp(),
dispatchRequest.getBitMap(), dispatchRequest.getPropertiesMap());
}
......
}
//将commitLog写入的事件转发到ComsumeQueue和IndexFile
public void doDispatch(DispatchRequest req) {
for (CommitLogDispatcher dispatcher : this.dispatcherList) {
dispatcher.dispatch(req);
}
}
public interface CommitLogDispatcher {
void dispatch(final DispatchRequest request);
}
该接口方法有其中下面两个实现，分别对应ConsumeQueue和IndexFile
//Consumequeue文件分发的构建器
class CommitLogDispatcherBuildConsumeQueue implements CommitLogDispatcher {
@Override
public void dispatch(DispatchRequest request) {
final int tranType = MessageSysFlag.getTransactionValue(request.getSysFlag());
switch (tranType) {
case MessageSysFlag.TRANSACTION_NOT_TYPE:
case MessageSysFlag.TRANSACTION_COMMIT_TYPE:
DefaultMessageStore.this.putMessagePositionInfo(request);
break;
case MessageSysFlag.TRANSACTION_PREPARED_TYPE:
case MessageSysFlag.TRANSACTION_ROLLBACK_TYPE:
break;
}
}
}
//IndexFile文件分发的构建器
class CommitLogDispatcherBuildIndex implements CommitLogDispatcher {
@Override
public void dispatch(DispatchRequest request) {
if (DefaultMessageStore.this.messageStoreConfig.isMessageIndexEnable()) {
DefaultMessageStore.this.indexService.buildIndex(request);
}
}
}
七、Consumer消费消息
消费者有两种：推模式Push消费者和拉模式Pull消费者
DefaultMQPushConsumer的start方法：
@Override
public void start() throws MQClientException {
setConsumerGroup(NamespaceUtil.wrapNamespace(this.getNamespace(), this.consumerGroup));
this.defaultMQPushConsumerImpl.start();
......
}
defaultMQPushConsumerImpl实现：
//消费者端实际启动过程
public synchronized void start() throws MQClientException {
switch (this.serviceState) {
case CREATE_JUST:
......
//客户端创建工厂，是核心对象
this.mQClientFactory = MQClientManager.getInstance().getOrCreateMQClientInstance(this.defaultMQPushConsumer, this.rpcHook);
......
//根据客户端配置实例化不同的consumeMessageService
......
//注册本地的消费者组缓存。
......
//启动消费者
mQClientFactory.start();
......
}
与Producer启动相同，都会走到MQClientInstance的start：
//核心启动过程
public void start() throws MQClientException {
synchronized (this) {
switch (this.serviceState) {
case CREATE_JUST:
this.serviceState = ServiceState.START_FAILED;
// If not specified,looking address from name server
if (null == this.clientConfig.getNamesrvAddr()) {
this.mQClientAPIImpl.fetchNameServerAddr();
}
// Start request-response channel
this.mQClientAPIImpl.start();
// Start various schedule tasks
this.startScheduledTask();
// Start pull service
//拉取消息
this.pullMessageService.start();
// Start rebalance service
//客户端负载均衡
this.rebalanceService.start();
// Start push service
this.defaultMQProducer.getDefaultMQProducerImpl().start(false);
log.info("the client factory [{}] start OK", this.clientId);
this.serviceState = ServiceState.RUNNING;
break;
......
}
}
}
//客户端负载均衡 针对当前消费者所属的每一个消费者组
public void doRebalance() {
for (Map.Entry<String, MQConsumerInner> entry : this.consumerTable.entrySet()) {
MQConsumerInner impl = entry.getValue();
if (impl != null) {
try {
impl.doRebalance();
} catch (Throwable e) {
log.error("doRebalance exception", e);
}
}
}
}
负载均衡
this.rebalanceService.start();客服端负载均衡，rebalanceService的run方法：
public class RebalanceService extends ServiceThread {
......
@Override
public void run() {
......
this.mqClientFactory.doRebalance();
......
}
......
MQConsumerInner接口：
public interface MQConsumerInner {
......
void doRebalance();
......
有以下实现类，分别有各自的负载均衡实现：
以DefaultMQPushConsumerImpl推模式消息消费者实现类为例：
@Override
public void doRebalance() {
if (!this.pause) {
this.rebalanceImpl.doRebalance(this.isConsumeOrderly());
}
}
RebalanceImpl：
public void doRebalance(final boolean isOrder) {
......
//客户端负载：真正进行负载都是根据主题来进行的。
this.rebalanceByTopic(topic, isOrder);
......
}
//客户端负载：对Topic进行负载的过程
private void rebalanceByTopic(final String topic, final boolean isOrder) {
switch (messageModel) {
//广播模式，不需要进行负载，每个消费者都要消费。只需要更新负载信息。
case BROADCASTING: {
Set<MessageQueue> mqSet = this.topicSubscribeInfoTable.get(topic);
......
break;
}
//集群模式，需要负载
case CLUSTERING: {
//订阅的主题
Set<MessageQueue> mqSet = this.topicSubscribeInfoTable.get(topic);
//客户端ID
List<String> cidAll = this.mQClientFactory.findConsumerIdList(topic, consumerGroup);
......
if (mqSet != null && cidAll != null) {
List<MessageQueue> mqAll = new ArrayList<MessageQueue>();
mqAll.addAll(mqSet);
//排序后才能保证消费者负载策略相对稳定。
Collections.sort(mqAll);
Collections.sort(cidAll);
//MessageQueue的负载策略，有六种实现类
AllocateMessageQueueStrategy strategy = this.allocateMessageQueueStrategy;
List<MessageQueue> allocateResult = null;
try {
//按负载策略进行分配，返回当前消费者实际订阅的MessageQueue集合。
allocateResult = strategy.allocate(
this.consumerGroup,
this.mQClientFactory.getClientId(),
mqAll,
cidAll);
} ......
}
拉取消息
this.pullMessageService.start();拉取消息，PullMessageService的run方法：
@Override
public void run() {
......
//拉取消息的请求队列
PullRequest pullRequest = this.pullRequestQueue.take();
//处理请求
this.pullMessage(pullRequest);
......
}
private void pullMessage(final PullRequest pullRequest) {
......
DefaultMQPushConsumerImpl impl = (DefaultMQPushConsumerImpl) consumer;
//推模式的消费者最终还是会使用拉消息的方式
impl.pullMessage(pullRequest);
......
}
DefaultMQPushConsumerImpl
//拉取消息的核心流程
public void pullMessage(final PullRequest pullRequest) {
......
//从数量进行流控
......
//从消息大小进行流控
......
//判断是并发还是顺序进行不同流控
......
final long beginTimestamp = System.currentTimeMillis();
//客户端默认的拉取的回调函数，在拉取到消息后会进入这个方法处理。
PullCallback pullCallback = new PullCallback() {
......//下面有说明
}
......
try {
//客户端实际与服务器交互，拉取消息的地方
this.pullAPIWrapper.pullKernelImpl(
pullRequest.getMessageQueue(),
subExpression,
subscriptionData.getExpressionType(),
subscriptionData.getSubVersion(),
pullRequest.getNextOffset(),
this.defaultMQPushConsumer.getPullBatchSize(),
sysFlag,
commitOffsetValue,
BROKER_SUSPEND_MAX_TIME_MILLIS,
CONSUMER_TIMEOUT_MILLIS_WHEN_SUSPEND,
CommunicationMode.ASYNC,
pullCallback
);
} ......
}
PullAPIWrapper：
//消费者拉取消息
public PullResult pullKernelImpl(
final MessageQueue mq,
final String subExpression,
final String expressionType,
final long subVersion,
final long offset,
final int maxNums,
final int sysFlag,
final long commitOffset,
final long brokerSuspendMaxTimeMillis,
final long timeoutMillis,
final CommunicationMode communicationMode,
final PullCallback pullCallback
) throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
//找到Broker
FindBrokerResult findBrokerResult =
this.mQClientFactory.findBrokerAddressInSubscribe(mq.getBrokerName(),
this.recalculatePullFromWhichNode(mq), false);
......
if (findBrokerResult != null) {
......
//构建请求
PullMessageRequestHeader requestHeader = new PullMessageRequestHeader();
requestHeader.setConsumerGroup(this.consumerGroup);
requestHeader.setTopic(mq.getTopic());
requestHeader.setQueueId(mq.getQueueId());
requestHeader.setQueueOffset(offset);
requestHeader.setMaxMsgNums(maxNums);
requestHeader.setSysFlag(sysFlagInner);
requestHeader.setCommitOffset(commitOffset);
requestHeader.setSuspendTimeoutMillis(brokerSuspendMaxTimeMillis);
requestHeader.setSubscription(subExpression);
requestHeader.setSubVersion(subVersion);
requestHeader.setExpressionType(expressionType);
String brokerAddr = findBrokerResult.getBrokerAddr();
if (PullSysFlag.hasClassFilterFlag(sysFlagInner)) {
brokerAddr = computPullFromWhichFilterServer(mq.getTopic(), brokerAddr);
}
//拉取消息
PullResult pullResult = this.mQClientFactory.getMQClientAPIImpl().pullMessage(
brokerAddr,
requestHeader,
timeoutMillis,
communicationMode,
pullCallback);
return pullResult;
}
throw new MQClientException("The broker[" + mq.getBrokerName() + "] not exist", null);
}
MQClientAPIImpl：
public PullResult pullMessage(
final String addr,
final PullMessageRequestHeader requestHeader,
final long timeoutMillis,
final CommunicationMode communicationMode,
final PullCallback pullCallback
) throws RemotingException, MQBrokerException, InterruptedException {
RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.PULL_MESSAGE, requestHeader);
//几种拉取方式
switch (communicationMode) {
case ONEWAY:
assert false;
return null;
case ASYNC://异步拉取消息，会回调pullCallback.onSuccess / pullCallback.onException
this.pullMessageAsync(addr, request, timeoutMillis, pullCallback);
return null;
case SYNC://同步拉取消息
return this.pullMessageSync(addr, request, timeoutMillis);
default:
assert false;
break;
}
return null;
}
回过头来看PullCallback回调：
//客户端默认的拉取的回调函数，在拉取到消息后会进入这个方法处理。
PullCallback pullCallback = new PullCallback() {
@Override
public void onSuccess(PullResult pullResult) {
......
//将消息保存到processQueue
boolean dispatchToConsume = processQueue.putMessage(pullResult.getMsgFoundList());
//消费者消息服务处理消费到的消息
DefaultMQPushConsumerImpl.this.consumeMessageService.submitConsumeRequest(
pullResult.getMsgFoundList(),
processQueue,
pullRequest.getMessageQueue(),
dispatchToConsume);
......
}
@Override
public void onException(Throwable e) {
......
}
};
ConsumeMessageService接口：
void submitConsumeRequest(
final List<MessageExt> msgs,
final ProcessQueue processQueue,
final MessageQueue messageQueue,
final boolean dispathToConsume);
有两个实现类：
其中，ConsumeMessageConcurrentlyService处理并发消息一次只拉取32条数据，不足32直接处理，超过32条就进行分页处理。ConsumeMessageOrderlyService顺序消息消费会在消费前把队列锁起来，优先保证拉取同一个队列里的消息，将并发的消息顺序进行消费。
总结流程图
扫一扫
320
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
