提示:文章写完后,目录可以自动生成,如何生成可参考右边的帮助文档
文章目录
前言
RocketMQ发送普通消息有三种实现方法:可靠同步发送、可靠消息异步发送、单向发送。
一、漫谈RocketMQ消息发送
三种消息发送:同步、异步、单向。
同步:发送者向MQ执行发送消息API时,同步等待,直到消息服务器返回结果
异步:发送者向MQ执行发送消息API时,指定消息发送成功会的回调函数,然后调用消息发送的API后,立即返回,消息发送者线程不阻塞,直到运行结束,消息发送成功或者失败的回调任务在一个新的线程中执行。
单向:消息发送者向MQ执行发送消息API时,直接返回,不等待消息服务器的结果,也不注册回调函数。
二、认识RocketMQ消息
代码如下(示例):
RocketMQ消息封装类:org.apache.rocketmq.common.message.Message
public class Message implements Serializable {
private static final long serialVersionUID = 8445773977080406428L;
//主题
private String topic;
//消息flage mq处理
private int flag;
//扩展属性
private Map<String, String> properties;
//消息体
private byte[] body;
//事务id
private String transactionId;
public Message() {
}
/**
* @param topic 主题
* @param tags 消息tag,用于消息过滤
* @param keys 索引key,多个用空格隔开 TODO 为什么隔开不生效
* @param waitStoreMsgOK 消息发送时是否等消息存储完成后再返回
* @param
*/
public Message(String topic, String tags, String keys, int flag, byte[] body, boolean waitStoreMsgOK) {
this.topic = topic;
this.flag = flag;
this.body = body;
if (tags != null && tags.length() > 0)
this.setTags(tags);
if (keys != null && keys.length() > 0)
this.setKeys(keys);
this.setWaitStoreMsgOK(waitStoreMsgOK);
}
//...省略代码
}
三、生产者启动流程
消息生产者代码在client模块中,相对于mq它是客户端,也是消息的消费者
1.初识DefaultMQProducer消息发送者
DefaultMQProducer是默认的消息生产者实现类,MQProducer 主要是消息的发送方式。
public class DefaultMQProducer extends ClientConfig implements MQProducer {
/**
* Wrapping internal implementations for virtually all methods presented in this class.
*/
protected final transient DefaultMQProducerImpl defaultMQProducerImpl;
private final InternalLogger log = ClientLogger.getLog();
/**
* Producer group conceptually aggregates all producer instances of exactly same role, which is particularly
* important when transactional messages are involved. </p>
* 生产者所属的组,消息服务器在回查事务状态时会随机选择该组件任何一个生产者发起事务回查请求
* For non-transactional messages, it does not matter as long as it's unique per process. </p>
*
* See {@linktourl http://rocketmq.apache.org/docs/core-concept/} for more discussion.
*/
private String producerGroup;
/**
* Just for testing or demo program
* 默认的 topickey
*/
private String createTopicKey = MixAll.AUTO_CREATE_TOPIC_KEY_TOPIC;
/**
* Number of queues to create per default topic.
* 默认主题在每个broker队列数量
*/
private volatile int defaultTopicQueueNums = 4;
/**
* Timeout for sending messages.
* 发送消息默认超时时间
*/
private int sendMsgTimeout = 3000;
/**
* Compress message body threshold, namely, message body larger than 4k will be compressed on default.
* 消息体超过该值则启用压缩,默认 4k
*/
private int compressMsgBodyOverHowmuch = 1024 * 4;
/**
* Maximum number of retry to perform internally before claiming sending failure in synchronous mode. </p>
* 同步方式消息发送重试次数,默认2次,总共执行三次
* This may potentially cause message duplication which is up to application developers to resolve.
*/
private int retryTimesWhenSendFailed = 2;
/**
* Maximum number of retry to perform internally before claiming sending failure in asynchronous mode. </p>
* 异步方式消息发送重试次数,默认2次,总共执行三次
* This may potentially cause message duplication which is up to application developers to resolve.
*/
private int retryTimesWhenSendAsyncFailed = 2;
/**
* Indicate whether to retry another broker on sending failure internally.
* 消息重试选择另外一个Broker是,是否不等待存储结果就返回,
*/
private boolean retryAnotherBrokerWhenNotStoreOK = false;
/**
* Maximum allowed message size in bytes.
* 最大消息长度
*/
private int maxMessageSize = 1024 * 1024 * 4; // 4M
/**
* Interface of asynchronous transfer data
*/
private TraceDispatcher traceDispatcher = null;
}
MQAdmin 方法讲解 主要是主题的创建,消息的查找
public interface MQAdmin {
/**
* Creates an topic
*
* @param key accesskey
* @param newTopic topic name
* @param queueNum topic's queue number
*/
void createTopic(final String key, final String newTopic, final int queueNum)
throws MQClientException;
/**
* Creates an topic
*
* @param key accesskey
* @param newTopic topic name
* @param queueNum topic's queue number
* @param topicSysFlag topic system flag
*/
void createTopic(String key, String newTopic, int queueNum, int topicSysFlag)
throws MQClientException;
/**
* Gets the message queue offset according to some time in milliseconds<br>
* be cautious to call because of more IO overhead
* 根根据时间戳从队列中查找偏移量
* @param mq Instance of MessageQueue
* @param timestamp from when in milliseconds.
* @return offset
*/
long searchOffset(final MessageQueue mq, final long timestamp) throws MQClientException;
/**
* Gets the max offset
* 查找消息队列的最大物理偏移量
* @param mq Instance of MessageQueue
* @return the max offset
*/
long maxOffset(final MessageQueue mq) throws MQClientException;
/**
* Gets the minimum offset
* 查找消息队列的最小物理偏移量
* @param mq Instance of MessageQueue
* @return the minimum offset
*/
long minOffset(final MessageQueue mq) throws MQClientException;
/**
* Gets the earliest stored message time
*
* @param mq Instance of MessageQueue
* @return the time in microseconds
*/
long earliestMsgStoreTime(final MessageQueue mq) throws MQClientException;
/**
* Query message according to message id
* 根据消息偏移量查看消息
* @param offsetMsgId message id
* @return message
*/
MessageExt viewMessage(final String offsetMsgId) throws RemotingException, MQBrokerException,
InterruptedException, MQClientException;
/**
* Query messages
* 根据条件查找消息
* @param topic message topic 主题
* @param key message key index word 消息索引字段
* @param maxNum max message number 本次获取消息最大条数
* @param begin from when 开始时间
* @param end to when 结束时间
* @return Instance of QueryResult
*/
QueryResult queryMessage(final String topic, final String key, final int maxNum, final long begin,
final long end) throws MQClientException, InterruptedException;
/**
* 根据主题与消息id查找消息
* @return The {@code MessageExt} of given msgId
*/
MessageExt viewMessage(String topic,
String msgId) throws RemotingException, MQBrokerException, InterruptedException, MQClientException;
}
2.消息生产者启动流程
代码如下(示例):org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl#start
public void start(final boolean startFactory) throws MQClientException {
switch (this.serviceState) {
case CREATE_JUST:
this.serviceState = ServiceState.START_FAILED;
//检查ProducerGroup是否符合要求,
this.checkConfig();
//改变生产者的 instanceName为进程id
if (!this.defaultMQProducer.getProducerGroup().equals(MixAll.CLIENT_INNER_PRODUCER_GROUP)) {
this.defaultMQProducer.changeInstanceNameToPID();
}
//获取 MQClientManager实例 为单例模式,
this.mQClientFactory = MQClientManager.getInstance().getOrCreateMQClientInstance(this.defaultMQProducer, rpcHook);
//向MQClientInstance注册,将目前生产者加入到MQClientInstance管理中,然后调用网网络请求,进行心跳检测
boolean registerOK = mQClientFactory.registerProducer(this.defaultMQProducer.getProducerGroup(), this);
// 省略部分代碼
}
private void checkConfig() throws MQClientException {
Validators.checkGroup(this.defaultMQProducer.getProducerGroup());
if (null == this.defaultMQProducer.getProducerGroup()) {
throw new MQClientException("producerGroup is null", null);
}
if (this.defaultMQProducer.getProducerGroup().equals(MixAll.DEFAULT_PRODUCER_GROUP)) {
throw new MQClientException("producerGroup can not equal " + MixAll.DEFAULT_PRODUCER_GROUP + ", please specify another one.",
null);
}
}
MQClientManager#getOrCreateMQClientInstance:
/**
* 创建MQClientInstance实例,一个jvm实例中只会存在一个MQClientManager实例,维护一个MQClientInstance缓存表
* ConcurrenMap<String * clientId *,MQClientInstance> factoryTable,
* 同一个clientId只会创建一个 MQClientInstance
* 问题:如果在同一台服务器部署两个应用程序,应用程序clienid相同怎么处理?
* 为了避免这个问题,如果instance为默认的DEFAULT的话 ,RocketMQ会自动将instance设置为进程id,这样避免了不同进程
* 的相相互影响!但是同一个jvm中的不同消费者和不同生产者在启动时获取到的MQClientInstance实例为同一个。
*
* MQClientInstance封装了RocketMQ网络处理API,是消息生产者,消息消费者与NameServer、Broker打交道的网络通信通道
*/
public MQClientInstance getOrCreateMQClientInstance(final ClientConfig clientConfig, RPCHook rpcHook) {
//ip@服务PID
String clientId = clientConfig.buildMQClientId();
MQClientInstance instance = this.factoryTable.get(clientId);
if (null == instance) {
instance =
new MQClientInstance(clientConfig.cloneClientConfig(),
this.factoryIndexGenerator.getAndIncrement(), clientId, rpcHook);
MQClientInstance prev = this.factoryTable.putIfAbsent(clientId, instance);
if (prev != null) {
instance = prev;
log.warn("Returned Previous MQClientInstance for clientId:[{}]", clientId);
} else {
log.info("Created new MQClientInstance for clientId:[{}]", clientId);
}
}
return instance;
}
org.apache.rocketmq.client.ClientConfig#buildMQClientId
public String buildMQClientId() {
StringBuilder sb = new StringBuilder();
sb.append(this.getClientIP());
sb.append("@");
sb.append(this.getInstanceName());
if (!UtilAll.isBlank(this.unitName)) {
sb.append("@");
sb.append(this.unitName);
}
return sb.toString();
}
四.消息发送基本流程
消息发送主要步奏:验证消息、查找路由、消息队列选择、消息发送(包含异常处理机制)
DefaultMQProducer#send->調用DefaultMQProducerImpl#send
1. 验证消息、查找路由
private SendResult sendDefaultImpl(
Message msg,
final CommunicationMode communicationMode,
final SendCallback sendCallback,
final long timeout
) throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
//确保生产者处于运行中
this.makeSureStateOK();
//校验消息 1.校验topic是否合法 2.校验消息体是否大于 最大设置大小 默认4M
Validators.checkMessage(msg, this.defaultMQProducer);
final long invokeID = random.nextLong();
long beginTimestampFirst = System.currentTimeMillis();
long beginTimestampPrev = beginTimestampFirst;
long endTimestamp = beginTimestampFirst;
//获取tpoic信息
TopicPublishInfo topicPublishInfo = this.tryToFindTopicPublishInfo(msg.getTopic());
//..省略部分代碼
第一次发送消息时,本地没有缓存topic路由信息,查询namerServer尝试获取,如果路由信息未找到,再次尝试用默认主题
DefaultMQProducerImpl.createTopicKey去查询,如果BrokerConfig# autoCreateTopicEnable为true,NameServer将返回路由信息
,如果 autoCreateTopicEnable 为false将抛出无法找到topic路由信息异常
/**
* 第一次发送消息时,本地没有缓存topic路由信息,查询namerServer尝试获取,如果路由信息未找到,再次尝试用默认主题
* DefaultMQProducerImpl.createTopicKey去查询,如果BrokerConfig# autoCreateTopicEnable为true,NameServer将返回路由信息
*,如果autoCreateTopicEnable 为false将抛出无法找到topic路由信息异常
*
*/
private TopicPublishInfo tryToFindTopicPublishInfo(final String topic) {
TopicPublishInfo topicPublishInfo = this.topicPublishInfoTable.get(topic);
// 如果路由信息为空或者 消息队列为空
if (null == topicPublishInfo || !topicPublishInfo.ok()) {
this.topicPublishInfoTable.putIfAbsent(topic, new TopicPublishInfo());
//缓存中没有包含消息队列, 则向NameServer查询该topic的路由信息。 如果最终未找到则抛出异常
this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic);
topicPublishInfo = this.topicPublishInfoTable.get(topic);
}
if (topicPublishInfo.isHaveTopicRouterInfo() || topicPublishInfo.ok()) {
return topicPublishInfo;
} else {
this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic, true, this.defaultMQProducer);
topicPublishInfo = this.topicPublishInfoTable.get(topic);
return topicPublishInfo;
}
}
public class TopicPublishInfo {
//是否顺序消息
private boolean orderTopic = false;
private boolean haveTopicRouterInfo = false;
//该主题的消息队列
private List<MessageQueue> messageQueueList = new ArrayList<MessageQueue>();
// 每选择一次消息队列,该值会自增1,如果为 Integer.MAX_VALUE 则重置为0,用于与选择选择消息队列
private volatile ThreadLocalIndex sendWhichQueue = new ThreadLocalIndex();
private TopicRouteData topicRouteData;
//省略部分代碼
}
public class TopicRouteData extends RemotingSerializable {
private String orderTopicConf;
//topic队列元数据
private List<QueueData> queueDatas;
//topic分布的broker元数据
private List<BrokerData> brokerDatas;
//broker上过滤服务器地址列表
private HashMap<String/* brokerAddr */, List<String>/* Filter Server */> filterServerTable;
//省略部分代碼
}
MQClientInstance#updateTopicRouteInfoFromNameServer
public boolean updateTopicRouteInfoFromNameServer(final String topic, boolean isDefault,
DefaultMQProducer defaultMQProducer) {
try {
// 锁定
if (this.lockNamesrv.tryLock(LOCK_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS)) {
try {
TopicRouteData topicRouteData;
// isDefault 设置为 true,则使用默认主题去查找,如果查询到路由信息,则替换路由信息中读写队列个数为消息生产者默认的
//队列个数
if (isDefault && defaultMQProducer != null) {
topicRouteData = this.mQClientAPIImpl.getDefaultTopicRouteInfoFromNameServer(defaultMQProducer.getCreateTopicKey(),
1000 * 3);
if (topicRouteData != null) {
for (QueueData data : topicRouteData.getQueueDatas()) {
int queueNums = Math.min(defaultMQProducer.getDefaultTopicQueueNums(), data.getReadQueueNums());
data.setReadQueueNums(queueNums);
data.setWriteQueueNums(queueNums);
}
}
} else {
//;如果为false则 使用topic去查找;
topicRouteData = this.mQClientAPIImpl.getTopicRouteInfoFromNameServer(topic, 1000 * 3);
}
//如果路由信息找到
if (topicRouteData != null) {
TopicRouteData old = this.topicRouteTable.get(topic);
// 与本地缓存中的路由信息进行对比,判断是否改变 未改变为false
boolean changed = topicRouteDataIsChange(old, topicRouteData);
if (!changed) {
changed = this.isNeedUpdateTopicRouteInfo(topic);
} else {
log.info("the topic[{}] route info changed, old[{}] ,new[{}]", topic, old, topicRouteData);
}
if (changed) {
TopicRouteData cloneTopicRouteData = topicRouteData.cloneTopicRouteData();
for (BrokerData bd : topicRouteData.getBrokerDatas()) {
this.brokerAddrTable.put(bd.getBrokerName(), bd.getBrokerAddrs());
}
// Update Pub info
{
//根据topicRouteData中的TopicRouteData List<QueueData> 转化为 TopicPublishInfo List<MessageQueue>
TopicPublishInfo publishInfo = topicRouteData2TopicPublishInfo(topic, topicRouteData);
publishInfo.setHaveTopicRouterInfo(true);
Iterator<Entry<String, MQProducerInner>> it = this.producerTable.entrySet().iterator();
while (it.hasNext()) {
Entry<String, MQProducerInner> entry = it.next();
MQProducerInner impl = entry.getValue();
if (impl != null) {
impl.updateTopicPublishInfo(topic, publishInfo);
}
}
}
// Update sub info
{
Set<MessageQueue> subscribeInfo = topicRouteData2TopicSubscribeInfo(topic, topicRouteData);
Iterator<Entry<String, MQConsumerInner>> it = this.consumerTable.entrySet().iterator();
while (it.hasNext()) {
Entry<String, MQConsumerInner> entry = it.next();
MQConsumerInner impl = entry.getValue();
if (impl != null) {
impl.updateTopicSubscribeInfo(topic, subscribeInfo);
}
}
}
log.info("topicRouteTable.put. Topic = {}, TopicRouteData[{}]", topic, cloneTopicRouteData);
this.topicRouteTable.put(topic, cloneTopicRouteData);
return true;
}
} else {
log.warn("updateTopicRouteInfoFromNameServer, getTopicRouteInfoFromNameServer return null, Topic: {}", topic);
}
} catch (MQClientException e) {
if (!topic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX) && !topic.equals(MixAll.AUTO_CREATE_TOPIC_KEY_TOPIC)) {
log.warn("updateTopicRouteInfoFromNameServer Exception", e);
}
} catch (RemotingException e) {
log.error("updateTopicRouteInfoFromNameServer Exception", e);
throw new IllegalStateException(e);
} finally {
this.lockNamesrv.unlock();
}
} else {
log.warn("updateTopicRouteInfoFromNameServer tryLock timeout {}ms", LOCK_TIMEOUT_MILLIS);
}
} catch (InterruptedException e) {
log.warn("updateTopicRouteInfoFromNameServer Exception", e);
}
return false;
}
public static TopicPublishInfo topicRouteData2TopicPublishInfo(final String topic, final TopicRouteData route) {
TopicPublishInfo info = new TopicPublishInfo();
info.setTopicRouteData(route);
if (route.getOrderTopicConf() != null && route.getOrderTopicConf().length() > 0) {
String[] brokers = route.getOrderTopicConf().split(";");
for (String broker : brokers) {
String[] item = broker.split(":");
int nums = Integer.parseInt(item[1]);
for (int i = 0; i < nums; i++) {
MessageQueue mq = new MessageQueue(topic, item[0], i);
info.getMessageQueueList().add(mq);
}
}
info.setOrderTopic(true);
} else {
// 循环遍历 QueueData信息(brokerName,读写队列等 ),
List<QueueData> qds = route.getQueueDatas();
Collections.sort(qds);
for (QueueData qd : qds) {
// 判断队列有无写权限
if (PermName.isWriteable(qd.getPerm())) {
BrokerData brokerData = null;
for (BrokerData bd : route.getBrokerDatas()) {
// 根据brokerName找打brokerData信息,
if (bd.getBrokerName().equals(qd.getBrokerName())) {
brokerData = bd;
break;
}
}
if (null == brokerData) {
continue;
}
// 如果不是主节点则查询下一个
if (!brokerData.getBrokerAddrs().containsKey(MixAll.MASTER_ID)) {
continue;
}
// 创建队列
for (int i = 0; i < qd.getWriteQueueNums(); i++) {
MessageQueue mq = new MessageQueue(topic, qd.getBrokerName(), i);
//填充队列到 TopicPublishInfo
info.getMessageQueueList().add(mq);
}
}
}
info.setOrderTopic(false);
}
return info;
}
2.消息队列选择
org.apache.rocketmq.client.impl.producer.TopicPublishInfo#selectOneMessageQueue()
选择消息队列存在两种方式:
2.1 sendLatencyFaultEnable=false 默认不启用Broker故障延迟机制
消息发送失败,下次进行消息队列选择时则规避上一次MessageQueue所在的Broker。该算法在第一次消息发送过程中能成功规避故障的Broker,但是Broker宕机,由于队列按照Broker队列排序,如果上一次根据路由算法选择的宕机的Broker的第一个队列,下次会选择Broker宕机的第二队列。
问题:Broker不可用,为什么路由信息还包含该Broker路由信息
NameServer检测Broker是否可用存在延迟, NameServer不会检测到Broker宕机后马上推送消息给消息生产者,而是消息生产者每隔30s更新一次路由信息,所以消息生产者最快感知Broker最新的路由信息也需要30s. 不过可以引入一种机制,在Broker宕机期间,如果一次消息发送失败后,可以将Broker暂时排除在消息队列的范围中
2.2 ·sendLatencyFaultEnable=true 启用故障延迟机制
MQFaultStrategy#selectOneMessageQueue
public MessageQueue selectOneMessageQueue(final TopicPublishInfo tpInfo, final String lastBrokerName) {
if (this.sendLatencyFaultEnable) {
try {
int index = tpInfo.getSendWhichQueue().getAndIncrement();
for (int i = 0; i < tpInfo.getMessageQueueList().size(); i++) {
int pos = Math.abs(index++) % tpInfo.getMessageQueueList().size();
if (pos < 0)
pos = 0;
MessageQueue mq = tpInfo.getMessageQueueList().get(pos);
//验证消息队列是否可用
if (latencyFaultTolerance.isAvailable(mq.getBrokerName())) {
if (null == lastBrokerName || mq.getBrokerName().equals(lastBrokerName))
return mq;
}
}
//尝试从规避的Broker中选择一个可用的Broker,如果没找到,将返回null
final String notBestBroker = latencyFaultTolerance.pickOneAtLeast();
int writeQueueNums = tpInfo.getQueueIdByBroker(notBestBroker);
if (writeQueueNums > 0) {
final MessageQueue mq = tpInfo.selectOneMessageQueue();
if (notBestBroker != null) {
mq.setBrokerName(notBestBroker);
mq.setQueueId(tpInfo.getSendWhichQueue().getAndIncrement() % writeQueueNums);
}
return mq;
} else {
latencyFaultTolerance.remove(notBestBroker);
}
} catch (Exception e) {
log.error("Error occurred when selecting message queue", e);
}
return tpInfo.selectOneMessageQueue();
}
3.消息发送
DefaultMQProducerImpl#sendKernelImpl
/**
*
* @param msg 待发送的消息
* @param mq 消息发送的消息队列队列
* @param communicationMode 发送方式 SYNC,ASYNC,ONEWAY
* @param sendCallback 一步发送回调函数
* @param topicPublishInfo 主题路由信息
* @param timeout 超时时间
*
*/
private SendResult sendKernelImpl(final Message msg,
final MessageQueue mq,
final CommunicationMode communicationMode,
final SendCallback sendCallback,
final TopicPublishInfo topicPublishInfo,
final long timeout) throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
long beginStartTime = System.currentTimeMillis();
// 获取brokerId 主节点
String brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName());
if (null == brokerAddr) {
tryToFindTopicPublishInfo(mq.getTopic());
brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName());
}
SendMessageContext context = null;
if (brokerAddr != null) {
brokerAddr = MixAll.brokerVIPChannel(this.defaultMQProducer.isSendMessageWithVIPChannel(), brokerAddr);
byte[] prevBody = msg.getBody();
try {
//for MessageBatch,ID has been set in the generating process
//为消息设置全局唯一id,
if (!(msg instanceof MessageBatch)) {
MessageClientIDSetter.setUniqID(msg);
}
boolean topicWithNamespace = false;
if (null != this.mQClientFactory.getClientConfig().getNamespace()) {
msg.setInstanceId(this.mQClientFactory.getClientConfig().getNamespace());
topicWithNamespace = true;
}
int sysFlag = 0;
boolean msgBodyCompressed = false;
//消息 默认超过设置压缩值则执行压缩,并将消息 标识为COMPRESSED_FLAG
if (this.tryToCompressMessage(msg)) {
sysFlag |= MessageSysFlag.COMPRESSED_FLAG;
msgBodyCompressed = true;
}
// 如果是事务Prepared消息,则设置消息系统标识TRANSACTION_PREPARED_TYPE
final String tranMsg = msg.getProperty(MessageConst.PROPERTY_TRANSACTION_PREPARED);
if (tranMsg != null && Boolean.parseBoolean(tranMsg)) {
sysFlag |= MessageSysFlag.TRANSACTION_PREPARED_TYPE;
}
if (hasCheckForbiddenHook()) {
CheckForbiddenContext checkForbiddenContext = new CheckForbiddenContext();
checkForbiddenContext.setNameSrvAddr(this.defaultMQProducer.getNamesrvAddr());
checkForbiddenContext.setGroup(this.defaultMQProducer.getProducerGroup());
checkForbiddenContext.setCommunicationMode(communicationMode);
checkForbiddenContext.setBrokerAddr(brokerAddr);
checkForbiddenContext.setMessage(msg);
checkForbiddenContext.setMq(mq);
checkForbiddenContext.setUnitMode(this.isUnitMode());
this.executeCheckForbiddenHook(checkForbiddenContext);
}
//如果注册了钩子函数,则执行消息发送前的增强逻辑
if (this.hasSendMessageHook()) {
context = new SendMessageContext();
context.setProducer(this);
context.setProducerGroup(this.defaultMQProducer.getProducerGroup());
context.setCommunicationMode(communicationMode);
context.setBornHost(this.defaultMQProducer.getClientIP());
context.setBrokerAddr(brokerAddr);
context.setMessage(msg);
context.setMq(mq);
context.setNamespace(this.defaultMQProducer.getNamespace());
String isTrans = msg.getProperty(MessageConst.PROPERTY_TRANSACTION_PREPARED);
if (isTrans != null && isTrans.equals("true")) {
context.setMsgType(MessageType.Trans_Msg_Half);
}
if (msg.getProperty("__STARTDELIVERTIME") != null || msg.getProperty(MessageConst.PROPERTY_DELAY_TIME_LEVEL) != null) {
context.setMsgType(MessageType.Delay_Msg);
}
this.executeSendMessageHookBefore(context);
}
SendMessageRequestHeader requestHeader = new SendMessageRequestHeader();
//生产组赋值 生产者所属的组,消息服务器在回查事务状态时会随机选择该组件任何一个生产者发起事务回查请求
requestHeader.setProducerGroup(this.defaultMQProducer.getProducerGroup());
//主题
requestHeader.setTopic(msg.getTopic());
//默认创建主题key
requestHeader.setDefaultTopic(this.defaultMQProducer.getCreateTopicKey());
//单个Broker默认队列数
requestHeader.setDefaultTopicQueueNums(this.defaultMQProducer.getDefaultTopicQueueNums());
// 消息队列id
requestHeader.setQueueId(mq.getQueueId());
// 消息系统标记 压缩标识,事务消息标识等
requestHeader.setSysFlag(sysFlag);
//消息发送时间
requestHeader.setBornTimestamp(System.currentTimeMillis());
//消息标识
requestHeader.setFlag(msg.getFlag());
//消息扩容属性
requestHeader.setProperties(MessageDecoder.messageProperties2String(msg.getProperties()));
//消息重试次数
requestHeader.setReconsumeTimes(0);
requestHeader.setUnitMode(this.isUnitMode());
//是否批量消息
requestHeader.setBatch(msg instanceof MessageBatch);
//判断是否为重试主题
if (requestHeader.getTopic().startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {
String reconsumeTimes = MessageAccessor.getReconsumeTime(msg);
if (reconsumeTimes != null) {
requestHeader.setReconsumeTimes(Integer.valueOf(reconsumeTimes));
MessageAccessor.clearProperty(msg, MessageConst.PROPERTY_RECONSUME_TIME);
}
String maxReconsumeTimes = MessageAccessor.getMaxReconsumeTimes(msg);
if (maxReconsumeTimes != null) {
requestHeader.setMaxReconsumeTimes(Integer.valueOf(maxReconsumeTimes));
MessageAccessor.clearProperty(msg, MessageConst.PROPERTY_MAX_RECONSUME_TIMES);
}
} SendResult sendResult = null;
switch (communicationMode) {
case ASYNC:
Message tmpMessage = msg;
boolean messageCloned = false;
if (msgBodyCompressed) {
//If msg body was compressed, msgbody should be reset using prevBody.
//Clone new message using commpressed message body and recover origin massage.
//Fix bug:https://github.com/apache/rocketmq-externals/issues/66
tmpMessage = MessageAccessor.cloneMessage(msg);
messageCloned = true;
msg.setBody(prevBody);
}
if (topicWithNamespace) {
if (!messageCloned) {
tmpMessage = MessageAccessor.cloneMessage(msg);
messageCloned = true;
}
msg.setTopic(NamespaceUtil.withoutNamespace(msg.getTopic(), this.defaultMQProducer.getNamespace()));
}
long costTimeAsync = System.currentTimeMillis() - beginStartTime;
if (timeout < costTimeAsync) {
throw new RemotingTooMuchRequestException("sendKernelImpl call timeout");
}
sendResult = this.mQClientFactory.getMQClientAPIImpl().sendMessage(
brokerAddr,
mq.getBrokerName(),
tmpMessage,
requestHeader,
timeout - costTimeAsync,
communicationMode,
sendCallback,
topicPublishInfo,
this.mQClientFactory,
this.defaultMQProducer.getRetryTimesWhenSendAsyncFailed(),
context,
this);
break;
case ONEWAY:
case SYNC:
long costTimeSync = System.currentTimeMillis() - beginStartTime;
if (timeout < costTimeSync) {
throw new RemotingTooMuchRequestException("sendKernelImpl call timeout");
}
sendResult = this.mQClientFactory.getMQClientAPIImpl().sendMessage(
brokerAddr,
mq.getBrokerName(),
msg,
requestHeader,
timeout - costTimeSync,
communicationMode,
context,
this);
break;
default:
assert false;
break;
}
//注册消息钩子发送函数,执行after逻辑,就算发送异常 改方法也会执行
if (this.hasSendMessageHook()) {
context.setSendResult(sendResult);
this.executeSendMessageHookAfter(context);
}
return sendResult;
} catch (RemotingException e) {
if (this.hasSendMessageHook()) {
context.setException(e);
this.executeSendMessageHookAfter(context);
}
throw e;
} catch (MQBrokerException e) {
if (this.hasSendMessageHook()) {
context.setException(e);
this.executeSendMessageHookAfter(context);
}
throw e;
} catch (InterruptedException e) {
if (this.hasSendMessageHook()) {
context.setException(e);
this.executeSendMessageHookAfter(context);
}
throw e;
} finally {
msg.setBody(prevBody);
msg.setTopic(NamespaceUtil.withoutNamespace(msg.getTopic(), this.defaultMQProducer.getNamespace()));
}
}
throw new MQClientException("The broker[" + mq.getBrokerName() + "] not exist", null);
}
public MQClientInstance getmQClientFactory() {
return mQClientFactory;
}
private boolean tryToCompressMessage(final Message msg) {
if (msg instanceof MessageBatch) {
//batch dose not support compressing right now
return false;
}
byte[] body = msg.getBody();
if (body != null) {
if (body.length >= this.defaultMQProducer.getCompressMsgBodyOverHowmuch()) {
try {
byte[] data = UtilAll.compress(body, zipCompressLevel);
if (data != null) {
msg.setBody(data);
return true;
}
} catch (IOException e) {
log.error("tryToCompressMessage exception", e);
log.warn(msg.toString());
}
}
}
return false;
}
3.1.发送方式:
1.1同步
1.2. 异步
优:消息发送性能会显著提升,但为了保护消息服务器的负载压力,进行了并发控制
缺:异步消息发送可以设置重试次数,但是重试调用的入口是在收到服务端响应包时进行的,如果出现网络异常、网络超时等将不会重试
1.3. 单向发送
无需等待消息服务器返回本次消息发送结果,并且无需提供回调函数。其原理与异步 相同,只是无重试机制 </br>
1.4. 批量消息发送(同步)
批量消息发送对消息内容使用了固定格式的进行存储,方便服务端正确解析出每条消息
批量消息发送(同步)
五.总结
5.1:消息队列负载机制
消息生产者在发送消息时,如果本地路由表中未缓存topic的路由信息,向NameServer发送消息获取路由信息请求,
更新本地的路由信息表,并且消息生产者每隔30s从NameServer更新路由表
5.2:消息发送异常机制
消息发送高可用主要是通过两个手段:重试与Broker规避。Broker规避是在第一次消息发送过程中发现错误,在某一
时间段,消息生产者将不会选择 Broker上的消息队列,提高消息的成功率
5.3:批量消息发送
RocketMQ支持将同一主题下的多条信息一次性发送到消息端服务器