RocketMQ：事务消息

什么是事务消息

What is transactional message?
It can be thought of as a two-phase commit message implementation to ensure eventual consistency in distributed system. Transactional message ensures that the execution of local transaction and the sending of message can be performed atomically.

就是说是一种两阶段提交来实现分布式系统中的最终一致性。事务消息保证了本地事务和消息发送的原子性。

使用约束

(1) Messages of the transactional have no schedule and batch support.
(2) In order to avoid a single message being checked too many times and lead to half queue message accumulation， we limited the number of checks for a single message to 15 times by default, but users can change this limit by change the “transactionCheckMax” parameter in the configuration of the broker， if one message has been checked over “transactionCheckMax” times， broker will discard this message and print an error log at the same time by default. Users can change this behavior by override the “AbstractTransactionCheckListener” class.
(3) A transactional message will be checked after a certain period of time that determined by parameter “transactionTimeout” in the configuration of the broker. And users also can change this limit by set user property “CHECK_IMMUNITY_TIME_IN_SECONDS” when sending transactional message, this parameter takes precedence over the “transactionMsgTimeout” parameter.
(4) A transactional message maybe checked or consumed more than once.
(5) Committed message reput to the user’s target topic may fail. Currently, it depends on the log record. High availability is ensured by the high availability mechanism of RocketMQ itself. If you want to ensure that the transactional message isn’t lost and the transaction integrity is guaranteed, it is recommended to use synchronous double write. mechanism.
(6) Producer IDs of transactional messages cannot be shared with producer IDs of other types of messages. Unlike other types of message, transactional messages allow backward queries. MQ Server query clients by their Producer IDs.

翻译官网如下：

1. 事务消息不支持延时发送和批量发送。
2. 为了避免单条消息被检查太多次数，导致队列消息阻塞，默认限制单挑消息检查次数上限为15，你可以修改transactionCheckMax
参数。如果达到最大次数，broker默认会丢弃消息，并打印错误日志。你可以修改这个行为：通过重写AbstractTransactionCheckListener
类。
3. 事务消息被检查再一段时间之后，时间是transactionTimeout配置的，当发送事务消息时用户可以改变该值通过CHECK_IMMUNITY_TIME_IN_SECONDS，该参数优先于transactionMsgTimeout。
4. 事务消息可以被检查或者消费多次。
5. 被提交的消息重新放到目标topic可能会失败。现在，取决于日志记录。RocketMQ的高可用机制保证了高可用，如果你想确保事务消息不会丢失，推荐使用同步双写机制。
6. 事务消息的id不能和其他消息类型的生产者id共享。不像其他类型的消息，事务消息可以回查。可以通过Producer IDS进行查询。

事务架构设计

事务消息的三种状态

(1) TransactionStatus.CommitTransaction: commit transaction，it means that allow consumers to consume this message.

TransactionStatus.CommitTransaction
提交状态，消费者可以消费消息。

(2) TransactionStatus.RollbackTransaction: rollback transaction，it means that the message will be deleted and not allowed to consume.

TransactionStatus.RollbackTransaction
回滚状态，消息会被删除，消息不会被消费。

(3) TransactionStatus.Unknown: intermediate state，it means that MQ is needed to check back to determine the status.

TransactionStatus.Unknown
未知状态，是一种中间状态，需要检查是否提交或者回滚。

架构设计

举一个实际案例：

1. 生产者执行本地事务，修改订单支付状态，并提交事务；
2. 生产者发送事务消息到broker上，消息发送到broker上在没有确认之前，消息对于consumer是不可见状态；
3. 生产者确认事务消息，使得发送到broker上的消息对于消费者可见；
4. 消费者获取到消息消费，完成后执行ack进行确认；

上面的流程可能存在一个问题，生产者本地事务成功后，发送事务确认消息到broker上失败了，导致消息一直是unKnown状态，消费者获取不到消息，该怎么办？
Rocket提供了消息会查机制，如果消息一直处于中间状态，broker会发起重试去查询broker上这个事务的处理状态，一旦发现事务处理成功，则把当前消息设置为可见。否则就是回滚。上面使用约束第2条可以设置check最大次数。

实践

以官网的例子：

发送事务消息

Create the transactional producer
Use TransactionMQProducer class to create producer client, and specify a unique producerGroup, and you can set up a custom thread pool to process check requests. After executing the local transaction, you need to reply to MQ according to the execution result，and the reply status is described in the above section.

使用TransactionMQProducer创建发送者对象，指定唯一的producerGroup，你也可以设置自定义的线程池来处理check请求。在执行本地事务后，需要回复mq根据事务执行结果。

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.common.message.MessageExt;
import java.util.List;

public class TransactionProducer {
    public static void main(String[] args) throws MQClientException, InterruptedException {
    
        TransactionListener transactionListener = new TransactionListenerImpl();
        //指定唯一的发送组
        TransactionMQProducer producer = new TransactionMQProducer("please_rename_unique_group_name");
        //自定义线程池处理check请求
        ExecutorService executorService = new ThreadPoolExecutor(2, 5, 100, TimeUnit.SECONDS, new                    ArrayBlockingQueue<Runnable>(2000), new ThreadFactory() {
            @Override
            public Thread newThread(Runnable r) {
                Thread thread = new Thread(r);
                thread.setName("client-transaction-msg-check-thread");
                return thread;
            }
        });

        producer.setExecutorService(executorService);
        producer.setTransactionListener(transactionListener);
        producer.start();

        String[] tags = new String[] {"TagA", "TagB", "TagC", "TagD", "TagE"};
        for (int i = 0; i < 10; i++) {
            try {
                Message msg = new Message("TopicTest1234", tags[i % tags.length], "KEY" + i,
                        ("Hello RocketMQ " + i).getBytes(RemotingHelper.DEFAULT_CHARSET));
                //发送事务消息        
                SendResult sendResult = producer.sendMessageInTransaction(msg, null);
                
                System.out.printf("%s%n", sendResult);

                Thread.sleep(10);
            } catch (MQClientException | UnsupportedEncodingException e) {
                e.printStackTrace();
            }
        }

        for (int i = 0; i < 100000; i++) {
            Thread.sleep(1000);
        }
        producer.shutdown();
    }
}

实现接口TransactionListener

Implement the TransactionListener interface
The “executeLocalTransaction” method is used to execute local transaction when send half message succeed. It returns one of three transaction status mentioned in the previous section.The “checkLocalTransaction” method is used to check the local transaction status and respond to MQ check requests. It also returns one of three transaction status mentioned in the previous section.

实现TransactionListener接口。
executeLocalTransaction方法用来执行本地事务，当发送消息成功一半，会返回三个事务状态TransactionStatus之一。
checkLocalTransaction方法用来检查本地事务状态，响应回查请求，返回三个事务状态TransactionStatus之一。

 public class TransactionListenerImpl implements TransactionListener {
 
       private AtomicInteger transactionIndex = new AtomicInteger(0);
   
       private ConcurrentHashMap<String, Integer> localTrans = new ConcurrentHashMap<>();
   
        //执行本地事务方法
       @Override
       public LocalTransactionState executeLocalTransaction(Message msg, Object arg) {
           int value = transactionIndex.getAndIncrement();
           int status = value % 3;
           localTrans.put(msg.getTransactionId(), status);
           //返回中间状态，事务消息对消费者不可见
           return LocalTransactionState.UNKNOW;
       }
   
         //回查本地事务方法，提供给broker回调
       @Override
       public LocalTransactionState checkLocalTransaction(MessageExt msg) {
           Integer status = localTrans.get(msg.getTransactionId());
           if (null != status) {
               switch (status) {
                   case 0:
                       return LocalTransactionState.UNKNOW;
                   case 1:
                       return LocalTransactionState.COMMIT_MESSAGE;
                   case 2:
                       return LocalTransactionState.ROLLBACK_MESSAGE;
               }
           }
           return LocalTransactionState.COMMIT_MESSAGE;
       }
   }