3、Canal-配置类-dynamicTopic详细解

这是Canal instance实例中的动态topic配置,我们在分库分表等需要特殊映射的场景下会用到此配置，这点在作者官方文档说明的不是很清楚网上的文档也比较乱七八糟尝试过几种配置都不行，我尝试通过代码调试的方式一点点摸索出规律，下面给总结下。
​

# 这块要精确过滤，防止找不到topic
canal.instance.filter.regex=tms\\.tms_station,tms\\.tms_station_delivery_area,tms_0\\.tms_delivery_order_\\d+_q\\d+,tms_0\\.tms_delivery_order_detail_\\d+_q\\d+,tms_0\\.tms_delivery_order_settlement_\\d+_q\\d+
# table black regex
canal.instance.filter.black.regex=
# table field filter(format: schema1.tableName1:field1/field2,schema2.tableName2:field1/field2)
#canal.instance.filter.field=test1.t_product:id/subject/keywords,test2.t_company:id/name/contact/ch
# table field black filter(format: schema1.tableName1:field1/field2,schema2.tableName2:field1/field2)
#canal.instance.filter.black.field=test1.t_product:subject/product_image,test2.t_company:id/name/contact/ch

# mq config
#canal.mq.topic=
# dynamic topic route by schema or table regex 
canal.mq.dynamicTopic=canal_tms_station_v1:tms\\.tms_station,canal_tms_station_delivery_area_v1:tms\\.tms_station_delivery_area,canal_tms_delivery_order_v1:tms_0\\.tms_delivery_order_\\d+_q\\d+,canal_tms_delivery_order_detail_v1:tms_0\\.tms_delivery_order_detail_\\d+_q\\d+,canal_tms_delivery_order_settlement_v1:tms_0\\.tms_delivery_order_settlement_\\d+_q\\d+
canal.mq.partition=0   

动态topic的源码在MQMessageUtils.java这个文件中。这个类中有dynamicTopic这个关键字的代码都给打断点，然后调试慢慢找出规律。下面贴出一段关键的入口代码。

/**
 * 按 schema 或者 schema+table 将 message 分配到对应topic
 * @param message 原message
 * @param defaultTopic 默认topic
 * @param dynamicTopicConfigs 动态topic规则
 * @return 分隔后的message map
 */  
public static Map<String, Message> messageTopics(Message message, String defaultTopic, String dynamicTopicConfigs) {
        List<CanalEntry.Entry> entries;
        if (message.isRaw()) {
            List<ByteString> rawEntries = message.getRawEntries();
            entries = new ArrayList<>(rawEntries.size());
            for (ByteString byteString : rawEntries) {
                CanalEntry.Entry entry;
                try {
                    entry = CanalEntry.Entry.parseFrom(byteString);
                } catch (InvalidProtocolBufferException e) {
                    throw new RuntimeException(e);
                }
                entries.add(entry);
            }
        } else {
            entries = message.getEntries();
        }
        Map<String, Message> messages = new HashMap<>();
        for (CanalEntry.Entry entry : entries) {
            if (entry.getEntryType() == CanalEntry.EntryType.TRANSACTIONBEGIN
                || entry.getEntryType() == CanalEntry.EntryType.TRANSACTIONEND) {
                continue;
            }

            String schemaName = entry.getHeader().getSchemaName();
            String tableName = entry.getHeader().getTableName();

            //注意点binlog一定要传递 schemaName、tableName不然不生效
            if (StringUtils.isEmpty(schemaName) || StringUtils.isEmpty(tableName)) {
                put2MapMessage(messages, message.getId(), defaultTopic, entry);
            } else {
                Set<String> topics = matchTopics(schemaName + "." + tableName, dynamicTopicConfigs);
                if (topics != null) {
                    for (String topic : topics) {
                        put2MapMessage(messages, message.getId(), topic, entry);
                    }
                } else {
                    topics = matchTopics(schemaName, dynamicTopicConfigs);
                    if (topics != null) {
                        for (String topic : topics) {
                            put2MapMessage(messages, message.getId(), topic, entry);
                        }
                    } else {
                        put2MapMessage(messages, message.getId(), defaultTopic, entry);
                    }
                }
            }
        }
        return messages;
    }