canal同步es数据到mysql

参考地址
碰到的bug
第一次按照启动canal.adapter的时候报错，经典空指针异常，代码如下：

2024-08-03 22:29:11.937 [pool-3-thread-1] ERROR c.a.otter.canal.adapter.launcher.loader.AdapterProcessor - java.lang.NullPointerException
java.lang.RuntimeException: java.lang.NullPointerException
	at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:112)
	at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:60)
	at com.alibaba.otter.canal.client.adapter.es.core.ESAdapter.sync(ESAdapter.java:104)
	at com.alibaba.otter.canal.client.adapter.es.core.ESAdapter.sync(ESAdapter.java:83)
	at com.alibaba.otter.canal.client.adapter.ProxyOuterAdapter.sync(ProxyOuterAdapter.java:42)
	at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.batchSync(AdapterProcessor.java:139)
	at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.lambda$null$1(AdapterProcessor.java:97)
	at java.util.concurrent.CopyOnWriteArrayList.forEach(CopyOnWriteArrayList.java:890)
	at com.alibaba.otter.canal.adapter.launcher.loader.AdapterProcessor.lambda$null$2(AdapterProcessor.java:94)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: null
	at com.alibaba.otter.canal.client.adapter.es7x.support.ES7xTemplate.insert(ES7xTemplate.java:70)
	at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.singleTableSimpleFiledInsert(ESSyncService.java:448)
	at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.insert(ESSyncService.java:135)
	at com.alibaba.otter.canal.client.adapter.es.core.service.ESSyncService.sync(ESSyncService.java:95)

看了官网和网上各种文档都没找到原因，于是就取找canal报错位置的源码，打印内容重新打包，使用这个jar

   @Override
    public Object getValFromData(ESMapping mapping, Map<String, Object> dmlData, String fieldName, String columnName) {
        String esType = getEsType(mapping, fieldName);
        logger.info("gwb1dmlData==="+dmlData);
        Object value = dmlData.get(columnName);
        logger.info("gwb1===");
        logger.info("value1:"+columnName+value);
        if (value instanceof Byte) {
            if ("boolean".equals(esType)) {
                value = ((Byte) value).intValue() != 0;
            }
        }

日志中发现从mysql数据库中查到的内容和mysql的字段没有对应，数据库中的主键是id，但是canal查出来的竟然是article_id

CREATE TABLE `test_es_articles` (
  `id` INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
  `user_id` INT UNSIGNED,
  `article_title` VARCHAR(255) NOT NULL,
  `article_content` TEXT,
  `publish_date` DATETIME DEFAULT CURRENT_TIMESTAMP
)  ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4 COMMENT='文章表';

2024-08-03 21:55:31.247 [pool-3-thread-1] INFO  c.a.o.canal.client.adapter.es.core.support.ESTemplate - gwb1dmlData==={article_id=1, user_id=1, article_title=小黑子1, article_content=这是文章1的内容, publish_date=2024-07-01 10:00:00.0}
2024-08-03 21:55:31.247 [pool-3-thread-1] INFO  c.a.o.canal.client.adapter.es.core.support.ESTemplate - gwb1===
2024-08-03 21:55:31.247 [pool-3-thread-1] INFO  c.a.o.canal.client.adapter.es.core.support.ESTemplate - value1:id

所以导致在查询id的时候找不到,导致报空指针异常，就很蒙比，我在想会不会是从mysql里面查的不是最新的表结构，重启mysql，重启canal

2024-08-03 22:59:54.787 [pool-3-thread-1] INFO  c.a.o.canal.client.adapter.logger.LoggerAdapterExample - DML: {"data":[{"id":5,"canal_content":"这是一条示例数据，包含一些文本内容。"},{"id":6,"canal_content":"这是第二条示例数据，同样包含文本信息。"},{"id":7,"canal_content":"示例数据三，继续包含文本，用于测试同步和索引。"},{"id":8,"canal_content":"第四条数据，展示了不同长度的文本内容。"}],"database":"ry-vue","destination":"example","es":1722697194000,"groupId":"g1","isDdl":false,"old":null,"pkNames":["id"],"sql":"","table":"test_canal","ts":1722697194494,"type":"INSERT"}
2024-08-03 22:59:54.787 [pool-3-thread-1] INFO  c.a.o.canal.client.adapter.es.core.support.ESTemplate - 字段名：[com.alibaba.otter.canal.client.adapter.es.core.config.SchemaItem$FieldItem@d1b, com.alibaba.otter.canal.client.adapter.es.core.config.SchemaItem$FieldItem@3398835]
2024-08-03 22:59:54.919 [pool-3-thread-1] WARN  org.elasticsearch.client.RestClient - request [GET http://127.0.0.1:9200/test_canal/_mapping?master_timeout=30s&ignore_throttled=false&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true] returned 2 warnings: [299 Elasticsearch-7.17.3-5ad023604c8d7416c9eb6c0eadb62b14e766caff "Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.17/security-minimal-setup.html to enable security."],[299 Elasticsearch-7.17.3-5ad023604c8d7416c9eb6c0eadb62b14e766caff "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."]
2024-08-03 22:59:55.093 [pool-3-thread-1] INFO  c.a.o.canal.client.adapter.es.core.support.ESTemplate - gwb1dmlData==={id=5, canal_content=这是一条示例数据，包含一些文本内容。}
2024-08-03 22:59:55.093 [pool-3-thread-1] INFO  c.a.o.canal.client.adapter.es.core.support.ESTemplate - gwb1===
2024-08-03 22:59:55.093 [pool-3-thread-1] INFO  c.a.o.canal.client.adapter.es.core.support.ESTemplate - value1:id5
2024-08-03 22:59:55.093 [pool-3-thread-1] INFO  c.a.o.canal.client.adapter.es.core.support.ESTemplate - gwb2===
2024-08-03 22:59:55.093 [pool-3-thread-1] INFO  c.a.o.canal.client.adapter.es.core.support.ESTemplate - value2:5
2024-08-03 22:59:55.093 [pool-3-thread-1] INFO  c.a.o.canal.client.adapter.es.core.support.ESTemplate - gwb1dmlData==={id=5, canal_content=这是一条示例数据，包含一些文本内容。}

ok啦铁子，没毛病，但是我不能笃定问题在这，于是我有重新创建了es索引和mysql表，然后同步mysql数据到es中

CREATE TABLE `test_canal` (
  `id` INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
  `canal_content` TEXT,
)  ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8mb4 COMMENT='canal测试表';

PUT /test_canal
{
  "settings": {
    "analysis": {
      "analyzer": {
        "ik_analyzer": {
          "type": "custom",
          "tokenizer": "ik_smart"
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "id": { "type": "integer" },
      "canal_content": {
        "type": "text"
      }
    }
  }
}

再往mysql中同步数据，发现数据已经同步到es所以，上面参考文档没问题，但是我们最好在使用canal前重启一下mysql