spring boot集成canal实现数据库同步，且mysql主库宕机canal自动切换到从库

auto-offset-reset: earliest

是否自动提交偏移量，默认值是true,为了避免出现重复数据和数据丢失，可以把它设置为false,然后手动提交偏移量

enable-auto-commit: false

键的反序列化方式

key-deserializer: org.apache.kafka.common.serialization.StringDeserializer

值的反序列化方式

value-deserializer: org.apache.kafka.common.serialization.StringDeserializer

listener:

在侦听器容器中运行的线程数。

concurrency: 5

#listner负责ack，每调用一次，就立即commit

ack-mode: manual_immediate

missing-topics-fatal: false

CanalClient.java 包含canal的连接，数据库的监听，日志数据的处理，sql的发送

package com.zks.canal.deploy;

import com.alibaba.fastjson.JSON;

import com.alibaba.otter.canal.client.CanalConnector;

import com.alibaba.otter.canal.client.CanalConnectors;

import com.alibaba.otter.canal.protocol.CanalEntry;

import com.alibaba.otter.canal.protocol.CanalEntry.Entry;

import com.alibaba.otter.canal.protocol.Message;

import com.google.protobuf.InvalidProtocolBufferException;

import com.zks.canal.producer.KafkaProducer;

import lombok.AllArgsConstructor;

import lombok.Data;

import lombok.NoArgsConstructor;

import lombok.extern.slf4j.Slf4j;

import org.springframework.boot.ApplicationArguments;

import org.springframework.boot.ApplicationRunner;

import org.springframework.stereotype.Component;

import javax.annotation.Resource;

import java.net.InetSocketAddress;

import java.util.List;

import java.util.Queue;

import java.util.concurrent.ConcurrentLinkedQueue;

/**

• @Package: com.zks.canal.deploy

• @ClassName: CanalClient

• @Author: ZhouKaiShun

• @CreateTime: 2021/9/13 11:54

• @Description: canal客户端

*/

@Component

@Slf4j

public class CanalClient implements ApplicationRunner {

@Resource

private KafkaProducer kafkaProducer;

private final static int BATCH_SIZE = 1000;

/**

• sql队列

*/

private Queue SQL_QUEUE = new ConcurrentLinkedQueue<>();

/**

• 订阅的数据库表

*/

public static String SUBSCRIBE_DB_TABLE = “kaishun.tb_user”;

@Override

public void run(ApplicationArguments args) throws Exception {

log.info(“启动canal服务，端口号：7111”);

// 创建链接

CanalConnector connector = CanalConnectors.newSingleConne