debezium+kafka connect的mysql增量同步解决方案

kafka_home: /server/kafka/kafka_2.12-2.5.0 (kafka主目录截图如下)

启动kafka：

cd kafka_home(kafka主目录)
bin/zookeeper-server-start.sh - daemon config/zookeeper.properties (启动zookepper)

bin/kafka-server-start.sh - daemon config/server.properties (启动kafka)

集成debezium到kafka connect

vi conf/mysql.properties
name=student-connector //kafka connect name
connector.class=io.debezium.connector.mysql.MySqlConnector //connect 执行类
tasks.max=1
database.hostname=192.168.1.251
database.port=3306
database.user=root
database.password=root
database.server.id=184054
database.server.name=dbserver1
database.whitelist=student
database.history.kafka.bootstrap.servers=localhost:9092
database.history.kafka.topic=schema-changes.student
database.serverTimezone=CTT //mysql7+强制指定时区
//database.tinyInt1isBit=false is passed to the JDBC URL. Jdbc url参数可以通过//database.作为前缀进行传给debezium

mkdir kafka_connect
解压 debezium-connector-mysql-1.2.0.Final-plugin.tar.gz到kafka_connect目录下

配置connect-standalone.properties
plugin.path=/server/kafka/kafka_2.12-2.5.0/kafka_connect/debezium-connector-mysql 指定插件位置。

启动kafka connect
./bin/connect-standalone.sh ./config/connect-standalone.properties
./config/mysql.properties
可能会报类没有发现异常。此时需要将插件导入到CLASSPATH环境变量中
export CLASSPATH=/server/kafka/kafka_2.12-2.5.0/kafka_connect/debezium-connector-mysql/*;

kafka官网
http:// kafka.apache.org/
Debezium官网：
https://debezium.io/documentation/reference/1.2/connectors/mysql.html