debezium部署

1.在vi /etc/my.cnf中开启binlog模式

#server-id = 223344

log_bin=mysql-bin

binlog_format=ROW

binlog_row_image=FULL

expire_logs_days=10

2.创建用户(不建议创建user用户)

CREATE USER 'user'@'localhost' IDENTIFIED BY 'password';

3.授权

GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'user' IDENTIFIED BY 'password';

4.刷新

Flush privileges;

 

5.检测binlog是否开启

SELECT variable_value as "BINARY LOGGING STATUS (log-bin) ::"

FROM information_schema.global_variables WHERE variable_name='log_bin';

6.从mysql5.7.6开始information_schema.global_status已经开始被舍弃，为了兼容性，此时需要打开 show_compatibility_56

7.Set global show_compatibility_56=on;

 

8.再检测binlog是否开启

 

9.将mysql连接器解压到kakfa/connector目录下：

下载路径：https://debezium.io/documentation/reference/1.3/install.html

 

10.将插件的父目录添加到kafka中的plugin.path中

将debezium-connector-mysql放在实际目录/kafka/connect目录下

若没有/connect目录,可以自己新建

  

    plugin.path属性修改的属性文件connect-distributed.sh

启动kafka connect的命令：

/目录/kafka/bin/connect-distributed.sh -daemon /目录g/connect-distributed.properties

 

11.配置连接mysql的binlog，新建name.json文件，配置如下:

{
  "name": "inventory-connector", #当在KafkaConnect服务注册时，连接器的名称
  "config": {
    "connector.class": "io.debezium.connector.mysql.MySqlConnector", #连接器的类名
    "database.hostname": "192.168.240.50", #MySQL服务器地址
    "database.port": "3306", #msql端口
    "database.user": "xiaoming", #账号
    "database.password": "xiaoming", #密码
    "database.server.id": "1",     #mysql的server.id
    "database.server.name": "mysql",#MySQL服务器或群集的逻辑名称
    "table.whitelist":"test.user", 
    "database.include.list": "test", #指定服务器承载的所有数据库的列表
    "database.history.kafka.bootstrap.servers": "ip1:9092,ip2:9092",#连接器用来将DDL语句写入和恢复到数据库历史主题的Kafka代理列表 
    "database.history.kafka.topic": "dbhistory.fullfillment", #数据库历史记录主题的名称
    "include.schema.changes": "true" #指定连接器是否应该在名为fulfillment可供使用者使用的带有DDL更改的事件
  }
}

配置官网链接：https://debezium.io/documentation/reference/1.3/connectors/mysql.html

12.使用curl建立连接(ip1指安装软件所在节点ip)

    --data写下json文件的目录

curl -X POST -H "Content-Type: application/json" --data "@/root/java/test.json" http://ip1:18083/connectors