docker-compose 安装 canal

docker-compose 安装 canal

canal 同步特点:

• 支持 修改，删除，创建等等语句
• logstash_jdbc 只支持 修改，创建， 不支持删除

配置 mysql my.cnf

# 开启 binlog
log-bin=mysql-bin 

# 选择 ROW 模式
binlog-format=ROW 

# 配置 MySQL replaction 需要定义，不要和 canal 的 slaveId 重复
server_id=1 

创建 复制账号

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 |      156 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set

mysql> CREATE USER canal IDENTIFIED BY 'canal'; 
Query OK, 0 rows affected

mysql> GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%';
Query OK, 0 rows affected

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected

mysql> 

安装 canal-server

docker-compose.yml 配置文件

[root@blue canal]# cat docker-compose.yml 
version: '3'
 
services: 
  canal-server: 
    image: canal/canal-server:v1.1.5
    container_name: canal-server
    ports: 
      - 11111:11111
    environment: 
      - canal.instance.mysql.slaveId=1234
      - canal.auto.scan=false
      - canal.destinations=test
      - canal.instance.master.address=172.16.xx:3306
      # mysql 复制账号
      - canal.instance.dbUsername=canal
      - canal.instance.dbPassword=canal
      - canal.mq.topic=test
      - canal.instance.filter.regex=esen_approval.apt_approval
    volumes: 
      - $PWD/conf/:/admin/canal-server/conf/
      - $PWD/logs/:/admin/canal-server/logs/

172.16.xx:3306 这个是docker 容器启动的mysql, 注意虚拟机关闭防火墙，不然 canal 连接不上mysql !!

Creating canal-server ... done
Attaching to canal-server
canal-server    | DOCKER_DEPLOY_TYPE=VM
canal-server    | ==> INIT /alidata/init/02init-sshd.sh
canal-server    | ==> EXIT CODE: 0
canal-server    | ==> INIT /alidata/init/fix-hosts.py
canal-server    | ==> EXIT CODE: 0
canal-server    | ==> INIT DEFAULT
canal-server    | Generating SSH1 RSA host key: [  OK  ]
canal-server    | Starting sshd: [  OK  ]
canal-server    | Starting crond: [  OK  ]
canal-server    | ==> INIT DONE
canal-server    | ==> RUN /home/admin/app.sh
canal-server    | ==> START ...
canal-server    | start canal ...
canal-server    | start canal successful
canal-server    | ==> START SUCCESSFUL ...

如果发现后期canal 接收不到 mysql 数据， 进入容器看日志！！，可以是连接不上mysql !!

测试

测试代码: canal demo 代码

================>; binlog[mysql-bin.000002:1123] , name[logstash,t_user] , eventType : UPDATE
------->; before
id : 34    update=false
user_name : hello222ewe    update=false
age : 577    update=false
create_time : 2021-08-11 15:30:58    update=false
update_time : 2021-08-11 15:31:32    update=false
------->; after
id : 34    update=false
user_name : ha    update=true
age : 577    update=false
create_time : 2021-08-11 15:30:58    update=false
update_time : 2021-08-11 15:31:32    update=false

================>; binlog[mysql-bin.000002:1472] , name[logstash,t_user] , eventType : INSERT
id : 38    update=true
user_name : sd    update=true
age :     update=true
create_time :     update=true
update_time :     update=true

================>; binlog[mysql-bin.000002:1774] , name[logstash,t_user] , eventType : DELETE
id : 38    update=false
user_name : sd    update=false
age :     update=false
create_time :     update=false
update_time :     update=false

good luck!