基于日志增量订阅和消费的业务包括:
数据库镜像
数据库实时备份
索引构建和实时维护(拆分异构索引、倒排索引等)
业务 cache 刷新
带业务逻辑的增量数据处理
当前的 canal 支持源端 MySQL 版本包括 5.1.x , 5.5.x , 5.6.x , 5.7.x , 8.0.x
一 实验环境
二 实验步骤
2.1 配置源端MariaDB
2.1.1 修改配置文件
my.cnf 中配置如下:
[mysqld]
log-bin=mysql-bin
binlog-format=ROW
server_id=1
2.1.2 为canal创建专用账号
CREATE USER CanalUser@'192.168.144.242' IDENTIFIED BY '密码';
GRANT ALL PRIVILEGES ON *.* TO 'CanalUser'@'192.168.144.242';
FLUSH PRIVILEGES;
2.2 下载安装canal
2.2.1 下载
下载 canal, 访问 release 页面 , 选择需要的包下载, 如以 1.1.4 版本为例:
打开网页后,往下拉,找到1.1.4:
再往下拉,展开Asserts:
cd /opt && mkdir canal && cd canal
将安装包上传到/opt/canal路径下。
2.2.2 安装jdk
可参考https://blog.csdn.net/yabingshi_tech/article/details/51498017
2.2.3 安装canal
mkdir /usr/local/canal
tar zxvf canal.deployer-1.1.4.tar.gz -C /usr/local/canal
cd /usr/local/canal
[root@localhost canal]# ll
总用量 4
drwxr-xr-x. 2 root root 76 11月 5 13:28 bin
drwxr-xr-x. 5 root root 123 11月 5 13:28 conf
drwxr-xr-x. 2 root root 4096 11月 5 13:28 lib
drwxrwxrwx. 2 root root 6 9月 2 2019 logs
2.2.4 修改配置文件
cd conf/example/
mv instance.properties instance.properties_bak_20201105
vi instance.properties
添加:
#不能与mysql的server_id重复
canal.instance.mysql.slaveId = 1234
#position info
canal.instance.master.address = 192.168.144.241:3306
canal.instance.master.journal.name =
canal.instance.master.position =
canal.instance.master.timestamp =
#canal.instance.standby.address =
#canal.instance.standby.journal.name =
#canal.instance.standby.position =
#canal.instance.standby.timestamp =
#username/password,需要改成自己的数据库信息
canal.instance.dbUsername = CanalUser
canal.instance.dbPassword = 密码
canal.instance.defaultDatabaseName =
canal.instance.connectionCharset = UTF-8
#table regex
canal.instance.filter.regex = .\*\\\\..\*
2.3 启动canal
2.3.1 启动
sh /usr/local/canal/bin/startup.sh
#对应的关闭命令是:
sh /usr/local/canal/bin/stop.sh
2.3.2 检查日志是否有报错
下面两个日志必须都是成功启动的状态:
#检查server 日志
vi /usr/local/canal/logs/canal/canal.log
2013-02-05 22:45:27.967 [main] INFO com.alibaba.otter.canal.deployer.CanalLauncher - ## start the canal server.
2013-02-05 22:45:28.113 [main] INFO com.alibaba.otter.canal.deployer.CanalController - ## start the canal server[10.1.29.120:11111]
2013-02-05 22:45:28.210 [main] INFO com.alibaba.otter.canal.deployer.CanalLauncher - ## the canal server is running now ......
#检查instance 的日志
vi /usr/local/canal/logs/example/example.log
2013-02-05 22:50:45.636 [main] INFO c.a.o.c.i.spring.support.PropertyPlaceholderConfigurer - Loading properties file from class path resource [canal.properties]
2013-02-05 22:50:45.641 [main] INFO c.a.o.c.i.spring.support.PropertyPlaceholderConfigurer - Loading properties file from class path resource [example/instance.properties]
2013-02-05 22:50:45.803 [main] INFO c.a.otter.canal.instance.spring.CanalInstanceWithSpring - start CannalInstance for 1-example
2013-02-05 22:50:45.810 [main] INFO c.a.otter.canal.instance.spring.CanalInstanceWithSpring - start successful....
--canal-admin管理界面安装请参考官网:https://github.com/alibaba/canal/wiki/Canal-Admin-QuickStart
--本篇文章主要参考自https://github.com/alibaba/canal/wiki/QuickStart