使用canal通过mysql复制协议从binlog实现热数据nosql缓存（2）

开启mysql binlog功能

以5.7版本为例，找到/etc/mysql/mysql.conf.d/mysqld.cnf

[mysqld]
pid-file        = /var/run/mysqld/mysqld.pid
socket          = /var/run/mysqld/mysqld.sock
datadir         = /var/lib/mysql
#log-error      = /var/log/mysql/error.log
# By default we only accept connections from localhost
#bind-address   = 127.0.0.1
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

# 开启binlog
log-bin=mysql-bin 
binlog-format=ROW
server_id=1	
# 开启binlog

配置完毕重启mysql服务，重启完成后，查看是否已经成功开启

show variables like 'log_%';

 

canal-server服务

点击下载ali-canal 配置canal-server，修改\canal.deployer-1.0.22\conf\example\instance.properties

## mysql serverId 务必与master数据库不同
canal.instance.mysql.slaveId = 1234

需要追踪复制的master数据库信息
canal.instance.master.address = 127.0.0.1:3307
canal.instance.master.journal.name = 
canal.instance.master.position = 
canal.instance.master.timestamp = 

#canal.instance.standby.address = 
#canal.instance.standby.journal.name =
#canal.instance.standby.position = 
#canal.instance.standby.timestamp = 

username/password
canal.instance.dbUsername = root
canal.instance.dbPassword = root
canal.instance.defaultDatabaseName = test
canal.instance.connectionCharset = UTF-8

启动项目
\canal.deployer-1.0.22\bin\

 

canal-client服务

https://github.com/wyfteam123/mysqlbinlog-canal-nosql/tree/master/canal-client
修改对应配置，使用maven打包启动项目。

 

mq-consummer服务

java实现
https://github.com/wyfteam123/mysqlbinlog-canal-nosql/tree/master/sync-server/java_sync_nosql
或
python实现
https://github.com/wyfteam123/mysqlbinlog-canal-nosql/tree/master/sync-server/python_sync_nosql

 

rabbit/redis/mysql

推荐docker容器安装

转载于:https://www.cnblogs.com/7motor28/p/10299710.html