Pulsar集群 监听mysql8的binlog

准备

版本 pulsar  2.9.1

mysql 8

下载连接器去找自己的版本

Index of /apache/pulsar/pulsar-2.9.1/connectors

放到指定目录

1,mysql设置

[mysqld]

log-bin=mysql-bin #添加这一行就ok

binlog-format=ROW #选择row模式

server_id=14477

2, 配置monitor-mysql.yaml

tenant: "public"
namespace: "default"
name: "debezium-mysql-source"
topicName: "debezium-mysql-topic"
archive: "data/pulsar-io-debezium-mysql-2.9.1.nar" # 上传的nar文件的存放位置

parallelism: 1

configs:
  database.hostname: "118.195.***.***"
  database.port: "3306"
  database.serverTimezone: "GMT+8"
  database.user: "root"
  database.password: "123456"
  database.server.id: "14477"
  database.server.name: "dbserver1" # 任意内容，对应查询的 topic
  database.whitelist: "delerium"  # 需要访问的数据库
  table.whitelist: "delerium.news_2022,delerium.news_2021" # 需要访问的数据表

  database.history: "org.apache.pulsar.io.debezium.PulsarDatabaseHistory"
  database.history.pulsar.topic: "my-history-topic"
  database.history.pulsar.service.url: "pulsar://118.195.***.***:6651,118.195.***.***:6652,118.195.***.***:6653"

  key.converter: "org.apache.kafka.connect.json.JsonConverter"
  value.converter: "org.apache.kafka.connect.json.JsonConverter"
  pulsar.service.url: "pulsar://118.195.***.***:6651,118.195.***.***:6652,118.195.***.***:6653"
  offset.storage.topic: "offset-topic"

3,创建source

./bin/pulsar-admin source localrun  --source-config-file sourceConfig/monitor-mysql.yaml

4,# 查看 pulsar topic

./bin/pulsar-admin topics list public/default

5,监听

./bin/pulsar-client consume -s "first-subscription" public/default/dbserver1.delerium.news_2021 -n 0

测试 insert 表一个数据

在java 上跑

代码   Pulsar集群监听mysql8的binlogjava代码-Java文档类资源-CSDN下载

参考 

CSDN  搭建pulsar集群

Pulsar IO之CDC Debezium Connector_bright_qian的博客-CSDN博客

Pulsar 集群搭建以及如何使用 debezium mysql connector_ヾ临江仙的博客-CSDN博客