Debeizum:Postgres to Kafka

一、环境配置

1、环境清单：
数据源：PostgreSQL 12.4
操作系统：SUSE 12 SP4 * 2台
目标端： kafka

2、安装包准备
源端PG数据库的安装这里就不赘述了，只讲Kafka和debezium。

ls -lrt
-rw-r--r-- 1 piccism nmgroup 68684719 Jan 19 15:36 kafka_2.12-2.7.0.tgz
drwxr-xr-x 7 root    root        4096 Jan 19 15:42 kafka_2.12-2.7.0
-rw-r--r-- 1 piccism nmgroup  6110424 Jan 19 16:27 debezium-connector-postgres-1.4.0.Final-plugin.tar.gz
drwxr-xr-x 2 root    root        4096 Jan 20 09:09 debezium-connector-postgres

二、KAKFA的安装

1、解压安装包，并进入

cd kafka_2.12-2.7.0/
ls -lrt
total 56
-rw-r--r-- 1 root root   337 Dec 16 21:58 NOTICE
-rw-r--r-- 1 root root 29975 Dec 16 21:58 LICENSE
drwxr-xr-x 2 root root  4096 Dec 16 22:01 site-docs
drwxr-xr-x 3 root root  4096 Jan 19 15:55 bin
drwxr-xr-x 2 root root  4096 Jan 20 09:49 libs
drwxr-xr-x 2 root root  4096 Jan 20 10:35 config
drwxr-xr-x 2 root root  4096 Jan 20 11:02 logs

2、kafka的目录主要有bin/libs/config/logs，所有的命令都在bin目录内，而且新版本的kafka内自带zookeeper，不需要另外安装

ls -lrt
total 424
-rwxr-xr-x 1 root root   1019 Dec 16 21:58 zookeeper-shell.sh
-rwxr-xr-x 1 root root   1366 Dec 16 21:58 zookeeper-server-stop.sh
-rwxr-xr-x 1 root root   1393 Dec 16 21:58 zookeeper-server-start.sh
-rwxr-xr-x 1 root root    867 Dec 16 21:58 zookeeper-security-migration.sh
drwxr-xr-x 2 root root   4096 Dec 16 21:58 windows
-rwxr-xr-x 1 root root   1714 Dec 16 21:58 trogdor.sh
-rwxr-xr-x 1 root root    958 Dec 16 21:58 kafka-verifiable-producer.sh
-rwxr-xr-x 1 root root    958 Dec 16 21:58 kafka-verifiable-consumer.sh
-rwxr-xr-x 1 root root    863 Dec 16 21:58 kafka-topics.sh
-rwxr-xr-x 1 root root    945 Dec 16 21:58 kafka-streams-application-reset.sh
-rwxr-xr-x 1 root root   1361 Dec 16 21:58 kafka-server-stop.sh
-rwxr-xr-x 1 root root   1376 Dec 16 21:58 kafka-server-start.sh
-rwxr-xr-x 1 root root  10072 Dec 16 21:58 kafka-run-class.sh
-rwxr-xr-x 1 root root    874 Dec 16 21:58 kafka-replica-verification.sh
-rwxr-xr-x 1 root root    874 Dec 16 21:58 kafka-reassign-partitions.sh
-rwxr-xr-x 1 root root    959 Dec 16 21:58 kafka-producer-perf-test.sh
-rwxr-xr-x 1 root root    886 Dec 16 21:58 kafka-preferred-replica-election.sh
-rwxr-xr-x 1 root root    862 Dec 16 21:58 kafka