maxwell安装部署

最新推荐文章于 2024-07-09 13:39:23 发布

潇洒-人生

最新推荐文章于 2024-07-09 13:39:23 发布

阅读量1.7k

点赞数

分类专栏：大数据 mysql maxwell kafka

本文链接：https://blog.csdn.net/qq_35744460/article/details/90674639

版权

大数据同时被 3 个专栏收录

45 篇文章 0 订阅

订阅专栏

mysql

3 篇文章 0 订阅

订阅专栏

maxwell

1 篇文章 0 订阅

订阅专栏

部署

maxwell

wget https://github.com/zendesk/maxwell/releases/download/v1.14.1/maxwell-1.14.1.tar.gz 	

tar -zxvf maxwell-1.14.1.tar.gz

mysql

vi my.cnf
server_id=1
log-bin=master
binlog_format=row
mysql> create database maxwell;
mysql> CREATE USER 'maxwell'@'%' IDENTIFIED BY '123456';
mysql> GRANT ALL ON maxwell.* TO 'maxwell'@'%';
mysql> GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE ON *.* TO 'maxwell'@'%';
mysql> flush privileges;

kafka
kafka client 对应的jar copy 到 maxwell-1.14.1/lib/kafka-clients 目录下

cp kafka-clients-0.10.2-kafka-2.2.0.jar /home/bigdata/maxwell/maxwell-1.14.1/lib/kafka-clients/

启动maxwell

bin/maxwell --user='maxwell' --password='123456' --host='spark003' \
   --producer=kafka --kafka.bootstrap.servers=spark003:9092,spark004:9092,spark005:9092 --kafka_topic=maxwell \
   --kafka_version=0.10.2-kafka-2.2.0 --client_id=1 --output_ddl=true

启动kafka消费

/opt/cloudera/parcels/KAFKA-2.2.0-1.2.2.0.p0.68/lib/kafka/bin/kafka-console-consumer.sh --bootstrap-server spark003:9092,spark004:9092,spark005:9092 --from-beginning --topic maxwell

结果

{"type":"table-create","database":"bigdata","table":"role","def":{"database":"bigdata","charset":"utf8","table":"role","primary-key":["r":"role_id","signed":true}]},"ts":1559118577000,"sql":"CREATE TABLE `role` (\r\n`role_id`  int NOT NULL ,\r\nPRIMARY KEY (`role_id`)\r\n
{"database":"bigdata","table":"role","type":"update","ts":1559118348,"xid":956,"commit":true,"data":{"role_id":1},"old":
{"type":"table-alter","database":"bigdata","table":"role","old":{"database":"bigdata","charset":"utf8","table":"role","primary-key":["ro:"role_id","signed":true}]},"def":{"database":"bigdata","charset":"utf8","table":"role","primary-key":["role_id"],"columns":[{"type":"inpe":"varchar","name":"role_name","charset":"utf8"}]},"ts":1559118626000,"sql":"ALTER TABLE `role`\r\nADD COLUMN `role_name`  varchar(255
{"type":"table-alter","database":"bigdata","table":"role","old":{"database":"bigdata","charset":"utf8","table":"role","primary-key":["ro:"role_id","signed":true},{"type":"varchar","name":"role_name","charset":"utf8"}]},"def":{"database":"bigdata","charset":"utf8","table":ns":[{"type":"int","name":"role_id","signed":true}]},"ts":1559118683000,"sql":"ALTER TABLE `role`\r\nDROP COLUMN `role_name`"}
{"type":"table-drop","database":"bigdata","table":"role","ts":1559118700000,"sql":"DROP TABLE `role` /* generated by server */"}