1、启动
kafka和zk一样,都是启动命令调用配置文件,一般命令加&,放在后台执行。
first:bin/zookeeper-server-start.sh config/zookeeper.properties &
Second:bin/kafka-server-start.sh config/server.properties &
2、创建topic
创建topic时需要指定副本和分区,副本数不能大于Broker数量,分区数根据实际需求,不是越多越好。
bin/kafka-topic.sh --create --zookeeper zk1:2181,zk2:2181,zk3:2181 --replication-factor 3 --partitions 3 --topic test
3、查看topic
可以查看topic列表及描述,describe中能看到分区、副本及ISR信息。
bin/kafka-topic.sh --describe/list --zookeeper zk1:2181,zk2:2181,zk3:2181 --topic test
4、删除topic
直接删除topic,包含数据文件、zk节点,注意server.properties中要开启 delete.topic.enable=true。
bin/kafka-topic.sh --delete --zookeeper zk1:2181,zk2:2181,zk3:2181 --topic test
5、生产消息
生产可以命令中指定参数,也可以调用配置文件。
bin/kafka-console-producer.sh --broker-list broker1:9092,broker2:9092,broker3:9092 --topic test --producer.config config/producer.properties
6、消费消息
消费可以命令中指定参数,也可以调用配置文件。
bin/kafka-console-consumer.sh --zookeeper zk1:2181,zk2:2181,zk3:2181 --topic test --from-beginning --consumer.config config/consumer.properties
7、查看offset
0.8之后的版本命令做了修改,可以查看消费的offset值。
bin/kafka-consumer-offset-checker.sh --zookeeper zk1:2181,zk2:2181,zk3:2181 --topic test --group TestGroup (——0.8)
bin/kafka-run-class.sh kafka.tools.GetOffsetShell --topic test --broker-list broker1:9092, broker2:9092, broker3:9092 --time -1 (0.9——)
8、查看ConsumerGroup
显示当前所有消费组。
bin/kafka-consumer-groups.sh --list --zookeeper zk1:2181,zk2:2181,zk3:2181
9、查看组实时消费情况
显示topic、分区、消息量、offset、消费延迟、消费id等信息。
bin/kafka-consumer-groups.sh --describe --zookeeper zk1:2181,zk2:2181,zk3:2181 --group console-consumer-xxxxx
10、增加分区
动态增加分区,分区数字为增加到的值。
bin/kafka-topics.sh --alter --zookeeper zk1:2181,zk2:2181,zk3:2181 --topic test --partitions 7
11、重新分配分区
先创建一个json文件,格式如下,指定分区,执行--topics-to-move-json-file test.json,会显示当前分区情况和推荐分区方式。新建一个重新分区的json文件,可以参考推荐分区方式,执行--reassignment-json-file,完成分区,可以把分区调整到任意副本,一般用于Broker迁移,这属于强制指定。执行--verify 验证重分区结果。
[root@SZB-L0047709 kafka_2.10-0.10.2.0]# less test.json
{"topics": [{"topic": "test3"}], "version":1 }
[root@SZB-L0047709 kafka_2.10-0.10.2.0]# bin/kafka-reassign-partitions.sh --topics-to-move-json-file test.json --zookeeper zk1:2181,zk2:2181,zk3:2181 --broker-list “0,1,2" --generate
Current partition replica assignment
{"version":1,"partitions":[{"topic":"test3","partition":0,"replicas":[2]},{"topic":"test3","partition":1,"replicas":[0]},{"topic":"test3","partition":2,"replicas":[1]}]}
Proposed partition reassignment configuration
{"version":1,"partitions":[{"topic":"test3","partition":0,"replicas":[0]},{"topic":"test3","partition":1,"replicas":[0]},{"topic":"test3","partition":2,"replicas":[0]}]}
[root@SZB-L0047709 kafka_2.10-0.10.2.0]# bin/kafka-reassign-partitions.sh --execute --reassignment-json-file reassign.json --zookeeper zk1:2181,zk2:2181,zk3:2181
Current partition replica assignment
{"version":1,"partitions":[{"topic":"test3","partition":0,"replicas":[2]},{"topic":"test3","partition":1,"replicas":[0]},{"topic":"test3","partition":2,"replicas":[1]}]}
Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions.
[root@SZB-L0047709 kafka_2.10-0.10.2.0]# bin/kafka-reassign-partitions.sh --verify --reassignment-json-file reassign.json --zookeeper zk1:2181,zk2:2181,zk3:2181
Status of partition reassignment:
Reassignment of partition [test3,0] completed successfully
Reassignment of partition [test3,1] completed successfully
Reassignment of partition [test3,2] completed successfully
12、Producer压测命令
bin/kafka-producer-perf-test.sh --topic well --num-records 100 --record-size 1 --throughput 100 --producer-props bootstrap.servers=broker1:9092,broker2:9092,broker3:9092
13、Consumer压测命令
bin/kafka-consumer-perf-test.sh --topic well --messages 1000 --broker-list broker1:9092,broker2:9092,broker3:9092
14、Zookeeper常用命令
zkCli.sh –server 127.0.0.1:2181 —— 登录客户端
ls /znode、get /znode 、create /znode 、set /znode ……
echo stat |nc 127.0.0.1 2181 —— 四字命令