启动Zookeeper
> bin/zookeeper-server-start.sh -daemon ../config/zookeeper.properties
启动kafka
> bin/kafka-server-start.sh -daemon ../config/server.properties
创建topic(一个副本一个分区)
> bin/kafka-topics.sh --list --zookeeper localhost:2181
查看topic的详细信息
> bin/kafka-topics.sh -zookeeper localhost:2181 -describe -topic test
为topic增加partition
> bin/kafka-topics.sh –zookeeper localhost:2181 –alter –partitions 20 –topic test
发送消息
> bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
This is a message
This is another message
启动消费者
> bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test--from-beginning
This is a message
This is another message
删除topic
> bin/kafka-run-class.sh kafka.admin.DeleteTopicCommand --topic test --zookeeper localhost2181
> bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic test
查看consumer组内消费的offset
> bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zookeeper localhost:2181 --group test --topic test
> bin/kafka-consumer-offset-checker.sh --zookeeper localhost:2181 --group group1 --topic group1
> bin/zookeeper-server-start.sh -daemon ../config/zookeeper.properties
启动kafka
> bin/kafka-server-start.sh -daemon ../config/server.properties
创建topic(一个副本一个分区)
> bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
zookeeper参数是必须参数,用于配置kafka集群与Zookeeper连接地址,这里并不要求传递${zookeeper.connect}配置说的所有连接地址,为了容错,建议多个zookeeper节点的集群至少传递2个zookeeper连接配置,多个配置之间用逗号隔开
partiton参数用于设置分区数,该配置为必须参数,kafka使用分区分配策略,将一个topic的消息,分散到多个分区并分别保存在不同的代理上,以此来提高消息的吞吐量。
replication-factor 用来设置topic的副本数。必须参数,副本会被分配在不同的节点上,副本数不能超过节点数,否则创建topic会失败,例如3个节点的kafka集群最多能创建3个副本,若创建topic时指定副本数大于3个,则会抛出错误提示。
> bin/kafka-topics.sh --list --zookeeper localhost:2181
查看topic的详细信息
> bin/kafka-topics.sh -zookeeper localhost:2181 -describe -topic test
为topic增加partition
> bin/kafka-topics.sh –zookeeper localhost:2181 –alter –partitions 20 –topic test
发送消息
> bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
This is a message
This is another message
启动消费者
> bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test--from-beginning
This is a message
This is another message
删除topic
> bin/kafka-run-class.sh kafka.admin.DeleteTopicCommand --topic test --zookeeper localhost2181
> bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic test
查看consumer组内消费的offset
> bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zookeeper localhost:2181 --group test --topic test
> bin/kafka-consumer-offset-checker.sh --zookeeper localhost:2181 --group group1 --topic group1