Zookeeper+kafka集群环境搭建

Zookeeper+kafka集群环境搭建

版本号：zookeeper-3.4.6、kafka_2.11-0.9.0.0

环境地址：127.0.0.1/127.0.0.2

zookeeper安装路径：/opt/zookeeper-3.4.6

1.     解压zookeeper-3.4.6.tar.gz

2.     /opt/zookeeper-3.4.6/conf/ zoo_sample.cfg复制一份更名为zoo.cfg

3.     修改zoo.cfg配置信息如下

dataDir= /tmp/zookeeper（数据目录）

clientPort=2181

server.63=127.0.0.1:2888:3888

server.64=127.0.0.2:2888:3888

4.     127.0.0.1机器 /tmp/zookeeper下新增文件myid，文件内容63

127.0.0.2机器 /tmp/zookeeper下新增文件myid，文件内容64

5.     启动63、64环境

bin/zkServer.sh start  

 

kafka安装路径：/opt/kafka_2.11-0.9.0.0/

6.     解压kafka_2.11-0.9.0.0.tgz

7.     64环境修改/opt/kafka_2.11-0.9.0.0/config/ server.properties如下

broker.id=64

log.dirs=/tmp/kafka-logs

zookeeper.connect=127.0.0.1:2181,127.0.0.2:2181

advertised.host.name=127.0.0.2

advertised.port=9092

8.     63环境修改/opt/kafka_2.11-0.9.0.0/config/ server.properties如下

broker.id=63

log.dirs=/tmp/kafka-logs

zookeeper.connect=127.0.0.1:2181,127.0.0.2:2181

advertised.host.name=127.0.0.2

advertised.port=9092

9.     启动kafka

./kafka-server-start.sh-daemon ../config/server.properties

10. 创建topic

./kafka-topics.sh --create--zookeeper 127.0.0.1:2181, 127.0.0.2:2181 --replication-factor 2 --partitions1 --topic test_yyy

11. 查看

./kafka-topics.sh --list--zookeeper  127.0.0.1:2181,127.0.0.2:2181

查看明细

./kafka-topics.sh--describe --zookeeper  127.0.0.1:2181, 127.0.0.2:2181--topic test_yyy

消费

./kafka-console-consumer.sh-zookeeper 127.0.0.1:2181, 127.0.0.2:2181 --from-beginning --topic test_yyy

 

 

 

0.查看有哪些主题： ./kafka-topics.sh --list --zookeeper 192.168.0.201:12181   1.查看topic的详细信息 ./kafka-topics.sh -zookeeper 127.0.0.1:2181 -describe -topic testKJ1   2、为topic增加副本 ./kafka-reassign-partitions.sh -zookeeper 127.0.0.1:2181 -reassignment-json-file json/partitions-to-move.json -execute   3、创建topic ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic testKJ1   4、为topic增加partition ./bin/kafka-topics.sh –zookeeper 127.0.0.1:2181 –alter –partitions 20 –topic testKJ1   5、kafka生产者客户端命令 ./kafka-console-producer.sh --broker-list localhost:9092 --topic testKJ1   6、kafka消费者客户端命令 ./kafka-console-consumer.sh -zookeeper localhost:2181 --from-beginning --topic testKJ1     7、kafka服务启动 ./kafka-server-start.sh -daemon ../config/server.properties     8、下线broker ./kafka-run-class.sh kafka.admin.ShutdownBroker --zookeeper 127.0.0.1:2181 --broker #brokerId# --num.retries 3 --retry.interval.ms 60 shutdown broker     9、删除topic ./kafka-run-class.sh kafka.admin.DeleteTopicCommand --topic testKJ1 --zookeeper 127.0.0.1:2181 ./kafka-topics.sh --zookeeper localhost:2181 --delete --topic testKJ1     10、查看consumer组内消费的offset ./kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zookeeper localhost:2181 --group test --topic testKJ1  ./kafka-consumer-offset-checker.sh --zookeeper 192.168.0.201:12181 --group group1 --topic group1

安装包下载：http://download.csdn.net/detail/xiaozhi2012323/9881958