1. Kafka下载:
wget https://archive.apache.org/dist/kafka/0.8.1/kafka_2.9.2-0.8.1.tgz
解压 tar zxvf kafka_2.9.2-0.8.1.tgz
启动zookeeper
nohup bin/zookeeper-server-start.sh config/zookeeper.properties &
或者 zkServer.sh start 启动
启动kafka
nohup bin/kafka-server-start.sh -daemon config/server.properties &
创建主题
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
bin/kafka-topics.sh --create --zookeeper master:2181 --replication-factor 2 --partitions 8 --topic DreamTopic
bin/kafka-topics.sh --create --zookeeper 192.168.100.125:2181 --replication-factor 1 --partitions 1 --topic DreamTopic1
#replication-factor 表示该topic需要在不同的broker中保存几份, partitions为几个分区
查看topic个数
bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic test
bin/kafka-topics.sh --describe --zookeeper 192.168.100.125:2181 --topic DreamTopic1
bin/kafka-topics.sh --list --zookeeper master:2181,node2:2181,node1:2181
查看topic的生产者和消费者信息
./kafka-topics.sh --describe --zookeeper localhost:2181 --topic idoall_testTopic,DreamTopic
[root@linux-node2 bin]# ./kafka-topics.sh --describe --zookeeper localhost:2181 --topic dream Topic:dream PartitionCount:5 ReplicationFactor:2 Configs: Topic: dream Partition: 0 Leader: 1 Replicas: 1,2 Isr: 1,2 Topic: dream Partition: 1 Leader: 2 Replicas: 2,3 Isr: 2,3 Topic: dream Partition: 2 Leader: 3 Replicas: 3,1 Isr: 3,1 Topic: dream Partition: 3 Leader: 1 Replicas: 1,3 Isr: 1,3 Topic: dream Partition: 4 Leader: 2 Replicas: 2,1 Isr: 2,1
leader:负责处理消息的读和写,leader是从所有节点中随机选择的.
Replicas:列出了所有的副本节点,不管节点是否在服务中.
Lsr:是正在服务中的节点.
生产者测试
bin/kafka-console-producer.sh --broker-list kafka1:9092 --topic test
bin/kafka-console-producer.sh --broker-list 192.168.100.125:9092 --topic DreamTopic1
消费者测试
bin/kafka-console-consumer.sh --zookeeper 192.168.100.125:2181 --topic test --from-beginning
bin/kafka-console-consumer.sh --zookeeper kafka1:2181 --topic test --from-beginning
bin/kafka-console-consumer.sh --zookeeper 192.168.100.125:2181 --topic DreamTopic1 --from-beginning
kafka删除topic方法
1) bin/kafka-topics.sh --delete --zookeeper master:2181 --topic DreamTopic
如果删除后查看topic显示为:marked for deletion 则需要在每一台机器中的 config/server.properties 文件加入 delete.topic.enable=true,然后重启kafka
2) 删除kafka存储目录(server.properties文件log.dirs配置,默认为"/tmp/kafka-logs")相关topic目录删除zookeeper "/brokers/topics/"目录下相关topic节点