启动脚本
#! /bin/bash
case $1 in
"start"){
for i in hadoop102 hadoop103 hadoop104
do
echo "========== $i =========="
ssh $i '/opt/module/kafka/bin/kafka-server-start.sh -daemon
/opt/module/kafka/config/server.properties'
done
}};
"stop"){
for i in hadoop102 hadoop103 hadoop104
do
echo "========== $i =========="
ssh $i '/opt/module/kafka/bin/kafka-server-stop.sh'
done
}};
esac
kafka-topics.sh
创建topic后,会在log.dirs中产生以“topic的名字-分区号”的文件夹,这些文件夹,有的是topic的leader,有的是follower
–replication-factor 定义副本数,不能比broker的数量大,一个leader,多个follower,加起来等于副本数
partition随便
#列出topic
bin/kafka-topics.sh --zookeeper ip:2181 --list
#创建
bin/kafka-topics.sh --create --zookeeper ip:2181,ip:2181 --replication-factor 2 --partitions 1 --topic cheyo-topic
#删除 topic
bin/kafka-topics.sh --zookeeper hadoop102:2181 --delete --topic first
#查看某个 Topic 的详情
bin/kafka-topics.sh --zookeeper hadoop102:2181 --describe --topic first
#修改分区数
bin/kafka-topics.sh --zookeeper hadoop102:2181 --alter --topic first --partitions 6
生产者:
bin/kafka-console-producer.sh --broker-list hadoop102:9092 --topic first
消费者:集群内都可以收到,连接的是kafka的集群server,不再是zookeeper了,0.9版本前存在zookeeper里,0.9版本后存在了kafka系统创建的topic“_consumer_offsets”里,默认50个分区,1个副本
bin/kafka-console-consumer.sh --bootstrap-server ip:9092 --from-beginning --topic waf_log_websec