kafka + zookeeper

一、安装

1. 下载、解压

2. 配置文件修改

/opt/bigdata/zookeeper-3.4.6/conf
zoo.cfg:

# The number of milliseconds of each tick
tickTime=2000

# The number of ticks that the initial 
# synchronization phase can take
initLimit=10

# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5

# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/var/bigdata/hadoop/zk
dataLogDir=/var/bigdata/hadoop/zk
# the port at which the clients will connect
clientPort=2181

# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=master:2888:3888
server.2=anode1:2888:3888
server.3=anode2:2888:3888

3. 分发至其他节点

scp -r /opt/bigdata/zookeeper-3.4.6 root@anode1:/opt/bigdata/

4. 在三个节点上创建myid文件

目录：/var/bigdata/hadoop/zk/myid
内容分别为1、2、3

5. 每个节点分别启动/关闭zookeeper

cd /opt/bigdata/zookeeper-3.4.6/bin/
./zkServer.sh start
./zkServer.sh stop

二、kafka自带zookeeper

首先zookeeper启动

bin/zookeeper-server-start.sh （-daemon）config/zookeeper.properties

其次kafka启动

./bin/kafka-server-start.sh -daemon ../config/server.properties

关于消息

1. 创建topic

./bin/kafka-topics.sh --create --zookeeper ip:2181 --replication-factor 1 --partitions 1 --topic test

2. 查看topic

 ./bin/kafka-topics.sh --list --zookeeper ip:2181

3. 发送消息

 ./bin/kafka-console-producer.sh --broker-list ip:9092 --topic test

4. 接收消息

# 另开一个终端
./bin/kafka-console-consumer.sh --bootstrap-server 192.168.1.100:9093 --topic test --from-beginning

5. 查看特定主题的详细信息

# 详细信息
./bin/kafka-topics.sh --zookeeper master:2181 --describe  --topic test
# 具体内容
./bin/kafka-console-consumer.sh --bootstrap-server master:9092 --topic test --from-beginning