zookeeper+kafka环境搭建

下载软件包

下载kafka二进制包

$ mkdir software
$ cd software
$ wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.10/zookeeper-3.4.10.tar.gz
$ wget http://apache.fayea.com/kafka/0.11.0.0/kafka_2.11-0.11.0.0.tgz

环境java安装

$ yum -y install java-1.8.0-openjdk  

config and start zookeeper

$ tar -zxvf zookeeper-3.4.10.tar.gz
$ mv zookeeper-3.4.10 zookeeper
$ cd zookeeper/conf
$ cp zoo_smle.cfg zoo.cfg
$ vim zoo.cfg
dataDir=/var/local/doc/zookeeper
$ cd ..
$
$ ./bin/zkServer.sh  start
ZooKeeper JMX enabled by default
Using config: /root/doc_s/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
$
$ ./bin/zkServer.sh  status
ZooKeeper JMX enabled by default
Using config: /root/doc_s/zookeeper/bin/../conf/zoo.cfg
Mode: standalone

config kafka

参考文档：http://kafka.apache.org/quickstart

step 1: start kafka service

$ tar -zxvf kafka_2.11-0.11.0.0.tgz
$ mv kafka_2.11-0.11.0.0 kafka
$ cd kafka
$ vim config/server.properties

num.partitions=8
log.dirs=/var/local/kafka
zookeeper.connect=localhost:2181
advertised.host.name=192.168.1.111

// 启动kafka服务器
$ bin/kafka-server-start.sh config/server.properties 

[2017-08-21 19:06:55,424] INFO starting (kafka.server.KafkaServer)
[2017-08-21 19:06:55,425] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2017-08-21 19:06:55,433] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)

step 2: Create a topic

// 创建名字为nginx_log的topic
$ pwd
/root/doc_s/kafka
$ ./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 8 --topic nginx_log
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Created topic "nginx_log".
// 查看创建好的topic
$ ./bin/kafka-topics.sh --list --zookeeper localhost:2181
nginx_log

step 3: send some meassage

kafka的主目录中

$ ./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic  nginx_log
> This is a message
> This is another message

Step 4: Start a consumer

获取刚才发送的消息

$ ./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic nginx_log --from-beginning
This is another message
This is a message

已经获取到了消息，先发送先接受。