zookeeper+kafka集群

------------------------------------
zookeper：
59.110.173.251（xianshang  ssq）   /home/appinstall/zookeeper-3.4.9
47.94.133.25   （xianshang 5M host）/home/songlk/appinstall/zookeeper-3.4.9
60.205.136.57 （jsfund）  /home/songlk/appinstall/zookeeper-3.4.9

zoo.cfg:

dataDir=/home/centos/zookeeper

server.1=59.110.173.251:2888:3888
server.2=47.94.133.25:2888:3888
server.3=60.205.136.57:2888:3888

写集群文件
echo 1 > /home/centos/zookeeper/myid

echo 2 > /home/centos/zookeeper/myid

echo 3 > /home/centos/zookeeper/myid

--------------

iptables -A INPUT -p tcp --dport 2181 -j ACCEPT

iptables -A INPUT -p tcp --dport 2888 -j ACCEPT

iptables -A INPUT -p tcp --dport 3888 -j ACCEPT

分别启动
59.110.173.251
/home/appinstall/zookeeper-3.4.9/bin/zkServer.sh start
47.94.133.25
/home/songlk/appinstall/zookeeper-3.4.9/bin/zkServer.sh start
60.205.136.57
/home/songlk/appinstall/zookeeper-3.4.9/bin/zkServer.sh start

查看leader与slave状态

/home/appinstall/zookeeper-3.4.9/bin/zkServer.sh status

/home/songlk/appinstall/zookeeper-3.4.9/bin/zkServer.sh status
 
/home/songlk/appinstall/zookeeper-3.4.9/bin/zkServer.sh status

-----

flume

/home/songlk/appinstall/flume

----

kafka

59.110.173.251（xianshang  ssq）

/home/appinstall/kafka

47.94.133.25   （xianshang 5M host）

/home/songlk/appinstall/kafka

60.205.136.57（ jsfund）

/home/songlk/appinstall/kafka

配置server.properties

broker.id=251

listeners=PLAINTEXT://:9092

log.dirs=/home/appinstall/kafka/kafka-logs

zookeeper.connect=59.110.173.251:2181,47.94.133.25:2181,60.205.136.57:2181

broker.id=25
listeners=PLAINTEXT://:9092

log.dirs=/home/songlk/appinstall/kafka/kafka-logs

 zookeeper.connect=59.110.173.251:2181,47.94.133.25:2181,60.205.136.57:2181

broker.id=57

listeners=PLAINTEXT://:9092

log.dirs=/home/songlk/appinstall/kafka/kafka-logs

zookeeper.connect=59.110.173.251:2181,47.94.133.25:2181,60.205.136.57:2181

----
producer.properties
59.110.173.251:9092,47.94.133.25:9092,60.205.136.57:9092

启动kafka

/home/appinstall/kafka/bin/kafka-server-start.sh -daemon /home/appinstall/kafka/config/server.properties

/home/songlk/appinstall/kafka/bin/kafka-server-start.sh -daemon /home/songlk/appinstall/kafka/config/server.properties

/home/songlk/appinstall/kafka/bin/kafka-server-start.sh -daemon /home/songlk/appinstall/kafka/config/server.properties

关闭

/home/appinstall/kafka/bin/kafka-server-stop.sh

/home/songlk/appinstall/kafka/bin/kafka-server-stop.sh

/home/songlk/appinstall/kafka/bin/kafka-server-stop.sh

启动异常：

There is insufficient memory for the Java Runtime Environment to continue.

export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"

------------------------

./kafka-topics.sh --describe --zookeeper localhost:2181 --topic test

8.创建主题 
        $>/home/appinstall/kafka/bin/kafka-topics.sh --create --zookeeper 59.110.173.251:2181 --replication-factor 3 --partitions 3 --topic test

[root@iz2zej15counp893693urlz bin]# /home/appinstall/kafka/bin/kafka-topics.sh --create --zookeeper 59.110.173.251:2181 --replication-factor 3 --partitions 3 --topic test
Created topic "test".

    9.查看主题列表
        $>/home/appinstall/kafka/bin/kafka-topics.sh --list --zookeeper 59.110.173.251:2181

/home/songlk/appinstall/kafka/bin/kafka-topics.sh --list --zookeeper 59.110.173.251:2181
/home/songlk/appinstall/kafka/bin/kafka-topics.sh --list --zookeeper 59.110.173.251:2181

[root@iz2zej15counp893693urlz bin]# /home/appinstall/kafka/bin/kafka-topics.sh --list --zookeeper 59.110.173.251:2181
test

    10.启动控制台生产者
 /home/songlk/appinstall/kafka/bin/kafka-console-producer.sh --broker-list 60.205.136.57:9092 --topic test

    11.启动控制台消费者
  /home/songlk/appinstall/kafka/bin/kafka-console-consumer.sh --bootstrap-server 60.205.136.57:9092 --topic test --from-beginning --zookeeper 60.205.136.57:2181

 ====================

 

 

    12.在生产者控制台输入hello world

--------------------------------------------------------------------------------------------------------------