CentOS6.5下kafka＋ZooKeeper下载与安装详细教程-多机版本2-附录

最新推荐文章于 2021-08-06 10:30:33 发布
最新推荐文章于 2021-08-06 10:30:33 发布
阅读量1.8k 收藏
点赞数
分类专栏： Kafka 原创 文章标签： Kafka zookeeper
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/wang_zhenwei/article/details/48357219
版权
原创 同时被 2 个专栏收录
105 篇文章 0 订阅
订阅专栏
Kafka
24 篇文章 0 订阅
订阅专栏

Slave1启动kafka服务的显示结果：

[root@Slave1 kafka]# sh bin/kafka-server-start.sh config/server.properties
[2015-09-10 08:42:17,026] INFO Verifying properties (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,275] INFO Property advertised.host.name is overridden to Slave1 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,275] INFO Property broker.id is overridden to 0 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,276] INFO Property host.name is overridden to Slave1 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,276] INFO Property log.cleaner.enable is overridden to false (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,276] INFO Property log.dirs is overridden to /tmp/kafka-logs (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,288] INFO Property log.retention.check.interval.ms is overridden to 300000 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,289] INFO Property log.retention.hours is overridden to 168 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,289] INFO Property log.segment.bytes is overridden to 1073741824 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,289] INFO Property num.io.threads is overridden to 8 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,290] INFO Property num.network.threads is overridden to 3 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,290] INFO Property num.partitions is overridden to 2 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,290] INFO Property num.recovery.threads.per.data.dir is overridden to 1 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,291] INFO Property port is overridden to 9092 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,291] INFO Property socket.receive.buffer.bytes is overridden to 102400 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,291] INFO Property socket.request.max.bytes is overridden to 104857600 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,292] INFO Property socket.send.buffer.bytes is overridden to 102400 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,292] INFO Property zookeeper.connect is overridden to Slave1:2181,Slave2:2181,Slave3:2181 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:18,293] INFO Property zookeeper.connection.timeout.ms is overridden to 6000 (kafka.utils.VerifiableProperties)
[2015-09-10 08:42:19,005] INFO [Kafka Server 0], starting (kafka.server.KafkaServer)
[2015-09-10 08:42:19,072] INFO [Kafka Server 0], Connecting to zookeeper on Slave1:2181,Slave2:2181,Slave3:2181 (kafka.server.KafkaServer)
[2015-09-10 08:42:19,321] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2015-09-10 08:42:19,918] INFO Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,918] INFO Client environment:host.name=Slave1 (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,918] INFO Client environment:java.version=1.7.0_51 (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,918] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,918] INFO Client environment:java.home=/usr/local/java/jdk1.7.0_51/jre (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,918] INFO Client environment:java.class.path=:.:/usr/local/java/jdk1.7.0_51/lib:/usr/local/java/jdk1.7.0_51/jre/lib:.:/usr/local/java/jdk1.7.0_51/lib:/usr/local/java/jdk1.7.0_51/jre/lib:.:/usr/local/java/jdk1.7.0_51/lib:/usr/local/java/jdk1.7.0_51/jre/lib:.:/usr/local/java/jdk1.7.0_51/lib:/usr/local/java/jdk1.7.0_51/jre/lib:/usr/local/kafka/bin/../core/build/dependant-libs-2.10.4*/*.jar:/usr/local/kafka/bin/../examples/build/libs//kafka-examples*.jar:/usr/local/kafka/bin/../contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/usr/local/kafka/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/usr/local/kafka/bin/../clients/build/libs/kafka-clients*.jar:/usr/local/kafka/bin/../libs/jopt-simple-3.2.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1-javadoc.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1-scaladoc.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1-sources.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1-test.jar:/usr/local/kafka/bin/../libs/kafka-clients-0.8.2.1.jar:/usr/local/kafka/bin/../libs/log4j-1.2.16.jar:/usr/local/kafka/bin/../libs/lz4-1.2.0.jar:/usr/local/kafka/bin/../libs/metrics-core-2.2.0.jar:/usr/local/kafka/bin/../libs/scala-library-2.10.4.jar:/usr/local/kafka/bin/../libs/slf4j-api-1.7.6.jar:/usr/local/kafka/bin/../libs/slf4j-log4j12-1.6.1.jar:/usr/local/kafka/bin/../libs/snappy-java-1.1.1.6.jar:/usr/local/kafka/bin/../libs/zkclient-0.3.jar:/usr/local/kafka/bin/../libs/zookeeper-3.4.6.jar:/usr/local/kafka/bin/../core/build/libs/kafka_2.10*.jar (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,919] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,919] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,919] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,919] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,919] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,919] INFO Client environment:os.version=2.6.32-431.el6.x86_64 (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,919] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,919] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,919] INFO Client environment:user.dir=/usr/local/kafka (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:19,920] INFO Initiating client connection, connectString=Slave1:2181,Slave2:2181,Slave3:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@75ad0070 (org.apache.zookeeper.ZooKeeper)
[2015-09-10 08:42:20,269] INFO Opening socket connection to server Slave1/192.168.154.158:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2015-09-10 08:42:20,348] INFO Socket connection established to Slave1/192.168.154.158:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-09-10 08:42:20,577] INFO Session establishment complete on server Slave1/192.168.154.158:2181, sessionid = 0x14fb770af610000, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2015-09-10 08:42:20,580] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
[2015-09-10 08:42:24,122] INFO Loading logs. (kafka.log.LogManager)
[2015-09-10 08:42:24,226] INFO Logs loading complete. (kafka.log.LogManager)
[2015-09-10 08:42:24,228] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2015-09-10 08:42:24,250] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2015-09-10 08:42:24,778] INFO Awaiting socket connections on Slave1:9092. (kafka.network.Acceptor)
[2015-09-10 08:42:25,162] INFO [Socket Server on Broker 0], Started (kafka.network.SocketServer)
[2015-09-10 08:42:34,520] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
[2015-09-10 08:42:34,896] INFO 0 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
[2015-09-10 08:42:36,729] INFO Registered broker 0 at path /brokers/ids/0 with address Slave1:9092. (kafka.utils.ZkUtils$)
[2015-09-10 08:42:36,763] INFO New leader is 0 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
[2015-09-10 08:42:36,825] INFO [Kafka Server 0], started (kafka.server.KafkaServer)
[2015-09-10 08:42:37,597] INFO [ReplicaFetcherManager on broker 0] Removed fetcher for partitions [kafkatopic,0] (kafka.server.ReplicaFetcherManager)
[2015-09-10 08:42:38,691] INFO Completed load of log kafkatopic-0 with log end offset 0 (kafka.log.Log)
[2015-09-10 08:42:38,723] INFO Created log for partition [kafkatopic,0] in /tmp/kafka-logs with properties {segment.index.bytes -> 10485760, file.delete.delay.ms -> 60000, segment.bytes -> 1073741824, flush.ms -> 9223372036854775807, delete.retention.ms -> 86400000, index.interval.bytes -> 4096, retention.bytes -> -1, min.insync.replicas -> 1, cleanup.policy -> delete, unclean.leader.election.enable -> true, segment.ms -> 604800000, max.message.bytes -> 1000012, flush.messages -> 9223372036854775807, min.cleanable.dirty.ratio -> 0.5, retention.ms -> 604800000, segment.jitter.ms -> 0}. (kafka.log.LogManager)
[2015-09-10 08:42:38,747] WARN Partition [kafkatopic,0] on broker 0: No checkpointed highwatermark is found for partition [kafkatopic,0] (kafka.cluster.Partition)
[2015-09-10 08:42:39,064] INFO [ReplicaFetcherManager on broker 0] Removed fetcher for partitions [kafkatopic,0] (kafka.server.ReplicaFetcherManager)


 

Slave2启动kafka服务的显示结果：

[root@Slave2 kafka]# sh bin/kafka-server-start.sh config/server.properties
[2015-09-10 09:32:55,285] INFO Verifying properties (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,438] INFO Property advertised.host.name is overridden to Slave2 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,439] INFO Property broker.id is overridden to 1 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,440] INFO Property host.name is overridden to Slave2 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,440] INFO Property log.cleaner.enable is overridden to false (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,441] INFO Property log.dirs is overridden to /tmp/kafka-logs (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,443] INFO Property log.retention.check.interval.ms is overridden to 300000 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,444] INFO Property log.retention.hours is overridden to 168 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,446] INFO Property log.segment.bytes is overridden to 1073741824 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,446] INFO Property num.io.threads is overridden to 8 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,454] INFO Property num.network.threads is overridden to 3 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,454] INFO Property num.partitions is overridden to 2 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,454] INFO Property num.recovery.threads.per.data.dir is overridden to 1 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,455] INFO Property port is overridden to 9092 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,455] INFO Property socket.receive.buffer.bytes is overridden to 102400 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,487] INFO Property socket.request.max.bytes is overridden to 104857600 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,487] INFO Property socket.send.buffer.bytes is overridden to 102400 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,488] INFO Property zookeeper.connect is overridden to Slave1:2181,Slave2:2181,Slave3:2181 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:56,488] INFO Property zookeeper.connection.timeout.ms is overridden to 6000 (kafka.utils.VerifiableProperties)
[2015-09-10 09:32:57,417] INFO [Kafka Server 1], starting (kafka.server.KafkaServer)
[2015-09-10 09:32:57,467] INFO [Kafka Server 1], Connecting to zookeeper on Slave1:2181,Slave2:2181,Slave3:2181 (kafka.server.KafkaServer)
[2015-09-10 09:32:57,951] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2015-09-10 09:32:58,093] INFO Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,093] INFO Client environment:host.name=Slave2 (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,093] INFO Client environment:java.version=1.7.0_51 (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,093] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,093] INFO Client environment:java.home=/usr/local/java/jdk1.7.0_51/jre (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,093] INFO Client environment:java.class.path=:.:/usr/local/java/jdk1.7.0_51/lib:/usr/local/java/jdk1.7.0_51/jre/lib:.:/usr/local/java/jdk1.7.0_51/lib:/usr/local/java/jdk1.7.0_51/jre/lib:.:/usr/local/java/jdk1.7.0_51/lib:/usr/local/java/jdk1.7.0_51/jre/lib:.:/usr/local/java/jdk1.7.0_51/lib:/usr/local/java/jdk1.7.0_51/jre/lib:/usr/local/kafka/bin/../core/build/dependant-libs-2.10.4*/*.jar:/usr/local/kafka/bin/../examples/build/libs//kafka-examples*.jar:/usr/local/kafka/bin/../contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/usr/local/kafka/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/usr/local/kafka/bin/../clients/build/libs/kafka-clients*.jar:/usr/local/kafka/bin/../libs/jopt-simple-3.2.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1-javadoc.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1-scaladoc.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1-sources.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1-test.jar:/usr/local/kafka/bin/../libs/kafka-clients-0.8.2.1.jar:/usr/local/kafka/bin/../libs/log4j-1.2.16.jar:/usr/local/kafka/bin/../libs/lz4-1.2.0.jar:/usr/local/kafka/bin/../libs/metrics-core-2.2.0.jar:/usr/local/kafka/bin/../libs/scala-library-2.10.4.jar:/usr/local/kafka/bin/../libs/slf4j-api-1.7.6.jar:/usr/local/kafka/bin/../libs/slf4j-log4j12-1.6.1.jar:/usr/local/kafka/bin/../libs/snappy-java-1.1.1.6.jar:/usr/local/kafka/bin/../libs/zkclient-0.3.jar:/usr/local/kafka/bin/../libs/zookeeper-3.4.6.jar:/usr/local/kafka/bin/../core/build/libs/kafka_2.10*.jar (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,093] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,093] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,093] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,093] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,093] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,093] INFO Client environment:os.version=2.6.32-431.el6.x86_64 (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,093] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,093] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,093] INFO Client environment:user.dir=/usr/local/kafka (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,094] INFO Initiating client connection, connectString=Slave1:2181,Slave2:2181,Slave3:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@4f902864 (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:32:58,571] INFO Opening socket connection to server Slave2/192.168.154.159:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2015-09-10 09:32:58,831] INFO Socket connection established to Slave2/192.168.154.159:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-09-10 09:32:59,135] INFO Session establishment complete on server Slave2/192.168.154.159:2181, sessionid = 0x24fb79b843a0000, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2015-09-10 09:32:59,138] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
[2015-09-10 09:32:59,744] INFO Log directory '/tmp/kafka-logs' not found, creating it. (kafka.log.LogManager)
[2015-09-10 09:32:59,801] INFO Loading logs. (kafka.log.LogManager)
[2015-09-10 09:32:59,807] INFO Logs loading complete. (kafka.log.LogManager)
[2015-09-10 09:32:59,808] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2015-09-10 09:32:59,846] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2015-09-10 09:33:01,537] INFO Awaiting socket connections on Slave2:9092. (kafka.network.Acceptor)
[2015-09-10 09:33:01,564] INFO [Socket Server on Broker 1], Started (kafka.network.SocketServer)
[2015-09-10 09:33:03,274] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
[2015-09-10 09:33:05,480] INFO Registered broker 1 at path /brokers/ids/1 with address Slave2:9092. (kafka.utils.ZkUtils$)
[2015-09-10 09:33:07,243] INFO [Kafka Server 1], started (kafka.server.KafkaServer)


 

Slave3启动kafka服务的显示结果：

[root@Slave3 kafka]# sh bin/kafka-server-start.sh config/server.properties
[2015-09-10 09:36:04,984] INFO Verifying properties (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,476] INFO Property advertised.host.name is overridden to Slave3 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,476] INFO Property broker.id is overridden to 2 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,523] INFO Property host.name is overridden to Slave3 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,523] INFO Property log.cleaner.enable is overridden to false (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,523] INFO Property log.dirs is overridden to /tmp/kafka-logs (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,524] INFO Property log.retention.check.interval.ms is overridden to 300000 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,524] INFO Property log.retention.hours is overridden to 168 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,524] INFO Property log.segment.bytes is overridden to 1073741824 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,525] INFO Property num.io.threads is overridden to 8 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,525] INFO Property num.network.threads is overridden to 3 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,526] INFO Property num.partitions is overridden to 2 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,527] INFO Property num.recovery.threads.per.data.dir is overridden to 1 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,527] INFO Property port is overridden to 9092 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,527] INFO Property socket.receive.buffer.bytes is overridden to 102400 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,528] INFO Property socket.request.max.bytes is overridden to 104857600 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,528] INFO Property socket.send.buffer.bytes is overridden to 102400 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,528] INFO Property zookeeper.connect is overridden to Slave1:2181,Slave2:2181,Slave3:2181 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:05,529] INFO Property zookeeper.connection.timeout.ms is overridden to 6000 (kafka.utils.VerifiableProperties)
[2015-09-10 09:36:08,415] INFO [Kafka Server 2], starting (kafka.server.KafkaServer)
[2015-09-10 09:36:08,440] INFO [Kafka Server 2], Connecting to zookeeper on Slave1:2181,Slave2:2181,Slave3:2181 (kafka.server.KafkaServer)
[2015-09-10 09:36:08,498] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2015-09-10 09:36:08,796] INFO Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,796] INFO Client environment:host.name=Slave3 (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,796] INFO Client environment:java.version=1.7.0_51 (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,797] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,797] INFO Client environment:java.home=/usr/local/java/jdk1.7.0_51/jre (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,797] INFO Client environment:java.class.path=:.:/usr/local/java/jdk1.7.0_51/lib:/usr/local/java/jdk1.7.0_51/jre/lib:.:/usr/local/java/jdk1.7.0_51/lib:/usr/local/java/jdk1.7.0_51/jre/lib:.:/usr/local/java/jdk1.7.0_51/lib:/usr/local/java/jdk1.7.0_51/jre/lib:.:/usr/local/java/jdk1.7.0_51/lib:/usr/local/java/jdk1.7.0_51/jre/lib:/usr/local/kafka/bin/../core/build/dependant-libs-2.10.4*/*.jar:/usr/local/kafka/bin/../examples/build/libs//kafka-examples*.jar:/usr/local/kafka/bin/../contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/usr/local/kafka/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/usr/local/kafka/bin/../clients/build/libs/kafka-clients*.jar:/usr/local/kafka/bin/../libs/jopt-simple-3.2.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1-javadoc.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1-scaladoc.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1-sources.jar:/usr/local/kafka/bin/../libs/kafka_2.10-0.8.2.1-test.jar:/usr/local/kafka/bin/../libs/kafka-clients-0.8.2.1.jar:/usr/local/kafka/bin/../libs/log4j-1.2.16.jar:/usr/local/kafka/bin/../libs/lz4-1.2.0.jar:/usr/local/kafka/bin/../libs/metrics-core-2.2.0.jar:/usr/local/kafka/bin/../libs/scala-library-2.10.4.jar:/usr/local/kafka/bin/../libs/slf4j-api-1.7.6.jar:/usr/local/kafka/bin/../libs/slf4j-log4j12-1.6.1.jar:/usr/local/kafka/bin/../libs/snappy-java-1.1.1.6.jar:/usr/local/kafka/bin/../libs/zkclient-0.3.jar:/usr/local/kafka/bin/../libs/zookeeper-3.4.6.jar:/usr/local/kafka/bin/../core/build/libs/kafka_2.10*.jar (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,797] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,797] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,797] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,797] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,797] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,797] INFO Client environment:os.version=2.6.32-431.el6.x86_64 (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,797] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,797] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,797] INFO Client environment:user.dir=/usr/local/kafka (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:08,799] INFO Initiating client connection, connectString=Slave1:2181,Slave2:2181,Slave3:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@14be60b (org.apache.zookeeper.ZooKeeper)
[2015-09-10 09:36:10,145] INFO Opening socket connection to server Slave3/192.168.154.160:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2015-09-10 09:36:10,181] INFO Socket connection established to Slave3/192.168.154.160:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-09-10 09:36:10,257] INFO Session establishment complete on server Slave3/192.168.154.160:2181, sessionid = 0x34fb79a3f250000, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2015-09-10 09:36:10,260] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
[2015-09-10 09:36:16,636] INFO Log directory '/tmp/kafka-logs' not found, creating it. (kafka.log.LogManager)
[2015-09-10 09:36:16,710] INFO Loading logs. (kafka.log.LogManager)
[2015-09-10 09:36:16,716] INFO Logs loading complete. (kafka.log.LogManager)
[2015-09-10 09:36:16,716] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2015-09-10 09:36:16,754] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2015-09-10 09:36:17,095] INFO Awaiting socket connections on Slave3:9092. (kafka.network.Acceptor)
[2015-09-10 09:36:17,177] INFO [Socket Server on Broker 2], Started (kafka.network.SocketServer)
[2015-09-10 09:36:19,094] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
[2015-09-10 09:36:21,991] INFO Registered broker 2 at path /brokers/ids/2 with address Slave3:9092. (kafka.utils.ZkUtils$)
[2015-09-10 09:36:27,045] INFO [Kafka Server 2], started (kafka.server.KafkaServer)


 

 

在Slave1新建立一个TOPIC内容显示：

 

[root@Slave1 bin]# sh kafka-topics.sh --create --topic kafkatopic --replication-factor 1 --partitions 1 --zookeeper localhost:2181
Created topic "kafkatopic"


 

多次实验以后内容显示（报错）：

[root@Slave1 bin]# sh kafka-topics.sh --create --topic kafkatopic --replication-factor 1 --partitions 1 --zookeeper Slave1:2181
Error while executing topic command Topic "kafkatopic" already exists.
kafka.common.TopicExistsException: Topic "kafkatopic" already exists.
at kafka.admin.AdminUtils$.createOrUpdateTopicPartitionAssignmentPathInZK(AdminUtils.scala:187)
at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:172)
at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:93)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:55)
at kafka.admin.TopicCommand.main(TopicCommand.scala)
 
[root@Slave1 bin]#


 

 

在Slave2把KAFKA的生产者启动起来内容显示：

[root@Slave2 bin]# sh kafka-console-producer.sh --broker-list Slave2:9092 --sync --topic kafkatopic
[2015-09-10 09:47:02,625] WARN Property topic is not valid (kafka.utils.VerifiableProperties)


 

在Slave3把消费者启动起来内容显示：

[root@Slave3 bin]# sh kafka-console-consumer.sh --zookeeper Slave3:2181 --topic kafkatopic --from-beginning


 

验证结果：

 


 

Wang_Zhenwei
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
kafka配置项host.name advertised.host.name
weixin_30865427的博客
04-26 1190
遇到的问题： 在本机或者其他机器telnet IP 9092，通，使用域名也通，telnet 127.0.0.1 9092不通 host.name：按配置文件说明，是Kafka绑定的interface。其实这个说明有点误导，下面会见到。 advertised.host.name：是注册到zookeeper，client要访问的broker地址。（可能producer也是拿这个值，...
Kafka原理和集群测试
i am me，haha，么么哒。。。
05-07 1055
Kafka是一个消息系统，由LinkedIn贡献给Apache基金会，称为Apache的一个顶级项目。Kafka最初用作LinkedIn的活动流（activity stream）和运营数据处理管道（pipeline）的基础。它具有可扩展、吞吐量大和可持久化等特征，以及非常好的分区、复制和容错特征。 Kafka的关键设计决策 1). Kafka在设计之时为就将持久化消息作为通常的使用情况
分布式消息服务器kafka,分布式消息队列kafka系列介绍 --- 安装运行 | 知行天下
weixin_36356958的博客
08-06 258
第一步 包下载地址 ： https://www.apache.org/dyn/closer.cgi?path=/kafka/0.8.1.1/kafka_2.9.2-0.8.1.1.tgztar -zxvf kafka_2.9.2-0.8.1.1.tgzcd kafka_2.9.2-0.8.1.1第二步 启动服务启动zookeeper ：bin/zookeeper-server-start.sh c...
kafka的advertised.host.name参数 外网访问配置
大叶子不小的博客
05-25 2255
kafka的server.properties文件 ```host.name```开始只绑定在了内部IP上，对外网卡无法访问。 把值设置为空的话会kafka监听端口在所有的网卡上绑定。但是在外网访问时，客户端又遇到了```java.nio.channels.ClosedChannelException```异常信息，server端用tcpdump分析的时候发现客户端有传递kafka所在机器的机器名过来。在client里断点跟踪一下发现是findLeader的时候返回的元信息是机器名而不是IP。客户端无法
CentOS6.5下kafka＋ZooKeeper下载与安装详细教程-单机版
03-24
CentOS6.5下kafka＋ZooKeeper下载与安装详细教程-单机版
集群环境搭建-Centos+kafka+zookeeper+hadoop+Spark
12-19
通过VirtualBox安装多台虚拟机，实现集群环境搭建。 优势：一台电脑即可。 应用场景：测试，学习。...内附百度网盘下载地址，有hadoop+zookeeper+spark+kafka等等·····需要的安装包和配置文件
Centos6.5下Jdk+Tomcat+Mysql环境安装图文教程
09-02
总结起来，这个教程详细介绍了在CentOS 6.5上安装JDK 1.7.60、Tomcat 8.0.33和MySQL的过程。这些步骤是建立一个基础的Java Web服务器环境的关键组成部分，对于部署和管理Java应用程序至关重要。每个步骤都需要仔细...
kafka+zookeeper安装教程
12-21
kafka+zookeeper集群安装详细教程，有需要的可以下载，并实际操作一下。
CentOS 6.5下yum安装 MySQL-5.5全过程图文教程
01-19
下面通过两种方法给大家介绍CentOS 6.5下yum安装 MySQL-5.5全过程，一起学习吧。 方法一： 具体方法和步骤如下所示： 1.第一步就是看linu是否安装了mysql，经过rpm -qa|grep mysql查看到centos下安装了mysql5.1，那...
kafka的advertised.host.name参数
qq_39551064的博客
01-16 1176
参考文章点击查看原文
Kafka配置外网访问简要配置
小毛老头 MaoLT.Xiao
11-28 2451
低版本有两个配置参数需要修改： advertised.host.name=ip地址 #<==kafka开放到外网的IP advertised.port=9092 #<==9092映射到外网后的端口 新版本 broker配置弃用了advertised.host.name 和 advertised.port 这两个个配置项，就配置ad...
ZooKeeper配置引起Hbase和Kafka启动异常
小陌成长之路
02-08 2138
使用ambari部署的HDP大数据平台，启动Hbase和Kafka的时候出现问题。
容器环境下IP跨网闸映射kafka部署
weixin_38932035的博客
04-06 1539
一、 listeners 和 advertised.listeners 在公司内网部署 kafka 集群只需要用到 listeners，内外网需要作区分时才需要用到advertised.listeners。 listeners: 学名叫监听器，其实就是告诉外部连接者要通过什么协议访问指定主机名和端口开放的 Kafka 服务。我们要监听的uri的逗号分隔列表和监听器名称。指定hostname为0.0.0.0绑定所有接口。hostname为空以绑定到默认接口(JVM拿到的本机的hostname，java.ne
安装和测试Kafka（转）
weixin_33743880的博客
04-27 160
http://blog.javachen.com/2015/03/17/install-and-test-kafka/#   本文主要介绍如何在单节点上安装 Kafka 并测试 broker、producer 和 consumer 功能。 下载 进入下载页面：http://kafka.apache.org/downloads.html ，选择 Binary downloads下载 （S...
基于Kafka构建事件溯源模式的微服务
weixin_34074740的博客
06-16 360
2019独角兽企业重金招聘Python工程师标准>>> ...
kafka配置公网IP访问
phymat.nico的专栏
06-10 1978
0.8版本有两个配置参数需要修改： advertised.host.name=171.220.211.158 #<==kafka开放到外网的IP advertised.port=10012 #<==9092映射到外网后的端口 最新版本0.10.x broker配置弃用了advertised.host.name 和 advertised.port 这两个个配置项，就配置advertised.listeners就可以了：
格式化zookeeper时报错：拒绝连接
热门推荐
咕噜君的博客
04-10 1万+
首先启动zookeeper集群：在每个节点的Zookeeper安装目录的bin目录下执行命令： sh zkServer.sh start zookeeper集群启动后，在zk的leader节点上执行：这个指令的作用是在zookeeper集群上生成ha节点（ns节点） hdfs zkfc -formatZK 报如下错误： 20/04/10 14:58:47 WARN zookeeper....
CentOS6.5上搭建Hadoop-2.7.3+Zookeeper-3.4.8分布式环境教程
"这篇文档详细介绍了如何在四台运行CentOS 6.5的服务器上搭建基于hadoop-2.7.3、zookeeper-3.4.8和hadoop-2.7.3的分布式环境，作者王三旗亲测安装成功。涉及的组件包括Hadoop、ZooKeeper以及HBase，所有安装包的下载...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值