docker安装kafka

最新推荐文章于 2024-09-10 11:00:47 发布

卡冈都亚

最新推荐文章于 2024-09-10 11:00:47 发布

阅读量230

点赞数

分类专栏： kafka 文章标签： kafka

本文链接：https://blog.csdn.net/qq_34919833/article/details/108493947

版权

kafka 专栏收录该内容

2 篇文章 0 订阅

订阅专栏

1.安装:

1.docker pull wurstmeister/zookeeper
2.docker run -d --name zookeeper -p 2181:2181 -v /etc/localtime:/etc/localtime wurstmeister/zookeeper
注：使用dokcer inspect 查看zk的ip:172.18.0.14 在启动kafka时要指定该IP
3.docker pull wurstmeister/kafka
4.
docker run -itd --name kafka -p 9092:9092 -e KAFKA_BROKER_ID=0 -e KAFKA_ZOOKEEPER_CONNECT=172.18.0.14:2181/kafka -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://:9092 -e KAFKA_LISTENERS=PLAINTEXT://:9092 -v /etc/localtime:/etc/localtime -v /opt/kafka:/opt/kafka kafka:512 /bin/bash /usr/bin/start-kafka.sh

2 启动：

5 启动zk(默认启动)
进入容器：docker exec -it kafka /bin/bash
nohup /opt/kafka/bin/zookeeper-server-start.sh /opt/kafka/config/zookeeper.properties &


6 启动broker (默认启动）
进入容器：docker exec -it kafka /bin/bash
nohup /opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties &

3 测试

创建topic（相同的topic只创建一次 即可）
进入容器：docker exec -it kafka /bin/bash
7./opt/kafka/kafka-topics.sh --create --zookeeper 172.18.0.14:2181/kafka --topic mytopic --replication-factor 1 --partitions 1


8.模拟生产者
/opt/kafka/kafka-console-producer.sh --topic mytopic --broker-list 172.18.0.15:9092


9.模拟消费者
./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic mytopic --from-beginning
./kafka-console-consumer.sh --bootstrap-server 172.18.0.15:9092 --topic mytopic --from-beginning

4 问题：


[2020-09-09 06:51:09,062] WARN [Consumer clientId=consumer-console-consumer-15517-1, groupId=console-consumer-15517] Error connecting to node 364b2f046261:9092 (id: 0 rack: null) (org.apache.kafka.clients.NetworkClient)
java.net.UnknownHostException: 364b2f046261
        at java.net.InetAddress.getAllByName0(InetAddress.java:1281)
        at java.net.InetAddress.getAllByName(InetAddress.java:1193)
        at java.net.InetAddress.getAllByName(InetAddress.java:1127)
        at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
        at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:403)
        at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:363)
        at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:151)
        at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:955)
        at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:293)
        at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.trySend(ConsumerNetworkClient.java:495)
        at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:252)
        at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233)
        at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:212)
        at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:236)
        at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:463)
        at org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1275)
        at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1241)
        at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216)
        at kafka.tools.ConsoleConsumer$ConsumerWrapper.receive(ConsoleConsumer.scala:437)
        at kafka.tools.ConsoleConsumer$.process(ConsoleConsumer.scala:103)
        at kafka.tools.ConsoleConsumer$.run(ConsoleConsumer.scala:77)
        at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:54)
        at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala)

原因：消费者连的时候 是用容器id：364b2f046261 去找对应的ip. 本机没有设置hosts的话他就找不到了。
解决办法：在宿主机、客户端机器的/etc/hosts文件中添加 容器ip和容器id的对应
服务器ip:容器id


::1 VM_0_6_centos VM_0_6_centos
::1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
172.18.0.15     364b2f046261

参考：
作者：szgl_lucifer
链接：https://www.jianshu.com/p/e8c29cba9fae