k8s部署kafka程序连接异常

1. k8s部署的kafka程序连接异常

注意：由于问题处理后将问题复原现场做的记录，所以时间节点顺序有误

1）无法解析主机名

● 应用1

2022-11-18 11:34:53.284  WARN 7 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-2, groupId=group_asset_biyi] Error connecting to node kafka-0:9092 (id: 1 rack: null)
java.io.IOException: Can't resolve address: kafka-0:9092
....
Caused by: java.nio.channels.UnresolvedAddressException: null

● 应用2

2022-11-18 11:39:56.271 [kafka-producer-network-thread | producer-1] WARN  o.a.k.c.NetworkClient 950 - manager 0 0 - [Producer clientId=producer-1] Error connecting to node kafka-0:9092 (id: 1 rack: null)
java.net.UnknownHostException: kafka-0

无法解析kafka-0，这个名称是kafka的pod名称，由于kafka配置中没有设置advertised.listeners=参数导致，上面两个应用的报错都是由这一个原因导致的

● 修改kafka配置文件（由于第一次设置的是advertised.listeners=PLAINTEXT://localhost:9092，导致出现第二个错误问题）

broker.id=1 
# kafka的ip和port，（这里要是用实际IP地址或者域名）
advertised.listeners=PLAINTEXT://localhost:9092
# 实际应该配置问下下面的信息为正确，localhost服务时程序调用时创建topic
advertised.listeners=PLAINTEXT://kafka-dgms:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=419430400
log.dirs=/tmp/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000

# 连接zk
zookeeper.connect=zookeeper.dgms:2181
zookeeper.connection.timeout.ms=18000
group.initial.rebalance.delay.ms=0

2）无法建立到节点1的连接，Broker可能不可用

2022-11-18 10:57:11.283  WARN 8 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-2, groupId=group_asset_biyi] Connection to node 1 could not be established. Broker may not be available.
2022-11-18 10:57:12.136  WARN 8 --- [ntainer#0-0-C-1] org.apache.kafka.clients.NetworkClient   : [Consumer clientId=consumer-2, groupId=group_asset_biyi] Connection to node 1 could not be established. Broker may not be available.

● 由于kafka的server.properties中配置了是localhost，kafka创建topic主题时需要使用实际IP；导致java程序连接出现：无法建立到节点1的连接，Broker可能不可用
解决办法
● 修改配置文件中的IP为实际IP地址，由于使用k8s容器化部署，IP不是固定的，这里配置为kafka的svc的域名即可。

broker.id=1
# kafka的ip和port，（这里要是用实际IP地址或者域名）
advertised.listeners=PLAINTEXT://kafka.dgms:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=419430400
log.dirs=/tmp/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
# 连接zk
zookeeper.connect=zookeeper.dgms:2181
zookeeper.connection.timeout.ms=18000
group.initial.rebalance.delay.ms=0