kafka启用SASL/PLAIN+ACL后,kafka-consumer-group.sh查看消费组报错

kafka启用SASL/PLAIN+ACL后,kafka-consumer-group.sh查看消费组报错

问题

kafka启用用户认证和权限管理（SASL/PLAIN+ACL）之后，kafka-consumer-group.sh 查看消费组会报错

-bash-4.2$cd $KAFKA_HOME
-bash-4.2$bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server  192.168.15.11:9092,192.168.15.12:9092,192.168.15.13:9092 --list

Note: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers).

[2020-04-02 12:37:43,272] WARN Bootstrap broker 192.168.15.11:9092 disconnected (org.apache.kafka.clients.NetworkClient)
[2020-04-02 12:37:43,275] WARN Bootstrap broker 192.168.15.12:9092 disconnected (org.apache.kafka.clients.NetworkClient)
[2020-04-02 12:37:43,276] WARN Bootstrap broker 192.168.15.13:9092 disconnected (org.apache.kafka.clients.NetworkClient)
Error: Executing consumer group command failed due to Request METADATA failed on brokers List(192.168.15.12:9092 (id: -2 rack: null), 192.168.15.13:9092 (id: -3 rack: null), 192.168.15.11:9092 (id: -1 rack: null))

命令

kafka-consumer-group.sh 命令
新版本

cd $KAFKA_HOME
bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server 192.168.15.11:9092,192.168.15.12:9092,192.168.15.13:9092 --list

旧版本

cd $KAFKA_HOME
bin/kafka-consumer-groups.sh --zookeeper 192.168.15.11:2181,192.168.15.12:2181,192.168.15.13:2181 --list

kafka启用用户认证和权限管理（SASL/PLAIN+ACL）之后，只能使用新版本，已经不支持ZK去连接kafka broker,下面以kafka 0.10.2.0版本测试，kafka新版本也适用。

改动

改动如下

cd $KAFKA/bin
cp kafka-consumer-group.sh kafka-consumer-group_jaas.sh
vi kafka-consumer-group_jaas.sh

#!/bin/bash

# 方式1：Add jaas file
export KAFKA_OPTS="-Djava.security.auth.login.config=/home/app/software/kafka/config/kafka_client_jaas.conf"
exec $(dirname $0)/kafka-run-class.sh kafka.admin.ConsumerGroupCommand "$@"

# 方式2：Add jaas file
exec $(dirname $0)/kafka-run-class.sh -Djava.security.auth.login.config=/home/app/software/kafka/config/kafka_client_jaas.conf kafka.admin.ConsumerGroupCommand "$@"

cd $KAFKA_HOME/config
vi kafka_client_jaas.conf

KafkaClient{
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="admin"
        password="admin123!@";
};

vi command_jaas.properties 

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN

测试

-bash-4.2$cd $KAFKA_HOME
-bash-4.2$ bin/kafka-consumer-groups_jaas.sh --new-consumer --command-config config/command_jaas.properties  --bootstrap-server  192.168.15.11:9092,192.168.15.12:9092,192.168.15.13:9092 --list

Note: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers).

test_01
test_02
test_03
test_04
test_05

测试过程中发现kafka_client_jaas.conf文件中配置ACL普通用户查询时不报错，但是没有数据。只能用ACL管理用户admin