记录-JAVA Consumer连接开启kerberos的kafka集群

JAVA API连接Kafka很方便，但是在集群开启kerberos验证的情况下，需要注意以下情况。

环境及版本信息

集群环境：CDH：5.7

依赖：

<dependency>
		<groupId>org.springframework.kafka</groupId>
		<artifactId>spring-kafka</artifactId>
		<version>1.1.8.RELEASE</version>
</dependency>

消费者

public class KafkaConsumer {
	public static void main(String args[]) {
    //集群配置文件放置位置
	System.setProperty("java.security.krb5.conf",
"C:\\Users\\XXXX\\Desktop\\keytab\\krb5.conf");
	System.setProperty("java.security.auth.login.config", "C:\\Users\\XXXX\\Desktop\\kafka\\jaas.conf");
		KerberosConfig.getKerberosJaas();
    Properties props = new Properties(); 
    //集群IP端口配置
    props.put("bootstrap.servers", "ip1:9092,ip2:9092,ip3:9092");
    //
    props.put("group.id", "group1"); 
    props.put("enable.auto.commit", "false"); 
    props.put("auto.commit.interval.ms", "1000");
    props.put("auto.offset.reset", "earliest"); 
    props.put("session.timeout.ms", "30000"); 
    props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer"); 
        props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer"); 
        props.put("sasl.kerberos.service.name", "kafka");
        props.put("security.protocol", "SASL_PLAINTEXT");
        props.put("sasl.mechanism", "GSSAPI"); 
        KafkaConsumer<String, String> kafkaConsumer = new KafkaConsumer<>(props); 
//topic
        kafkaConsumer.subscribe(Arrays.asList("test"));
        while (true) { 
            ConsumerRecords<String, String> records = kafkaConsumer.poll(1); 
             for (ConsumerRecord<String, String> record : records)
                 System.out.println("Partition: " + record.partition() + " Offset: " + record.offset() + " Value: " + record.value() + " ThreadID: " + Thread.currentThread().getId());
                
        }
	}
}

注意：1.krb5.conf，在集群安装服务器上，默认在/etc文件下，；

          2.kafka_client_jaas.conf,详见kafka安装目录下jaas.conf文件，默认目录/var/run/cloudera-scm-agent/process/XXX-kafka-KAFKA_BROKER，该文件中的kafka.keytab和jaas.conf配件拷出到需要位置，在jaas.conf文件中添加：

KafkaClient {
   com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   storeKey=true
   keyTab="file:C:/Users/XXX/Desktop/kafka/kafka.keytab"
   principal="kafka/XXXXX@BIGDATA.COM";
};

参考链接：1.https://blog.csdn.net/TXBSW/article/details/82768955

                  2.https://blog.csdn.net/qq_36860032/article/details/91450809