37.Java连接Kafka

最新推荐文章于 2024-08-05 07:00:00 发布

大勇任卷舒

最新推荐文章于 2024-08-05 07:00:00 发布

阅读量1.5k

点赞数

分类专栏： Hadoop 文章标签： kafka java big data hadoop CDH

@大勇若怯任卷舒

本文链接：https://blog.csdn.net/m0_47454596/article/details/119903626

版权

37.1 实验环境

Intellij已安装且正常运行
Maven环境正常
RedHat7.2
CM和CDH版本为5.11.2
Kafka2.2.0-0.10.2
创建topic，test3有3个replication，3个partition

[ec2-user@ip-172-31-22-86~]$ kafka-topics --create --zookeeper ip-172-31-22-86.ap-southeast-1.compute.internal:2181 --replication-factor 3 --partitions 3 --topic test3

krb5.conf配置（直接使用CDH集群的Kerberos配置）

# Configuration snippets may beplaced in this directory as well
includedir /etc/krb5.conf.d/

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc =FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 default_realm = CLOUDERA.COM
 #default_ccache_name = KEYRING:persistent:%{uid}

[realms]
 CLOUDERA.COM = {
  kdc =ip-172-31-22-86.ap-southeast-1.compute.internal
  admin_server = ip-172-31-22-86.ap-southeast-1.compute.internal
 }

[domain_realm]
 .ip-172-31-22-86.ap-southeast-1.compute.internal= CLOUDERA.COM
 ip-172-31-22-86.ap-southeast-1.compute.internal= CLOUDERA.COM

Kerberos的keytab文件
- 使用kadmin为Kerberos账号生成keytab，fayson.keytab文件生成在当前目录下。

[ec2-user@ip-172-31-22-86~]$ sudo kadmin.local
Authenticating as principal hdfs/admin@CLOUDERA.COM with password.
kadmin.local:  xst -norandkey -k fayson.keytab fayson@CLOUDERA.COM   
...
kadmin.local:  exit
[ec2-user@ip-172-31-22-86~]$