kafka sasl_ssl配置

一、切换到存储证书的路径

我这里在家目录中的创建了ssl文件夹
mkdir ssl && cd ssl

二、生成服务端密钥库

keytool -keystore server.keystore.jks -alias localhost -validity 365 -genkey

验证证书:

keytool -list -v -keystore server.keystore.jks

三、创建CA 并将CA导入到truststore中

生成CA:
openssl req -new -x509 -keyout ca-key -out ca-cert -days 365

将CA导入到truststore中:
keytool -keystore server.truststore.jks -alias CARoot -import -file ca-cert
keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert

四、从KeyStore导出证书,并用上述步骤生成的CA来签名生成的证书:

导出证书:
keytool -keystore server.keystore.jks -alias localhost -certreq -file cert-file

签名:
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days 365 -CAcreateserial -passin pass:ka123456

 五、将证书导入KeyStore

keytool -keystore server.keystore.jks -alias CARoot -import -file ca-cert
keytool -keystore server.keystore.jks -alias localhost -import -file cert-signed

六、生成客户端密钥库

keytool -keystore client.keystore.jks -alias localhost -validity 365 -keyalg RSA -genkey

七、配置zookeeper认证:

1、zookeeper是homebrew安装kafka是安装,配置文件路径为:/usr/local/opt/kafka/libexec/config/,在此目录下创建zookeeper-server-jaas.conf配置文件,定义了两个用户,可提供给生产者和消费者使用,格式为:user_用户名="用户密码",内容如下:

Server {
  org.apache.zookeeper.server.auth.DigestLoginModule required
  user_admin="qaz!@#QAZ"
  user_kafka="qaz123!@#QAZ";
};

2、zookeeper配置文件zookeeper.properties中新增SASL认证配置,如下:

dataDir=/usr/local/var/lib/zookeeper
# the port at which the clients will connect
clientPort=2181
# disable the per-ip limit on the number of connections since this is a non-production config
maxClientCnxns=0
# Disable the adminserver by default to avoid port conflicts.
# Set the port to something non-conflicting if choosing to enable this
admin.enableServer=false
# admin.serverPort=8080
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000

3、在/usr/local/opt/zookeeper/libexec/bin/zkEnv.sh脚本中新增jvm参数,让其启动时加载jaas配置文件

export SERVER_JVMFLAGS="-Xmx${ZK_SERVER_HEAP}m $SERVER_JVMFLAGS -Djava.security.auth.login.config=/usr/local/opt/kafka/libexec/config/zookeeper-server-jaas.conf"

# default heap for zookeeper client
ZK_CLIENT_HEAP="${ZK_CLIENT_HEAP:-256}"
export CLIENT_JVMFLAGS="-Xmx${ZK_CLIENT_HEAP}m $CLIENT_JVMFLAGS"

八、配置kafka安全认证:

1、在/usr/local/opt/kafka/libexec/config目录下创建kafka-server-jaas.conf和kafka-client-jaas.conf配置文件,内容如下:

kafka-server-jaas.conf:
KafkaServer {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="admin"
  password="qazQAZ123!@#"
  user_admin="qazQAZ123!@#"
  user_kafka="qaz!@#123";
};

Client {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="kafka"
  password="qaz123!@#QAZ";
};



kafka-client-jaas.conf:
KafkaClient {
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="kafka"
        password="qaz!@#123";
};

2、在kafka启动脚本(/usr/local/opt/kafka/libexec/bin/kafka-server-start.sh)配置环境变量,指定jaas.conf文件,增加如下代码:

-Djava.security.auth.login.config=/usr/local/Cellar/kafka/3.2.0/libexec/config/kafka-server-jaas.conf

3、修改kafka配置文件:/usr/local/opt/kafka/libexec/config/server.properties

九、java端代码

将步骤三生成的客户信任库client.truststore.jks复制到到java项目中,以便client可以信任这个CA

生产者:

// 1. 创建 kafka 生产者的配置对象
Properties props = new Properties();

// 2. 给 kafka 配置对象添加配置信息:bootstrap.servers
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "127.0.0.1:9093");  props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,StringSerializer.class.getName());
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
props.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG,"SASL_SSL");
props.put(SaslConfigs.SASL_MECHANISM,"PLAIN"); props.put(SaslConfigs.SASL_JAAS_CONFIG,"org.apache.kafka.common.security.plain.PlainLoginModule required username='kafka' password='qaz!@#123';");
props.put(SslConfigs.SSL_ENDPOINT_IDENTIFICATION_ALGORITHM_CONFIG,"");      props.put(SslConfigs.SSL_TRUSTSTORE_LOCATION_CONFIG,DEFAULT_BASE_CONF_PATH+"client.truststore.jks");
props.put(SslConfigs.SSL_TRUSTSTORE_PASSWORD_CONFIG,"ka123456");
// 3. 创建 kafka 生产者对象
KafkaProducer<String, String> producer = new KafkaProducer<>(props);
System.out.println("开始发送数据");

String msg = "Hello World";
// 4. 调用 send 方法,发送消息
producer.send(new ProducerRecord<String, String>("topic1","value ",msg));
// 5. 关闭资源
producer.close();

消费者:

String filePath = file.getAbsolutePath();
// 1.创建消费者的配置对象
Properties props = new Properties();
// 2.给消费者配置对象添加参数
props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "127.0.0.1:9093");
// 配置序列化 必须
props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());

// 配置消费者组(组名任意起名) 必须
props.put(ConsumerConfig.GROUP_ID_CONFIG, "test-group1");
props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG,"earliest");
props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG,false);  //关闭自动提交
props.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG,1);
props.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG,"SASL_SSL");
props.put(SaslConfigs.SASL_MECHANISM,"PLAIN");  
     props.put(SaslConfigs.SASL_JAAS_CONFIG,"org.apache.kafka.common.security.plain.PlainLoginModule required username='kafka' password='qaz!@#123';");

props.put(SslConfigs.SSL_ENDPOINT_IDENTIFICATION_ALGORITHM_CONFIG,"");      props.put(SslConfigs.SSL_TRUSTSTORE_LOCATION_CONFIG,filePath+"/conf/client.truststore.jks");
props.put(SslConfigs.SSL_TRUSTSTORE_PASSWORD_CONFIG,"ka123456");

// 创建消费者对象
KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
// 注册要消费的主题(可以消费多个主题)
ArrayList<String> topics = new ArrayList<>();
topics.add("topic1");
consumer.subscribe(topics);
System.out.println("准备接收数据.........");
// 拉取数据打印
while (true) {
    ConsumerRecords<String, String> records = consumer.poll(Duration.ofSeconds(10));
    for(ConsumerRecord<String,String> record : records){
        String value = record.value();
        System.out.println(value);
    }
    if(records.count() > 0) {
        //手动异步提交offset,当前线程提交offset不会阻塞,可以继续处理后面的程序逻辑
        consumer.commitAsync(new OffsetCommitCallback() {
            @Override
            public void onComplete(Map<TopicPartition, OffsetAndMetadata> offsets, Exception exception) {
                if (exception != null) {
                    System.err.println("Commit falide for" + offsets);
                    System.err.println("Commit falide exception" + exception.getStackTrace());
                }
            }
         });
    }
}

  • 2
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值