ELK和Kerberos认证

tof21

已于 2023-02-15 16:37:23 修改

阅读量898

点赞数 1

分类专栏： ELK Kerberos 文章标签： Elasticsearch Kerberos Kibana FileBeat

于 2021-01-12 11:48:06 首次发布

本文链接：https://blog.csdn.net/sjdl9396/article/details/112514213

版权

ELK Kerberos 专栏收录该内容

2 篇文章 0 订阅

订阅专栏

ELK和Kerberos认证

1.简介

ES从6.4.0开始支持Kerberos认证，此功能不开源，需要会员。但是下载安装功能，可以开启免费试用30天。

注：Elasticsearch简称ES

2.版本

名称	服务器	版本	IP
KDC服务端	centos7	1.15.1-50.el7	192.168.174.132
Kerberos客户端	centos7	1.15.1-50.el7	192.168.174.133
Kerberos Windows客户端	window10	kfw-4.1-amd64.msi	192.168.174.1
FireFox	window10	84.0.2	192.168.174.1
ES	centos7	7.10.1	192.168.174.133
Kibana	centos7	7.10.1	192.168.174.133
Filebeat	centos7	7.10.1	192.168.174.133

3.安装和启动

3.1.进行国内源配置

cat <<EOF > /etc/yum.repos.d/elk.repo
[elk]
name=elk
baseurl=https://mirrors.bfsu.edu.cn/elasticstack/yum/elastic-7.x/
enable=1
gpgcheck=0
EOF

3.2.安装ES

更多详细说明请参见官方指南：Install Elasticsearch with RPM | Elasticsearch Guide [8.1] | Elastic

yum makecache
#安装指定版本
yum install --enablerepo=elk elasticsearch-7.10.1

# 开始
sudo -i service elasticsearch start
# 停止
sudo -i service elasticsearch stop
# 重启
sudo -i service elasticsearch restart
# 查看状态
systemctl status elasticsearch.service -l

# 日志默认位置
/var/log/elasticsearch/

# 配置文件默认位置
/etc/elasticsearch

# /etc/elasticsearch中配置访问地址
# 外网无法访问，需要配置网络地址
network.host: 0.0.0.0
discovery.seed_hosts: ["192.168.174.123", "[::1]"]

#验证
#浏览器访问
http://192.168.174.133:9200
#shell命令中
curl http://192.168.174.133:9200
#显示以下信息表示成功
{
  "name" : "Cp8oag6",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "AT69_T_DTp-1qgIJlatQqA",
  "version" : {
    "number" : "7.10.1",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "f27399d",
    "build_date" : "2016-03-30T09:51:41.449Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "1.2.3",
    "minimum_index_compatibility_version" : "1.2.3"
  },
  "tagline" : "You Know, for Search"
}

3.3.安装Kibana

更多详细说明请参见官方指南：Install Kibana with RPM | Kibana Guide [7.10] | Elastic

#安装指定版本
yum install --enablerepo=elk kibana-7.10.1

# 开始
sudo -i service kibana start
# 停止
sudo -i service kibana stop
# 重启
sudo -i service kibana restart
# 查看状态
systemctl status kibana.service -l

# 日志默认位置
/var/log/kibana/

# 配置文件默认位置
/etc/kibana

# /etc/kibana/kibana.yml
server.host: "192.168.174.133"
elasticsearch.hosts: ["http://192.168.174.133:9200"]

# 验证
# 浏览器中输入地址
http://192.168.174.132:5601

3.4.安装Filebeat

更多详细说明请参见官方指南：Filebeat quick start: installation and configuration | Filebeat Reference [8.1] | Elastic

#安装指定版本
yum install --enablerepo=elk filebeat-7.10.1

# 开始
sudo -i service filebeat start
# 停止
sudo -i service filebeat stop
# 重启
sudo -i service filebeat restart
# 查看状态
systemctl status filebeat.service -l

# 日志默认位置
/var/log/filebeat/

# 配置文件默认位置
/etc/filebeat

# /etc/filebeat/filebeat.yml

# 增加调试日志
logging.level: trace
logging.to_files: true
logging.selectors: ['*']
logging.files:
  path: /var/log/filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0644

# 文本日志传输
- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /etc/filebeat/spring.log
    
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["192.168.174.133:9200"]

# 验证
# 可参看filebeat的日志，会显示传输成功信息，或通过ES的api接口查看filebeat的索引是否创建，或通过kibana查看

4.ES开启Kerberos

更多详细说明请参见官方指南：Kerberos authentication | Elasticsearch Guide [8.1] | Elastic

4.1.Kerberos的服务端增加ES相关配置

为ES服务器创建用户信息

[root@sunjing132 /]# kadmin.local
Authenticating as principal admin/admin@EXAMPLE.COM with password.
# 必须是 HTTP/服务名@EXAMPLE.COM
# 输入密码
kadmin.local:   addprinc HTTP/192.168.174.133@EXAMPLE.COM
# 导出keystore
kadmin.local:   ktadd -k /var/kerberos/sunjing/es.keytab -norandkey HTTP/192.168.174.133@EXAMPLE.COM

创建映射ES用户的信息，创建test和elastic用户

[root@sunjing132 /]# kadmin.local
Authenticating as principal admin/admin@EXAMPLE.COM with password.
kadmin.local:   addprinc test@EXAMPLE.COM
kadmin.local:   addprinc elastic@EXAMPLE.COM

4.2.配置用户和用户密码

启动ES，运行命令设置用户和用户密码

# https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-passwords.html
# /usr/share/elasticsearch/bin
bin/elasticsearch-setup-passwords interactive

4.4.配置ES

将kerberos中导出HTTP/192.168.174.133@EXAMPLE.COM的keystab放置到/etc/elasticsearch目录下

/etc/elasticsearch/elasticsearch.yml配置开启

# 试用版本 + 必须开启xpack的security的验证
xpack.license.self_generated.type: trial
xpack.security.enabled: true
xpack.security.audit.enabled: true

http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization

# 开启kerberos验证
xpack.security.authc.realms.kerberos.kerb1:
  order: 0
  # kerberos中导出HTTP/192.168.174.133@EXAMPLE.COM的keytab
  keytab.path: es.keytab
  remove_realm_name: false
  krb.debug: true

/etc/elasticsearch/jvm.options中增加kerberos的conf和debug调试信息

-Djava.security.krb5.conf=/etc/krb5.conf
#调试信息
-Dsun.security.krb5.debug=true
-Dsun.security.spnego.debug=true
-Dsun.security.jgss.debug=true

4.2.验证

查看/etc/log/elasticsearch/elasticsearch.log，表示是试用版本且安全功能已开启。

注意：需要看多更多详细日志，可以更改/ect/elasticsearch/log4j2.properties日志级别

Active license is now [TRIAL]; Security is enabled

curl命令验证

# kerberos未认证用户，会提示401的错误，且有WWW-Authenticate中包含Negotiate的头
[root@sunjing133 ~]# curl --negotiate -u : -X  GET http://192.168.174.133:9200/
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Negotiate"]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Negotiate"]}},"status":401}

# 认证用户
[root@sunjing133 ~]# kinit test@EXAMPLE.COM
Password for test@EXAMPLE.COM:

# 再次访问
[root@sunjing133 ~]# curl --negotiate -u : -X  GET http://192.168.174.133:9200/
{
  "name" : "sunjing133",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "Q3LUh1HtSQiU6mkMjbElFg",
  "version" : {
    "number" : "7.10.1",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "1c34507e66d7db1211f66f3513706fdf548736aa",
    "build_date" : "2020-12-05T01:00:33.671820Z",
    "build_snapshot" : false,
    "lucene_version" : "8.7.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

window浏览器验证

打开火狐浏览器(具体配置参见Kerberos中在window中的配置)，输入地址后，会弹出认证用户的对话框，输入kerberos中存在的用户，即可正常显示页面，认证成功后，当kerberos认证用户未过期时，不会再弹出Kerberos认证对话框

4.3.ES生成证书

kibana访问开启Kerberos的ES，必须使用ssl

1.Create a certificate authority for your Elasticsearch cluster

# /usr/share/elasticsearch/bin
./bin/elasticsearch-certutil ca

2.Generate a certificate and private key for each node in your cluster

./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12

3.generate certificate signing requests (CSR) for the nodes in your cluster

# 需要指定IP地址的参数配置，可能在访问时，ES日志会提示相关错误
elasticsearch-certutil csr

4.Generate additional certificates specifically for encrypting HTTP client communications

./bin/elasticsearch-certutil http

The output is a .zip file containing one directory each for both Elasticsearch and Kibana:

/elasticsearch
|_ README.txt
|_ http.p12
|_ sample-elasticsearch.yml

/kibana
|_ README.txt
|_ elasticsearch-ca.pem
|_ sample-kibana.yml

4.4.ES配置开启https

/etc/elasticsearch/elasticsearch.yml中增加

xpack.security.transport.ssl.enabled: true

xpack.security.http.ssl.enabled: true
# 复制elasticsearch-ssl-http.zip\elasticsearch\http.p12到/etc/elasticsearch目录下
xpack.security.http.ssl.keystore.path: "http.p12"
xpack.security.authc.token.enabled: true

验证https配置后，Kerberos是否成功，在window的firefox浏览器中通过https://192.168.174.133验证即可，因为绑定的证书，使用curl命令时，需要指定证书，此命令暂时未进行尝试。

5.Kibana配置开启Kerberos

更多详细说明，请参考官方文档：Authentication in Kibana | Kibana Guide [7.10] | Elastic

5.1.配置Kerberos

/etc/kibana/kibana.yml

elasticsearch.hosts: ["https://192.168.174.133:9200"]

# 在es中elasticsearch-setup-passwords设置的用户密码，此处不设置，kibana无法启动，可能是因为某些功能内置打开
elasticsearch.username: "kibana_system"
elasticsearch.password: "sunjing21"

# elasticsearch-certutil http生成的kibana认证文件
elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/config/elasticsearch-ca.pem"]

# 配置支持普通登录(需要设置ES用户映射Kerberos用户角色) + kerberos登录
xpack.security.authc.providers:
  kerberos.kerberos1:
    order: 0
    description: "Log in with Kerberos"
  basic.basic1:
    order: 1

启动kibana，使用普通登录，输入用户名elastic，登录后配置ES的角色映射(也可以使用ES的API设置，具体可参看官方文档)，设置成功后，退出。

5.2.使用Kerberos登录ES

当Kerberos用户未认证时，会弹出认证输入对话框，认证成功后，第二次无需在输入Kerberos的用户名和密码，当认证信息未过期

注：Kerberos认证对话框输入的用户信息必须是在ES中进行了角色映射的，否则会提示用户信息不合法，需要从日志中查看。

5.Filebeat输出到ES

更多详细说明，请参考官方文档：Configure Kerberos | Filebeat Reference [7.10] | Elastic

/etc/filebeat/filebeat.yml

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["192.168.174.133:9200"]

  # Protocol - either `http` (default) or `https`.
  protocol: "https"

  # Authentication credentials - either API key or username/password.
  #ssl.enabled: true
  #ssl.renegotiation: freely
  kerberos.enabled: true
  kerberos.auth_type: keytab
  kerberos.keytab: "/etc/elasticsearch/es.keytab"
  kerberos.config_path: "/etc/krb5.conf"
  kerberos.username: "test"
  kerberos.realm: "EXAMPLE.COM"
  # 同kibana中配置的certificateAuthorities文件相同
  ssl.certificate_authorities: ["/etc/filebeat/elasticsearch-ca.pem"]
  #kerberos.password: "sunjing21"

查看日志，会发现打印Kerberos Client创建成功，但是传输的时候，报401的错误，此问题暂时未解决

2021-01-11T17:28:50.601+0800 INFO eslegclient/connection.go:99 elasticsearch url: http://192.168.174.133:9200
2021-01-11T17:28:50.604+0800 INFO eslegclient/connection.go:159 kerberos client created

2021-01-11T17:39:28.399+0800 ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(elasticsearch(https://192.168.174.133:9200)): 401 Unauthorized: {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Negotiate","Bearer realm=\"security\"","ApiKey"]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Negotiate","Bearer realm=\"security\"","ApiKey"]}},"status":401}

2021-01-11T17:39:28.399+0800 INFO [publisher_pipeline_output] pipeline/output.go:145 Attempting to reconnect to backoff(elasticsearch(https://192.168.174.133:9200)) with 5 reconnect attempt(s)

6.参考资料

配置 SSL、TLS 以及 HTTPS 来确保 Elasticsearch、Kibana、Beats 和 Logstash 的安全 | Elastic Blog

https://www.elastic.co/guide/en/elasticsearch/reference/7.x/trb-security-kerberos.htm

https://github.com/elastic/logstash/issues/9785

Kerberos Authentication for Elasticsearch · Issue #7404 · elastic/beats · GitHub

Kafka安全认证：SASL/GSSAPI(kerberos)_daijiguo的博客-CSDN博客_kafka kerberos认证

配置 SSL、TLS 以及 HTTPS 来确保 Elasticsearch、Kibana、Beats 和 Logstash 的安全 | Elastic Blog

tof21

关注

1
点赞
踩
0

收藏

觉得还不错? 一键收藏
打赏
0
评论
ELK和Kerberos认证

ELK和Kerberos认证1.简介ES从6.4.0开始支持Kerberos认证，此功能不开源，需要会员。但是下载安装功能，可以开启免费试用30天。注：Elasticsearch简称ES2.版本名称服务器版本 IP KDC服务端 centos7 1.15.1-50.el7 192.168.174.132 Kerberos客户端 centos7 1.15.1-50.el7 192.168.174.133 Kerberos
复制链接

扫一扫