问题原因
6.5.4 更新了x-pack,在ES外置时如果使用用户认证时就需要开启x-pack，但是需要购买x-pack,此时就在网上找了一个插件支持对elasticsearch的认证。
解决办法
在代码里面增加了searchguard插件的支持，此插件使用transport连接也需要进行SSL认证，认证的过程需要使用到证书。
版本
es 版本6.5.4
集群名称请使用默认名称（elasticsearch 不自己修改默认就是这个）
配置elasticsearch的search-guard插件
下载search-guard插件
下载tsl tool 插件生成证书
https://releases.floragunn.com/search-guard-tlstool/1.8/search-guard-tlstool-1.8.zip
关闭现有的服务（可选）
curl -Ss -XPUT 'https://{esIp}:9200/_cluster/settings?pretty' \
-H 'Content-Type: application/json' -d'
{
"persistent": {
"cluster.routing.allocation.enable": "none"
}
}
'
如果有数据建议执行此步骤
安装search-guard插件
如果多个结点，每个结点都要执行
bin/elasticsearch-plugin install -b file:///path/to/search-guard-6-<version>.zip
tls tool生成证书
环境要求：需要有JDK，建议不低于1.8
解压
tar -zxvf search-guard-tlstool-1.8.tar.gz
编辑模板
root@glp-dsg:~/search-guard-tlstool# pwd
/root/search-guard-tlstool
root@glp-dsg:~/search-guard-tlstool# cd config/
root@glp-dsg:~/search-guard-tlstool/config# cp template.yml skyguard.yml
root@glp-dsg:~/search-guard-tlstool/config# vi skyguard.yml
配置信息可以参考这个
###
### Self-generated certificate authority
###
#
# If you want to create a new certificate authority, you must specify its parameters here.
# You can skip this section if you only want to create CSRs
#
ca:
root:
# The distinguished name of this CA. You must specify a distinguished name.
# example: dn: CN=root.ca.example.com,OU=CA,O=Example Com\, Inc.,DC=example,DC=com
dn: CN=root.ca.test.com,OU=CA,O=test Com\, Inc.,DC=test,DC=com
# The size of the generated key in bits
keysize: 2048
# The validity of the generated certificate in days from now
validityDays: 3650
# Password for private key
# Possible values:
# - auto: automatically generated password, returned in config output;
# - none: unencrypted private key;
# - other values: other values are used directly as password
pkPassword: test
# The name of the generated files can be changed here
file: root-ca-test.pem
# If you have a certificate revocation list, you can specify its distribution points here
# crlDistributionPoints: URI:https://raw.githubusercontent.com/floragunncom/unittest-assets/master/revoked.crl
# If you want to use an intermediate certificate as signing certificate,
# please specify its parameters here. This is optional. If you remove this section,
# the root certificate will be used for signing.
#intermediate:
# # The distinguished name of this CA. You must specify a distinguished name.
# # example: dn: CN=root.ca.example.com,OU=CA,O=Example Com\, Inc.,DC=example,DC=com
# dn: ...
# The size of the generated key in bits
# keysize: 2048
# The validity of the generated certificate in days from now
# validityDays: 3650
# Password for private key
# pkPassword: auto
# The name of the generated files can be changed here
# file: signing-ca.pem
# If you have a certificate revocation list, you can specify its distribution points here
# crlDistributionPoints: URI:https://raw.githubusercontent.com/floragunncom/unittest-assets/master/revoked.crl
###
### Default values and global settings
###
defaults:
# The validity of the generated certificate in days from now
validityDays: 3650
# Password for private key
# Possible values:
# - auto: automatically generated password, returned in config output;
# - none: unencrypted private key;
# - other values: other values are used directly as password
pkPassword: test
# Specifies to recognize legitimate nodes by the distinguished names
# of the certificates. This can be a list of DNs, which can contain wildcards.
# Furthermore, it is possible to specify regular expressions by
# enclosing the DN in //.
# Specification of this is optional. The tool will always include
# the DNs of the nodes specified in the nodes section.
#
# Examples:
# - "CN=*.example.com,OU=Ops,O=Example Com\\, Inc.,DC=example,DC=com"
# - 'CN=node.other.com,OU=SSL,O=Test,L=Test,C=DE'
# - 'CN=*.example.com,OU=SSL,O=Test,L=Test,C=DE'
# - 'CN=elk-devcluster*'
# - '/CN=.*regex/'
# nodesDn:
# If you want to use OIDs to mark legitimate node certificates,
# the OID can be included in the certificates by specifying the following
# attribute
# nodeOid: "1.2.3.4.5.5"
# The length of auto generated passwords
generatedPasswordLength: 12
# Set this to true in order to generate config and certificates for
# the HTTP interface of nodes
httpsEnabled: true
# Set this to true in order to re-use the node transport certificates
# for the HTTP interfaces. Only recognized if httpsEnabled is true
# reuseTransportCertificatesForHttp: false
# Set this to true to enable hostname verification
#verifyHostnames: false
# Set this to true to resolve hostnames
#resolveHostnames: false
# Use elliptic curves instead of RSA, Default is false.
#useEllipticCurves: true
# The curve to be used (default is NIST P-384)
#ellipticCurve: secp256k1
# Signature algorithm (for RSA default is SHA256withRSA, for EC its SHA256withECDSA)
#signatureAlgorithm: SHA512withRSA
###
### Nodes
###
#
# Specify the nodes of your ES cluster here
#
nodes:
- name: node1
dn: CN=node1.test.com,OU=Ops,O=test Com\, Inc.,DC=test,DC=com
dns: node1.test.com
- name: node2
dn: CN=node2.test.com,OU=Ops,O=test Com\, Inc.,DC=test,DC=com
dns: node2.test.com
- name: node3
dn: CN=node3.test.com,OU=Ops,O=test Com\, Inc.,DC=test,DC=com
dns: node3.example.com
###
### Clients
###
#
# Specify the clients that shall access your ES cluster with certificate authentication here
#
# At least one client must be an admin user (i.e., a super-user). Admin users can
# be specified with the attribute admin: true
#
clients:
- name: sps
dn: CN=sps.test .com,OU=Ops,O=test Com\, Inc.,DC=test ,DC=com
admin: true
生成证书及配置信息
root@glp-dsg:~/search-guard-tlstool/tools# chmod 777 *
root@glp-dsg:~/search-guard-tlstool/tools# ./sgtlstool.sh -c ../config/skyguard.yml -ca -crt
在out目录下可以看到生成的配置信息
-rw-r--r-- 1 root root 294 Feb 23 14:51 client-certificates.readme
-rw-r--r-- 1 root root 1384 Feb 23 14:51 node1_elasticsearch_config_snippet.yml
-rw-r--r-- 1 root root 1801 Feb 23 14:51 node1_http.key
-rw-r--r-- 1 root root 1655 Feb 23 14:51 node1_http.pem
-rw-r--r-- 1 root root 1801 Feb 23 14:51 node1.key
-rw-r--r-- 1 root root 1655 Feb 23 14:51 node1.pem
-rw-r--r-- 1 root root 1384 Feb 23 14:51 node2_elasticsearch_config_snippet.yml
-rw-r--r-- 1 root root 1801 Feb 23 14:51 node2_http.key
-rw-r--r-- 1 root root 1655 Feb 23 14:51 node2_http.pem
-rw-r--r-- 1 root root 1801 Feb 23 14:51 node2.key
-rw-r--r-- 1 root root 1655 Feb 23 14:51 node2.pem
-rw-r--r-- 1 root root 1384 Feb 23 14:51 node3_elasticsearch_config_snippet.yml
-rw-r--r-- 1 root root 1801 Feb 23 14:51 node3_http.key
-rw-r--r-- 1 root root 1651 Feb 23 14:51 node3_http.pem
-rw-r--r-- 1 root root 1801 Feb 23 14:51 node3.key
-rw-r--r-- 1 root root 1651 Feb 23 14:51 node3.pem
-rw-r--r-- 1 root root 1801 Feb 23 14:51 root-ca-skyguard.key
-rw-r--r-- 1 root root 1379 Feb 23 14:51 root-ca-skyguard.pem
-rw-r--r-- 1 root root 1801 Feb 23 14:51 sps.key
-rw-r--r-- 1 root root 1594 Feb 23 14:51 sps.pem
配置信息介绍：
根证书和私钥
root-ca-skyguard.pem root-ca-skyguard.key
结点证书和私钥
Node*.pem Node*.key
Node*_http.pem Node*_http.key
连接证书和私钥
sps.pem sps.key
每个结点的配置信息
node*_elasticsearch_config_snippet.yml
修改ES的配置文件
root@glp-dsg:~/search-guard-tlstool/tools/skyguard# vi /etc/elasticsearch/elasticsearch.yml
# 开关
#searchguard.disabled: true
#需要设置为false不然使用x-pack报错
xpack.security.enabled: false
#使用社区版的不需要授权
searchguard.enterprise_modules_enabled: false
# node*_elasticsearch_config_snippet.yml里面的配置信息
searchguard.ssl.transport.pemcert_filepath: node1.pem
searchguard.ssl.transport.pemkey_filepath: node1.key
searchguard.ssl.transport.pemkey_password: test
searchguard.ssl.transport.pemtrustedcas_filepath: root-ca-test.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.resolve_hostname: false
#方便curl 命令访问改为false ,是true curl请加https和 -k
searchguard.ssl.http.enabled: false
searchguard.ssl.http.pemcert_filepath: node1_http.pem
searchguard.ssl.http.pemkey_filepath: node1_http.key
searchguard.ssl.http.pemkey_password: test
searchguard.ssl.http.pemtrustedcas_filepath: root-ca-test.pem
searchguard.nodes_dn:
- CN=node1.test.com,OU=Ops,O=test Com\, Inc.,DC=test,DC=com
- CN=node2.test.com,OU=Ops,O=test Com\, Inc.,DC=test,DC=com
- CN=node3.test.com,OU=Ops,O=test Com\, Inc.,DC=test,DC=com
#下面两行如果不注释，使用连接证书认证，密码可以随便填写，注释之后即需要证书也需要密码
#searchguard.authcz.admin_dn:
#- CN=sps.test .com,OU=Ops,O=test Com\, Inc.,DC=test,DC=com
证书配置
把下面证书copy到对应的结点
node*.pem
node*.key
node*_http.pem
node*_http.key
root-ca-skyguard.pem
配置完成之后启动elasticsearch服务,并检查服务是否正常
导入相关的searchguard用户（多个结点也只需要执行一次）
修改内置用户文件
root@glp-dsg:/usr/share/elasticsearch/plugins/search-guard-6/sgconfig/
vi sg_internal_users.yml
# This is the internal user database
# The hash value is a bcrypt hash and can be generated with plugin/tools/hash.sh
#password is: admin
admin:
readonly: true
#请自己生成
hash: $2a$12$VcCDgh2NDk07JGN0rjGbM.Ad41qVR/YFJcgHp0UGns5JDymv..TOG
roles:
- admin
attributes:
#no dots allowed in attribute names
attribute1: value1
attribute2: value2
attribute3: value3
#password is: logstash
#logstash:
# hash: $2y$12$FAm8N/u0RTJ3idZqazufK.Ga1kwrJ5SFhLn39q1oAHQ/eKGOiWrmG
# roles:
# - logstash
#password is: kibanaserver
#kibanaserver:
# readonly: true
# hash: $2y$12$FAm8N/u0RTJ3idZqazufK.Ga1kwrJ5SFhLn39q1oAHQ/eKGOiWrmG
#password is: kibanaro
#kibanaro:
# hash: $2y$12$FAm8N/u0RTJ3idZqazufK.Ga1kwrJ5SFhLn39q1oAHQ/eKGOiWrmG
# roles:
# - kibanauser
# - readall
#password is: readall
#readall:
# hash: $2y$12$FAm8N/u0RTJ3idZqazufK.Ga1kwrJ5SFhLn39q1oAHQ/eKGOiWrmG
#password is: readall
# roles:
# - readall
#password is: snapshotrestore
#snapshotrestore:
# hash: $2y$12$FAm8N/u0RTJ3idZqazufK.Ga1kwrJ5SFhLn39q1oAHQ/eKGOiWrmG
# roles:
# - snapshotrestore
这里面的密码是****,请自己通过/tools/hash.sh 工具生成，这里面把其他的用户注释掉，可以根据自己的需求设置不同的。
导入用户数据
root@glp-dsg:/usr/share/elasticsearch/plugins/search-guard-6/tools# chomd 777 *
#执行
./sgadmin.sh -cd ../sgconfig/ -icl -nhnv -cacert /etc/elasticsearch/root-ca-skyguard.pem -cert /etc/elasticsearch/temp/sps.pem -key /etc/elasticsearch/temp/sps.key -keypass ***
sps.pem sps.key 是连接证书 skyguard是私钥的秘钥，在生成证书的时候设置的。
验证
curl http://{esIp}:9200/_cat/indices -u admin:****
如果开启https 请用
curl https://{esIp}:9200/_cat/indices -u admin:***** -k
重新开启
开启现有的服务（可选）
curl -Ss -XPUT 'https://{esIp}:9200/_cluster/settings?pretty' \
-H 'Content-Type: application/json' -d'
{
"persistent": {
"cluster.routing.allocation.enable": "true"
}
}
JAVA连接代码
public class ElasticsearchClientFactory implements FactoryBean<Client> {
//search-guard switch
public static final String SEARCHGUARD_SSL_TRANSPORT_ENABLED = "searchguard.ssl.transport.enabled";
//search-guard SSL ConfigConstants
private static final String SEARCHGUARD_SSL_TRANSPORT_PEMCERT_FILEPATH = "searchguard.ssl.transport.pemcert_filepath";
private static final String SEARCHGUARD_SSL_TRANSPORT_PEMKEY_FILEPATH = "searchguard.ssl.transport.pemkey_filepath";
private static final String SEARCHGUARD_SSL_TRANSPORT_PEMKEY_PASSWORD = "searchguard.ssl.transport.pemkey_password";
private static final String SEARCHGUARD_SSL_TRANSPORT_PEMTRUSTEDCAS_FILEPATH = "searchguard.ssl.transport.pemtrustedcas_filepath";
private static final String SEARCHGUARD_SSL_TRANSPORT_ENFORCE_HOSTNAME_VERIFICATION = "searchguard.ssl.transport.enforce_hostname_verification";
private String host;
private int port;
private String username;
private String password;
public void setHost(String host) {
this.host = host;
}
public void setPort(int port) {
this.port = port;
}
public void setUsername(String username) {
this.username = username;
}
public void setPassword(String password) {
this.password = password;
}
@SuppressWarnings("resource")
@Override
public Client getObject() throws Exception {
if (StringUtils.isBlank(username) || "${elasticsearch.username}".equals(username)) {
return new PreBuiltTransportClient(Settings.EMPTY)
.addTransportAddress(new TransportAddress(InetAddress.getByName(host), port));
} else {
String search_guard_ssl_transport_enabled = BaseConfig.getItem(SEARCHGUARD_SSL_TRANSPORT_ENABLED, "false");
Settings.Builder builder = Settings.builder();
builder.put("cluster.name", BaseConfig.getItem(CLUSTER_NAME, "elasticsearch"));
if (Boolean.valueOf(search_guard_ssl_transport_enabled)) {
builder.put(SEARCHGUARD_SSL_TRANSPORT_PEMCERT_FILEPATH, BaseConfig.getItem(SEARCHGUARD_SSL_TRANSPORT_PEMCERT_FILEPATH,""));
builder.put(SEARCHGUARD_SSL_TRANSPORT_PEMKEY_FILEPATH, BaseConfig.getItem(SEARCHGUARD_SSL_TRANSPORT_PEMKEY_FILEPATH,""));
builder.put(SEARCHGUARD_SSL_TRANSPORT_PEMKEY_PASSWORD, BaseConfig.getItem(SEARCHGUARD_SSL_TRANSPORT_PEMKEY_PASSWORD,""));
builder.put(SEARCHGUARD_SSL_TRANSPORT_PEMTRUSTEDCAS_FILEPATH, BaseConfig.getItem(SEARCHGUARD_SSL_TRANSPORT_PEMTRUSTEDCAS_FILEPATH,""));
builder.put(SEARCHGUARD_SSL_TRANSPORT_ENFORCE_HOSTNAME_VERIFICATION, BaseConfig.getItem(SEARCHGUARD_SSL_TRANSPORT_ENFORCE_HOSTNAME_VERIFICATION,""));
Settings settings = builder.build();
Client client = new PreBuiltTransportClient(settings, SearchGuardPlugin.class).addTransportAddress(new TransportAddress(InetAddress.getByName(host), port));
//官方给的连接方式，只是当前thread有效不能用在工厂模式
client = client.filterWithHeader(Collections.singletonMap("Authorization", "Basic " + Base64.getEncoder().encodeToString((username + ":" + StringUtils.defaultString(password)).getBytes())));
return client;
} else{
//OTHER client
}
}
}
@Override
public Class<?> getObjectType() {
return Client.class;
}
@Override
public boolean isSingleton() {
return true;
}
}
注意事项：
- 生成根证书和私钥最好保存下，方便后续使用。
- 可以选择只用客户端证书不使用密码方法，但请保存好自己证书和私钥
- 如果es是集群请修改索引searchguard,让他的副本数大于0，该索引保存的是用户信息。
参考文档：
版本选择 https://docs.search-guard.com/latest/search-guard-versions
文档地址 https://docs.search-guard.com/6.x-25/
下载生产环境TLS https://docs.search-guard.com/6.x-25/offline-tls-tool
初始化文档 https://docs.search-guard.com/6.x-25/search-guard-installation
663
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
