一、准备
Elasticsearch集群搭建好；我使用的docker方式部署；
二、生成证书
1、进入ES主节点容器，在${ES_HOME}/bin 下执行：
./elasticsearch-certutil ca #创建一个证书颁发机构
再执行：
./elasticsearch-certutil cert --ca elastic-stack-ca.p12
elastic-stack-ca.p12 就是上一步生成的（默认就在当前目录中）
此时在当前目录生成名为 elastic-certificates.p12 的文件，它是单个PKCS#12密钥存储库，其中包括节点证书、节点密钥和CA证书。默认情况下，elasticsearch-certutil 生成没有主机名信息的证书(也就是说，它们没有任何主题可选名称字段)。这意味着可以对集群中的每个节点使用此证书，但您必须关闭主机名验证。
将其从容器中拷贝出来。
备注：elasticsearch-certutil 的更多用法和参数，请参照：https://www.cnblogs.com/sanduzxcvbnm/p/12053850.html
2、生成PKI客户端证书，供KIbana等组件到ES的校验使用
PKI身份验证的证书必须由与用于加密HTTP通信的证书相同的CA进行签名。因此我们直接使用拷贝出的 elastic-certificates.p12，在主机上执行：
# Private Key 私钥
openssl pkcs12 -in elastic-certificates.p12 -nocerts -nodes > client.key
# Public Certificate 公共证书
openssl pkcs12 -in elastic-certificates.p12 -clcerts -nokeys > client.cer
# CA Certificate 签署公共证书的CA
openssl pkcs12 -in elastic-certificates.p12 -cacerts -nokeys -chain > client-ca.cer
3、分发证书文件
将生成的 client.key、client.cer、client-ca.cer 以及 elastic-certificates.p12文件拷贝进（或者另起容器-v挂载进）各个ES节点的${ES_HOME}/config/目录下；
也要拷贝进 kibana 容器的${KIBANA_HOME}/config/certs/目录下；
也要拷贝进logstash容器的${LOGSTASH_HOME}/config/目录下
其实可以拷贝进任意目录，只要后面配置文件中指定正确就行
注意，在这之前先将这些文件 chmod 777
三、各组件配置文件详情
1、elasticsearch.yml
# 集群配置-主节点
cluster.name: "hnjt-es-cluster"
node.name: esm1
network.bind_host: 0.0.0.0
network.publish_host: 192.168.10.180
http.port: 9201
transport.tcp.port: 9301
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true
node.data: false
discovery.zen.ping.unicast.hosts: ["192.168.10.180:9301"]
discovery.zen.minimum_master_nodes: 1
# 加密节点间通信-每个节点容器都要添加
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate #只验证证书是否受信任，不执行主机名验证。还可以设置成 full(还要验证主机名)、none(不验证证书)
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 #指定证书
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12 #将密钥存储库也用做信任存储库
# 加密http通信-每个节点容器都要添加
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: elastic-certificates.p12 #指定使用的证书
xpack.security.http.ssl.truststore.path: elastic-certificates.p12 #密钥存储库包含CA证书，因此也可以用作信任存储库
2、kibana.yml
server.port: 5601
server.host: 0.0.0.0
elasticsearch.hosts: ["https://192.168.10.180:9201"]
# 配置连接ES
xpack.security.enabled: true
elasticsearch.ssl.verificationMode: certificate #不验证主机名
elasticsearch.ssl.certificate: /usr/share/kibana/config/certs/client.cer #指定证书
elasticsearch.ssl.key: /usr/share/kibana/config/certs/client.key #指定私钥
elasticsearch.ssl.certificateAuthorities: ["/usr/share/kibana/config/certs/client-ca.cer"] #指定受信任的公共证书
elasticsearch.username: "username"
elasticsearch.password: "password"
# 使用https访问kibana
server.ssl.enabled: true
server.ssl.certificate: /usr/share/kibana/config/certs/client.cer
server.ssl.key: /usr/share/kibana/config/certs/client.key
3、logstash.conf
只摘取 output 部分，其他无特别
output {
stdout {
codec => rubydebug
}
elasticsearch {
hosts => ["https://192.168.10.180:9201"]
ssl => true #启用ssl/tls安全通信
ssl_certificate_verification => false #是否验证证书。在实际实验中我认为官网的介绍是错误的，应该是是否验证主机名，证书还是要验的
cacert => "/usr/share/logstash/config/client-ca.cer" #验证服务器证书的.cer或.pem文件
index => "logstash-%{+YYYY.MM.dd}"
user => "username"
password => "password"
}
}
4、metricbeat.yml
output.elasticsearch部分：
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["https://192.168.10.180:9201"]
username: "metricbeat-write"
password: "123456"
# protocol: "https"
ssl.enabled: true
ssl.verification_mode: none #是否验证服务器的证书和主机名。none：接受所有证书和主机名；full：验证证书和主机名。只有这两个值，设置为none的话，下面3个配置可以不添加
ssl.certificate_authorities: "/data/elk/es_dev/metricbeat/client-ca.cer"
ssl.certificate: "/data/elk/es_dev/metricbeat/client.cer"
ssl.key: "/data/elk/es_dev/metricbeat/client.key"
四、遇到的问题
1、PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
这是启动 logstash 时遇到的问题
原因是 在 logstash.conf 中虽然配置了 ssl_certificate_verification => false ，但是没配置 cacert => “/usr/share/logstash/config/client-ca.cer”。个人认为官网上对 ssl_certificate_verification 参数的介绍是错误的
2、cannot validate certificate for because it doesn’t contain any IP SANs
这是启动 metricbeat 时遇到的问题
原因：metricbeat 默认需要的证书要包含ip或主机名。而我生成证书没包含主机名和IP。
解决：如上，在 metricbeat.yml 中添加 ssl.verification_mode: none
3、[Error connection to Elasticsearch https://192.168.10.180:9201: 401 Unauthorized…“type”:“security_exception”,“reason”:“failed to authenticate user [elastic]”…
这是启动 metricbeat 时遇到的问题
原因：密码中含有符号 ‘$’ ，在yml文件和url里属于特殊字符，怀疑是这个原因。但是logstash.conf文件使用此密码没有问题。
解决：分别尝试用单引号、双引号、转义符、ASCII码来设置密码，都没有成功。更改密码为字母与数字就好了
五、java 连接elasticsearch
1、pom依赖
<dependencies>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
<version>6.8.10</version>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>6.8.10</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>6.8.10</version>
</dependency>
</dependencies>
2、导包
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.elasticsearch.action.main.MainResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
import org.apache.http.ssl.SSLContextBuilder;
import org.apache.http.ssl.SSLContexts;
import javax.net.ssl.SSLContext;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.security.KeyManagementException;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.CertificateException;
3、代码
final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("elastic", "P@$$W0rd"));
Path keyStorePath = Paths.get("G:\\TLS\\elastic-certificates.p12"); //密钥存储库
String keyStorePass = ""; //库的密码
KeyStore truststore = KeyStore.getInstance("jks");
try (InputStream is = Files.newInputStream(keyStorePath)) {
try {
truststore.load(is, keyStorePass.toCharArray());
} catch (CertificateException e) {
e.printStackTrace();
}
} catch (IOException e) {
e.printStackTrace();
}
SSLContextBuilder sslBuilder = null;
try {
sslBuilder = SSLContexts.custom().loadTrustMaterial(truststore, null);
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (KeyStoreException e) {
e.printStackTrace();
}
final SSLContext sslContext = sslBuilder.build();
// HttpHost 可以添加多个。new HttpHost("localhost", 19200, "http"),new HttpHost("localhost", 19201, "http")
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("192.168.10.180", 9201, "https"))
.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
httpClientBuilder.disableAuthCaching();
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
return httpClientBuilder
.setSSLContext(sslContext)
.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE); //不验证主机名
}
}));
// 测试连接
try {
MainResponse res = client.info(RequestOptions.DEFAULT);
System.out.println(res.getClusterName());
System.out.println(res.getVersion());
} catch (IOException e1) {
e1.printStackTrace();
}
client.close();
1311
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
