一、ELK配置X-Pack
1. elasticsearch配置x-pack
①替换原来的x-pack-core-6.4.2.jar
- 下载破解版,替换原有jar,路径如下:
# es安装目录
/....../elasticsearch-6.4.2/modules/x-pack-core/x-pack-core-6.4.2.jar
- 修改es配置文件
# 进入es安装目录
cd /....../elasticsearch-6.4.2
# 打开配置文件
vim config/elasticsearch.yml
# 添加以下内容
network.host: esIP地址(不要写127.0.0.1,否则外网无法访问)
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
- 启动es
# 进入es安装目录
cd /....../elasticsearch-6.4.2
# 后台启动es
nohup ./bin/elasticsearch &
# 查看启动日志
tail -f nohup.out
- 激活30天试用期
curl -H "Content-Type:application/json" -XPOST http://esIP:9200/_xpack/license/start_trial?acknowledge=true
- 设置密码(①手动设置密码,会为四个用户 elastic,kibana,logstash_system,beats_system设置密码,可以设置相同的密码,要不然容易忘记;②自动设置密码,会自动为四个用户 elastic,kibana,logstash_system,beats_system设置不同的密码)
# 进入es安装目录
cd /....../elasticsearch-6.4.2
# 执行以下命令手动设置密码
bin/elasticsearch-setup-passwords interactive
# 开始设置密码
Initiating the setup of passwords for reserved users elastic,kibana,logstash_system,beats_system.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y
Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [kibana]:
Reenter password for [kibana]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [elastic]
# 也可以执行以下命令自动设置密码
bin/elasticsearch-setup-passwords auto
- 测试
不加用户名密码,直接访问
# 不加用户名密码,直接访问
curl http://esIp:9200
# 返回401,认证失败,结果如下:
{
"error":{
"root_cause":[
{
"type":"security_exception",
"reason":"missing authentication token for REST request [/]",
"header":{
"WWW-Authenticate":"Basic realm="security" charset="UTF-8""
}
}
],
"type":"security_exception",
"reason":"missing authentication token for REST request [/]",
"header":{
"WWW-Authenticate":"Basic realm="security" charset="UTF-8""
}
},
"status":401
}
加用户名密码访问
# 加用户名密码访问
curl -u elastic:密码 http://esIp:9200
# 成功返回es信息,结果如下
{
"name" : "Jo78Uy0",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "EnmwXRoG0cASwFkFaUSWyl",
"version" : {
"number" : "6.4.2",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "71c1042",
"build_date" : "2018-09-26T13:34:09.098244Z",
"build_snapshot" : false,
"lucene_version" : "7.4.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
②更新license
现在license只有一个月试用期,需要破解
- 在es安装目录下新建license.json文件
# 进入es安装目录
cd /....../elasticsearch-6.4.2
# 新建license.json文件
vim license.json
# 添加如下内容
{
"license": {
"uid": "9gfhf46-5g78-4f1e-b5a4-afet359bc3a3",
"type": "platinum",
"issue_date_in_millis": 1534723200000,
"expiry_date_in_millis": 2544271999999,
"max_nodes": 100,
"issued_to": "www.plaza4me.com",
"issuer": "Web Form",
"signature": "AAAAAwAAAA3lQFlr4GED3cGRsdfgrDDFEWGN0hjZDBGYnVyRXpCOsdfasdfsgEfghgdg3423MVZwUzRxVk1PSmkxagfsdf3242UWh3bHZVUTllbXNPbzBUemtnbWpBbmlWRmRZb25KNFlBR2x0TXc2K2p1Y1VtMG1UQU9TRGZVSGRwaEJGUjE3bXd3LzRqZ05iLzRteWFNekdxRGpIYlFwYkJiNUs0U1hTVlJKNVlXekMrSlVUdFIvV0FNeWdOYnlESDc3MWhlY3hSQmdKSjJ2ZTcvYlBFOHhPQlV3ZHdDQ0tHcG5uOElCaDJ4K1hob29xSG85N0kvTWV3THhlQk9NL01VMFRjNDZpZEVXeUtUMXIyMlIveFpJUkk2WUdveEZaME9XWitGUi9WNTZVQW1FMG1DenhZU0ZmeXlZakVEMjZFT2NvOWxpZGlqVmlHNC8rWVVUYzMwRGVySHpIdURzKzFiRDl4TmM1TUp2VTBOUlJZUlAyV0ZVL2kvVk10L0NsbXNFYVZwT3NSU082dFNNa2prQ0ZsclZ4NTltbU1CVE5lR09Bck93V2J1Y3c9PQAAAQCGcZtOlZwj0Rnl2MUjERG94a+xcifpVAurIA+z4rroxaqaewpb2MJLZVJt1ZCGeKB0KIWRAm2pkPjM2JigjaPIUBhpW4/yUzbdRtRuQB4loEKd7/p9EbHDh5GzeI8qfkMh3j7QaAlz4Bk+eett+ZNqNXHEdkr+Re9psdnqfUESz1uROhMoYWbn/Bdd0AJLKzhRnEOE972xdnAar8bCP1DIDljI9IOnYhEc6O6CboKCMJY4AWOvJY83bud4FO25hrKf6bMy0F2oO2yUkVV0UiFMX19JbhcC+WIAgxMk/KG7e/MqR8bJ1jNu2usMlgkvV97BxiPogTujFnTQxoHdpNdR",
"start_date_in_millis": 1534723200000
}
}
- 更新license
# 进入es安装目录
cd /....../elasticsearch-6.4.2
# 更新license
curl -XPUT -u elastic:密码 -H "Content-Type:application/json" -v "http://esIp:9200/_xpack/license?acknowledge=true" -d @license.json
- 查看license
curl -u elastic:密码 http://esIp:9200/_license
# 结果如下,过期时间已经到2050年了
{
"license" : {
"status" : "active",
"uid" : "9gfhf46-5g78-4f1e-b5a4-afet359bc3a3",
"type" : "platinum",
"issue_date" : "2018-08-20T00:00:00.000Z",
"issue_date_in_millis" : 1534723200000,
"expiry_date" : "2050-08-16T14:13:19.999Z",
"expiry_date_in_millis" : 2544271999999,
"max_nodes" : 100,
"issued_to" : "www.plaza4me.com",
"issuer" : "Web Form",
"start_date_in_millis" : 1534723200000
}
}
③配置证书
- 生成证书
# 进入es安装目录
cd /....../elasticsearch-6.4.2
# 生成证书
# 执行以下命令
bin/elasticsearch-certutil ca
# 输入文件名 elastic-stack-ca.p12 回车,
# 然后要输入密码,直接回车,不输密码,
# 这时会生成一个 elastic-stack-ca.p12 文件
# 再执行
bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
# 这时要输入生成 elastic-stack-ca.p12 文件的密码,
# 因为我们没有设置,所以直接回车,
# 然后输入文件名 elastic-certificates.p12 回车
# 不输密码,直接回车,
# 这时会生成一个 elastic-certificates.p12 文件
- 在config目录下,新建certs文件夹,将生成的两个文件 elastic-stack-ca.p12 和 elastic-certificates.p12 移动到certs目录
- 修改es配置文件,新增以下内容
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12
- 重启es
# 找到 es 进程
jps
# 杀死进程
kill -9 es进程id
# 启动es
nohup ./bin/elasticsearch &
至此,elasticsearch 配置 x-pack 就完成了。
2. kibana配置x-pack
①修改 kibana 配置文件
# 进入 kibana 安装目录
cd /....../kibana-6.4.2-linux-x86_64
# 打开 kibana 配置文件
vim config/kibana.yml
# 添加如下内容
elasticsearch.username: "kibana"
elasticsearch.password: "密码"
② 启动 kibana
# 找到 kibana 后台进程
fuser -n tcp 5601
# 杀死进程
kill -9 kibana进程id
# 启动 kibana
nohup ./bin/kibana &
③访问 kibana
http://kibanaIp:5601
这时就需要输入用户名密码了。
3. logstash配置x-pack
①修改 logstash 配置文件
# 进入 logstash 安装目录
cd /....../logstash-6.4.2
# 打开配置文件
vim config/logstash-demo.conf
# 添加如下内容
output {
elasticsearch {
hosts => ["esIp:9200"]
index => "***********"
user => "logstash_system"
password => "密码"
}
stdout { codec => rubydebug }
}
② 启动 logstash
# 找到 logstash 后台进程
jps
# 杀死进程
kill -9 logstash进程id
# 启动 logstash
nohup ./bin/logstash -f config/logstash-demo.conf &
二、TransprotClient连接ES
1. 连接未配置x-pack的es
①添加依赖
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>6.4.2</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
<version>6.4.2</version>
<exclusions>
<exclusion>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
</exclusion>
</exclusions>
</dependency>
②创建 TransportClient 客户端
TransportClient transportClient = null;
Settings esSetting = Settings.builder()
.put("cluster.name", clusterName) // 设置集群名称
.put("client.transport.sniff", true) // 设置嗅探机制,找到ES集群
.put("thread_pool.search.size", poolSize) // 设置线程池
.build();
TransportClient transportClient = new PreBuiltXPackTransportClient(esSetting);
TransportAddress transportAddress = new TransportAddress(InetAddress.getByName(hostName), port); // 设置 IP 端口
transportClient.addTransportAddresses(transportAddress);
③通过 TransportClient 操作es
2. 连接已配置x-pack的es
①添加依赖
- 下载 jar 包
maven无法导入x-pack-transport-6.4.2依赖,我这里直接下载两个jar包,x-pack-core-6.4.2.jar 和 x-pack-transport-6.4.2.jar,x-pack-core-6.4.2.jar 用上面的破解版就可以。
x-pack-core-6.4.2.jar下载
x-pack-transport-6.4.2.jar下载 - 导入到项目
在resources目录下新建lib目录,将两个jar包放入,右键add as libraries - 修改 pom 文件
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>6.4.2</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>transport</artifactId>
<version>6.4.2</version>
<exclusions>
<exclusion>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- 新增内容 -->
<dependency>
<groupId>com.unboundid</groupId>
<artifactId>unboundid-ldapsdk</artifactId>
<version>3.2.0</version>
</dependency>
<!-- 导入本地jar包 -->
<dependency>
<groupId>local-jar</groupId>
<artifactId>x-pack-transport</artifactId>
<version>6.4.2</version>
<scope>system</scope>
<systemPath>${project.basedir}/src/main/resources/lib/x-pack-transport-6.4.2.jar</systemPath>
</dependency>
<dependency>
<groupId>local-jar</groupId>
<artifactId>x-pack-core</artifactId>
<version>6.4.2</version>
<scope>system</scope>
<systemPath>${project.basedir}/src/main/resources/lib/x-pack-core-6.4.2.jar</systemPath>
</dependency>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<includeSystemScope>true</includeSystemScope>
</configuration>
</plugin>
......
</plugins>
</build>
这样打包的时候才会将本地 jar 打进入。
② 创建 TransportClient 客户端
注意:这里要使用 PreBuiltXPackTransportClient 所以需要导入上述依赖
- 将上边生成的 elastic-certificates.p12 证书放到 resources 目录下
- 创建客户端
TransportClient transportClient = null;
// 获取 target 路径
String targetPath = new ApplicationHome(getClass()).getSource().getParentFile().toString();
// 获取证书地址
String certPath = targetPath + "/elastic-certificates.p12";
// 修改配置信息
Settings esSetting = Settings.builder()
.put("cluster.name", clusterName) // 设置集群名字
.put("client.transport.sniff", true) // 设置嗅探机制,找到ES集群
.put("thread_pool.search.size", poolSize) // 设置线程池个数
.put("xpack.security.user", "elastic:密码") // 设置用户名密码
.put("xpack.security.transport.ssl.enabled", true)
.put("xpack.security.transport.ssl.truststore.path", certPath)
.put("xpack.security.transport.ssl.keystore.path", certPath)
.put("xpack.security.transport.ssl.verification_mode", "certificate")
.build();
transportClient = new PreBuiltXPackTransportClient(esSetting);
TransportAddress transportAddress = new TransportAddress(InetAddress.getByName(hostName), port);
transportClient.addTransportAddresses(transportAddress);