ES大数据搜索引擎安装配置及故障排查

Elasticsearch安装配置及故障排查
搜索引擎使用就是对数据的检索，数据一般分为两种类型，结构化数据和非结构化数据，
结构化数据：二维表结构数据，严格遵照数据的格式和长度规范，主要是通过关系型数据库进行存储和管理。
非结构化数据：，不定长度和不固定格式，可以有办公文档、XML、HTML、Word 文档，邮件，各类报表、图片和咅频、视频信息等。
对于结构化数据，我们一般使用oracle、mysql、postgres等来存储和检索。而非结构化数据在全文搜索中主要使用全文搜索引擎主要是 Solr 和 Elasticsearch。
Elasticsearch 是使用java编写的一种开源搜索引擎，对于 PB 级全文搜索性能比结构化数据库性能好很多，因为当索引上的数据量太大的时候，ES 通过水平拆分的方式将一个索引上的数据拆分出来分配到不同的数据块上，拆分出来的数据库块称之为一个分片。而MySQL、postgres通过物理分库分表等来分摊支持大数据检索时IO压力等，而且需要借助第三方组件而 ES 内部自身实现了此功能，而且Elasticsearch 是使用 Java 构建，通过JVM来存储数据，在性能上确实有一定的优势，内存检索数据比磁盘检索数据快不是一般的级别。
但是在选择 Elasticsearch 版本时，还需注意 JDK版本，不然会发现后台启动成功，状态也是running，但是使用curl 方式来验证会提示拒绝连接，如下：

[root@localhost elasticsearch]# curl http://10.100.81.167:9200/
curl: (7) Failed connect to 10.100.81.167:9200; 拒绝连接
因为每个大版本所依赖的 JDK 版本也不同，，既然我们了解ES的优势，在决定使用 Elasticsearch 的时候首先要考虑的是版本问题，Elasticsearch常用的稳定的主版本：2.x，5.x，6.x，7.x，目前 7.2 版本已经可以支持 JDK11，其他的版本使用JDK1.8。

安装配置：
可以到该地址下载对应的版本，例如使用比较多的是6.6版本，可以通过如下地址下载，先安装JDK ，在安装ES https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.6.0.rpm

然后 使用 rpm -Uvh elasticsearch-6.6.0.rpm 安装配置
[root@localhost pg10]# rpm -Uvh elasticsearch-6.6.0.rpm
警告：elasticsearch-6.6.0.rpm: 头V4 RSA/SHA512 Signature, 密钥 ID d88e42b4: NOKEY
准备中... ################################# [100%]
Creating elasticsearch group... OK
Creating elasticsearch user... OK
正在升级/安装...
1:elasticsearch-0:6.6.0-1 ################################# [100%]

确保安装成功功修改对应的配置文件，确保能被访问
需修改elasticsearch.yml
把 network.host 和http.port 开放，例如9200是端口，如果要能被访问，需要再防火墙开放对应端口，

启动ES
通过修改上图，配置，然后启动elasticsearch
[root@localhost pg10]# sudo systemctl start elasticsearch.service

查看是否启动成功：
[root@localhost pg10]# sudo systemctl status elasticsearch.service
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
Active: active (running) since 一 2019-07-22 17:55:59 CST; 17h ago
Docs: http://www.elastic.co
Main PID: 1514 (java)
CGroup: /system.slice/elasticsearch.service
└─1514 /usr/java/jdk1.8.0_112//bin/java -Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccup...

7月 22 17:55:59 localhost.localdomain systemd[1]: Starting Elasticsearch...
7月 22 17:55:59 localhost.localdomain systemd[1]: Started Elasticsearch.
7月 23 11:06:23 localhost.localdomain systemd[1]: Started Elasticsearch.

验证可用性
通过如下方式验证elasticsearch是否真的可用，
通过curl http://10.100.81.167:9200 ，得到一个 JSON 对象，其中包含当前节点、集群、版本等信息。
curl http://10.100.81.167:9200
{
"name" : "WkKXH2o",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "Hm4mCu5IQ6qG9f3hXMtT_Q",
"version" : {
"number" : "6.6.0",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "a9861f4",
"build_date" : "2019-01-24T11:27:09.439740Z",
"build_snapshot" : false,
"lucene_version" : "7.6.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"

[root@localhost ~]# netstat -ntap | grep 9200
tcp6 0 0 :::9200 :::* LISTEN 24230/java
tcp6 0 0 10.100.81.167:9200 10.100.81.67:8942 ESTABLISHED 24230/java
tcp6 0 0 10.100.81.167:9200 10.100.81.67:8578 ESTABLISHED 24230/java

问题排查1：
如果使用错了版本，导致能正常启动，但是在使用curl http://10.100.81.167:9300/
报错：curl: (7) Failed connect to 10.100.81.167:9300; 拒绝连接

问题原因分析：
因为配置的是9300端口，JDK1.8版本，elasticsearch7.2版本
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
Active: active (running) since 二 2019-07-23 18:11:21 CST; 2s ago
Docs: http://www.elastic.co
Main PID: 3445 (java)
CGroup: /system.slice/elasticsearch.service
└─3445 /usr/java/jdk1.8.0_112/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupa...

7月 23 18:11:21 localhost.localdomain systemd[1]: Started Elasticsearch.
7月 23 18:11:21 localhost.localdomain systemd[1]: Starting Elasticsearch...
7月 23 18:11:21 localhost.localdomain elasticsearch[3445]: future versions of Elasticsearch will require Ja...ent
Hint: Some lines were ellipsized, use -l to show in full.

这时查看端口，发现elasticsearch 没有真正启动
[root@localhost elasticsearch]# netstat -ntap | grep 9300

[root@localhost elasticsearch]# curl http://10.100.81.167:9300/
curl: (7) Failed connect to 10.100.81.167:9300; 拒绝连接

这时通过操作系统日志分析如下，提升7.2版本需要jdk11
Jul 23 18:08:34 localhost elasticsearch: future versions of Elasticsearch will require Java 11; your Java version
from [/usr/java/jdk1.8.0_112/jre] does not meet this requirement
Jul 23 18:08:38 localhost systemd: Started Elasticsearch.
Jul 23 18:08:41 localhost systemd: Stopping Elasticsearch...
Jul 23 18:08:41 localhost systemd: Started Elasticsearch.
Jul 23 18:08:41 localhost systemd: Starting Elasticsearch...
Jul 23 18:08:41 localhost elasticsearch: future versions of Elasticsearch will require Java 11; your Java version
from [/usr/java/jdk1.8.0_112/jre] does not meet this requirement
Jul 23 18:08:50 localhost systemd: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Jul 23 18:08:50 localhost systemd: Unit elasticsearch.service entered failed state.

问题排查2：
因担心elasticsearch在使用久后，会因日志文件和数据文件增大导致/var空间不足，所以重新设置存储路径，但是因为设置路径后没有对于赋予权限导致启动失败，如下elasticsearch.yml 重新设置了日志路径和数据存储路径：

错误信息如下：
[root@localhost elasticsearch]# sudo systemctl start elasticsearch.service
[root@localhost elasticsearch]# sudo systemctl status elasticsearch.service
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since 四 2019-08-01 15:06:28 CST; 4s ago
Docs: http://www.elastic.co
Process: 13320 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 13320 (code=exited, status=1/FAILURE)

8月 01 15:06:28 localhost.localdomain elasticsearch[13320]: 2019-08-01 15:06:28,025 main ERROR Null object returned for Rol...ders.
8月 01 15:06:28 localhost.localdomain elasticsearch[13320]: 2019-08-01 15:06:28,025 main ERROR Unable to locate appender "r...root"
8月 01 15:06:28 localhost.localdomain elasticsearch[13320]: 2019-08-01 15:06:28,026 main ERROR Unable to locate appender "i...ndex"
8月 01 15:06:28 localhost.localdomain elasticsearch[13320]: 2019-08-01 15:06:28,026 main ERROR Unable to locate appender "a...rail"
8月 01 15:06:28 localhost.localdomain elasticsearch[13320]: 2019-08-01 15:06:28,026 main ERROR Unable to locate appender "i...wlog"
8月 01 15:06:28 localhost.localdomain elasticsearch[13320]: 2019-08-01 15:06:28,026 main ERROR Unable to locate appender "d...rail"
8月 01 15:06:28 localhost.localdomain elasticsearch[13320]: 2019-08-01 15:06:28,026 main ERROR Unable to locate appender "d...tion"
8月 01 15:06:28 localhost.localdomain systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
8月 01 15:06:28 localhost.localdomain systemd[1]: Unit elasticsearch.service entered failed state.
8月 01 15:06:28 localhost.localdomain systemd[1]: elasticsearch.service failed.
Hint: Some lines were ellipsized, use -l to show in full.
[root@localhost elasticsearch]#

这时通过查看系统日志分析，发现时重新创建了日志路径和数据存储路径后没有设置权限导致，
Aug 1 15:03:02 localhost dbus[753]: [system] Successfully activated service 'org.freedesktop.problems'
Aug 1 15:03:02 localhost dbus-daemon: dbus[753]: [system] Successfully activated service 'org.freedesktop.problems'
Aug 1 15:06:26 localhost systemd: Started Elasticsearch.
Aug 1 15:06:26 localhost systemd: Starting Elasticsearch...
Aug 1 15:06:28 localhost elasticsearch: 2019-08-01 15:06:27,998 main ERROR Unable to create file /home/es_log/elasticsearch.log jav
a.io.IOException: 权限不够
Aug 1 15:06:28 localhost elasticsearch: at java.io.UnixFileSystem.createFileExclusively(Native Method)
Aug 1 15:06:28 localhost elasticsearch: at java.io.File.createNewFile(File.java:1012)
Aug 1 15:06:28 localhost elasticsearch: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager$RollingFileManagerFact
ory.createManager(RollingFileManager.java:628)

解决方法：

root@localhost home]# chmod 777 /home/es_log
root@localhost home]# chmod 777 /home/es_data

[root@localhost elasticsearch]# sudo systemctl start elasticsearch.service
[root@localhost elasticsearch]# sudo systemctl status elasticsearch.service
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
Active: active (running) since 四 2019-08-01 15:11:51 CST; 1s ago
Docs: http://www.elastic.co
Main PID: 13695 (java)

Elasticsearch 卸载
[root@localhost pg10]# yum remove elasticsearch-7.2.0
已加载插件：fastestmirror, langpacks
正在解决依赖关系
--> 正在检查事务
---> 软件包 elasticsearch.x86_64.0.7.2.0-1 将被 删除
--> 解决依赖关系完成
base/7/x86_64 | 3.6 kB 00:00:00
epel/x86_64 | 5.4 kB 00:00:00
epel/x86_64/updateinfo | 993 kB 00:00:00
epel/x86_64/primary_db | 6.8 MB 00:00:01
extras/7/x86_64 | 3.4 kB 00:00:00
pgdg10/7/x86_64 | 3.6 kB 00:00:00
pgdg11/7/x86_64 | 3.6 kB 00:00:00
pgdg94/7/x86_64 | 3.6 kB 00:00:00
pgdg95/7/x86_64 | 3.6 kB 00:00:00
pgdg96/7/x86_64 | 3.6 kB 00:00:00
updates/7/x86_64 | 3.4 kB 00:00:00

依赖关系解决

==================================================================================================================
Package 架构 版本 源 大小

正在删除:
elasticsearch x86_64 7.2.0-1 installed 511 M

事务概要

移除 1 软件包

安装大小：511 M
是否继续？[y/N]：y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
警告：RPM 数据库已被非 yum 程序修改。
Stopping elasticsearch service... OK
正在删除 : elasticsearch-7.2.0-1.x86_64 1/1
警告：/etc/sysconfig/elasticsearch 已另存为 /etc/sysconfig/elasticsearch.rpmsave
警告：/etc/elasticsearch/jvm.options 已另存为 /etc/elasticsearch/jvm.options.rpmsave
警告：/etc/elasticsearch/elasticsearch.yml 已另存为 /etc/elasticsearch/elasticsearch.yml.rpmsave
Deleting log directory... OK
验证中 : elasticsearch-7.2.0-1.x86_64 1/1

删除:
elasticsearch.x86_64 0:7.2.0-1

完毕！
[root@localhost pg10]#