超详细之Spring Boot结合Jest实现对ElasticSearch的全文检索(从mysql导入数据)

1 Elasticsearch与Solr区别

在Java的全文检索里面有solr和elasticsearch两大高级玩意
首先我们来看看他们的区别：

1）Solr建立索引时候，搜索效率下降，实时搜索效率不高，es实时搜索效率高
2）Solr利用Zookeeper进行分布式管理，而Elasticsearch自身带有分布式协调管理功能。
3）Solr支持更多格式的数据，比如JSON、XML、CSV，而Elasticsearch仅支持json文件格式。
4）Solr官方提供的功能更多，而Elasticsearch本身更注重于核心功能，高级功能多有第三方插件提供
5）Solr在传统的搜索应用中表现好于Elasticsearch，但在处理实时搜索应用时效率明显低于Elasticsearch。
6）Solr是传统搜索应用的有力解决方案，但Elasticsearch更适用于新兴的实时搜索应用。
因此我们选择Elasticsearch

2 elasticsearch的安装

2.1 创建用户

因为elasticsearch出于系统安全考虑,不能使用root启动.所以要建立其他的普通用户.
创建用户dev，密码：123456

[root@dev-2 ~]# useradd dev
[root@dev-2 ~]# passwd dev
Changing password for user dev.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
[root@dev-2 ~]# id dev

2.2 下载elasticsearch且配置环境变量

下载：

[root@dev-2 elasticsearch]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-
6.3.0.tar.gz
--2018-09-05 11:32:45-- https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.0.tar.gz
Resolving artifacts.elastic.co (artifacts.elastic.co)... 107.21.253.15, 184.72.242.47, 107.21.237.95,
...
Connecting to artifacts.elastic.co (artifacts.elastic.co)|107.21.253.15|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 91423553 (87M) [application/x-gzip]
Saving to: ‘elasticsearch-6.3.0.tar.gz’

100%
[=======================================================================================================
==============================================>] 91,423,553 947KB/s in 1m 55s
2018-09-05 11:34:31 (776 KB/s) - ‘elasticsearch-6.3.0.tar.gz’ saved [91423553/91423553]
[root@dev-2 elasticsearch]#

解压

[root@dev-2 elasticsearch]# tar -xvf elasticsearch-6.3.0.tar.gz

配置环境变量

[root@dev-2 elasticsearch-6.3.0]# vim /etc/profile
# import elasticsearch
export SEARCH_HOME='/opt/soft/elasticsearch/elasticsearch-6.3.0'
export PATH=$PATH:$SEARCH_HOME/bin
[root@dev-2 elasticsearch-6.3.0]# source /etc/profile

设置所有者为dev

[root@dev-2 elasticsearch]# ll
total 89284
drwxr-xr-x. 8 root root 143 Sep 05 11:43 elasticsearch-6.3.0
[root@dev-2 elasticsearch]# chown -R dev:dev elasticsearch-6.3.0
[root@dev-2 elasticsearch]# ll
total 89284
drwxr-xr-x. 8 dev dev 143 Sep 05 11:43 elasticsearch-6.3.0
[root@dev-2 elasticsearch]

2.3elasticsearch的配置

修改JVM参数
-Xms和-Xmx的值要一致.

[root@dev-2 elasticsearch-6.3.0]# cd config/
[root@dev-2 config]# ls
elasticsearch.yml jvm.options log4j2.properties role_mapping.yml roles.yml users users_roles
[root@dev-2 config]# vim jvm.options
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space
-Xms1g
-Xmx1g
[root@dev-2 config]#

修改启动参数

[root@dev-2 config]# vim elasticsearch.yml
# Set the bind address to a specific IP (IPv4 or IPv6):
#使用 云服务器时用以下地址；若为本地服务器时则使用本地服务器地址；
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# 开启跨域访问支持,默认为false
#
http.cors.enabled: true
#
# 跨域访问允许的域名地址,(允许所有域名)以上使用正则
#
http.cors.allow-origin: /.*/
[root@dev-2 config]#

3 启动Elasticsearch

切换到dev启动

[root@dev-2 elasticsearch-6.3.0]# su dev
[dev@dev-2 elasticsearch-6.3.0]$ elasticsearch
[2018-09-05T11:50:22,771][INFO ][o.e.n.Node ] [] initializing ...

但启动报错，如下

[2018-09-05T11:50:40,429][INFO ][o.e.b.BootstrapChecks ] [eme3ye6] bound or publishing to a nonloopback address, enforcing bootstrap checks
ERROR: [4] bootstrap checks failed
[1]: initial heap size [268435456] not equal to maximum heap size [536870912]; this can cause resize
pauses and prevents mlockall from locking the entire heap
[2]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[3]: max number of threads [3852] for user [dev] is too low, increase to at least [4096]
[4]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2018-09-05T11:50:40,506][INFO ][o.e.n.Node ] [eme3ye6] stopping ...
[2018-09-05T11:50:40,606][INFO ][o.e.n.Node ] [eme3ye6] stopped
[2018-09-05T11:50:40,606][INFO ][o.e.n.Node ] [eme3ye6] closing ...
[2018-09-05T11:50:40,619][INFO ][o.e.n.Node ] [eme3ye6] closed

请切换到root,修改 /etc/security/limits.conf文件

[root@dev-2 elasticsearch-6.3.0]# vim /etc/security/limits.conf
#@student - maxlogins 4
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096
# End of file
[root@dev-2 elasticsearch-6.3.0]#

修改/etc/sysctl.conf文件

[root@dev-2 etc]# vim /etc/sysctl.conf
vm.max_map_count=655360
[root@dev-2 etc]# sysctl -p
vm.max_map_count = 655360

再切换到dev用户启动elasticsearch

[dev@dev-2 elasticsearch-6.3.0]$ nohup elasticsearch &
[dev@dev-2 elasticsearch-6.3.0]$ cat nohup.out
....
[2018-09-05T12:15:27,438][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [eme3ye6]
publish_address {
   47.98.143.72:9200}, bound_addresses {
   47.98.143.72:9200}
[2018-09-05T12:15:27,438][INFO ][o.e.n.Node ] [eme3ye6] started
[2018-09-05T12:15:27,949][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [eme3ye6] Failed to clear cache
for realms [[]]
[2018-07-04T10:17:37,982][INFO ][o.e.l.LicenseService ] [eme3ye6] license [36a89374-568c-4f0d-83ce-
37475ca3015c] mode [basic] - valid
[2018-07-04T10:17:37,996][INFO ][o.e.g.GatewayService ] [eme3ye6] recovered [0] indices into
cluster_state
....
[dev@dev-2 elasticsearch-6.3.0]$ jps
21874 Jps
4969 Elasticsearch
[dev@dev-2 elasticsearch-6.3.0]# netstat -lnp|grep 9200
tcp        0      0 0.0.0.0:9200            0.0.0.0:*               LISTEN      4969/java   
[dev@dev-2 elasticsearch-6.3.0]# curl 47.98.143.72:9200
{
   
  "name" : "jDY6CwJ",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "LhuAiJqpRVqKOxIUZ0Xa3Q",
  "version" : {
   
    "number" : "6.3.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "424e937",
    "build_date" : "2018-06-11T23:38:03.357887Z",
    "build_snapshot" : false,
    "lucene_version" : "7.3.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

4 中文分词

4.1 安装IK分词器

**Github下载和Elasticsearch相同版本的IK分词器: **
IK分词器下载地址
在 plugins/ 文件夹下建立ik 文件夹,并把压缩包内容解压到 ik 文件夹.

[root@dev-2 plugins]# pwd
/opt/soft/elasticsearch/elasticsearch-6.3.0/plugins
[dev@dev-2 plugins]$ mkdir ik
[dev@dev-2 plugins]$ cd ik
[dev@dev-2 ik]$ unzip elasticsearch-analysis-ik-6.3.0.zip -d .
[dev@dev-2 ik]$ ls
commons-codec-1.9.jar config httpclient-4.5.2.jar plugindescriptor.properties
commons-logging-1.2.jar elasticsearch-analysis-ik-6.3.0.jar httpcore-4.4.4.jar

重新启动ElasticSear

[dev@dev-2 elasticsearch-6.3.0]$ nohup bin/elasticsearch &
[1] 17149
[dev@dev-2 elasticsearch-6.3.0]$ jps
17236 Jps
17149 Elasticsearch