spring/springboot集成spring-data-elasticsearch 3.1.6。从安装到集成附spring-boot源码

    ElasticSearch是一个基于Lucene的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎，基于RESTful web接口。Elasticsearch是用Java开发的，并作为Apache许可条款下的开放源码发布，是当前流行的企业级搜索引擎。设计用于云计算中，能够达到实时搜索，稳定，可靠，快速，安装使用方便。

一. ElasticSearch6.2.2下载安装步骤

说明：ElasticSearch的运行不能用root执行，自己用useradd命令新建一个用户如下所示：
sueradd test
chown -R test  /user/local/elasticsearch6.2.2   此处给新增的用户授权
下载地址：https://www.elastic.co/downloads/elasticsearch 下载最新的安装包并解压、然后执行、命令如下：
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.2.tar.gz
 
tar -zxvf elasticsearch-6.2.2
cd elasticsearch-6.2.2
./bin/elasticsearch
执行之前就得修改配置文件：

vim config/elasticsearch.yml
vim config/elasticsearch.yml 
修改方法参考如下：

    cluster.name: elk_test.cluster
    node.name: node-01
    node.master: true
    node.data: true
    network.host: 0.0.0.0
    http.port: 9200
    discovery.zen.ping.unicast.hosts: ["192.168.0.153","192.168.0.154","192.168.0.155"]
    #discovery.zen.ping.multicast.enabled: true
    discovery.zen.minimum_master_nodes: 2
    http.cors.enabled: true
    http.cors.allow-origin: "*"
    

（1）cluster.name
如果要配置集群需要两个节点上的elasticsearch配置的cluster.name相同，都启动可以自动组成集群，这里如果不改cluster.name则默认是cluster.name=my-application，
（2）nodename随意取但是集群内的各节点不能相同
（3）修改后的每行前面不能有空格，修改后的“：”后面必须有一个空格
解释说明：
bootstrap.memory_lock: false
bootstrap.system_call_filter: false 一看就知道是关于内用访问的方面的配置
cluster.name 集群名字，同一个集群中使用相同名字，单机就随意
node.name: node-01 节点名字
node.master: 是否为集群的master机器
node.data: true 是否作为数据节点
network.host: 192.168.0.153 这个不用自然是配置ip地址的
http.port: 9200 端口号，不配置的话默认9200
discovery.zen.ping.unicast.hosts: [“192.168.0.153”,”192.168.0.154”,”192.168.0.155”] 这个就是配置集群的时候要用的到了，[]中填上集群中其他集群的ip的地址，如果是master的话请把所有salve的机器地址填上
discovery.zen.minimum_master_nodes: 2 关于这个值配置多少合适的话大家去搜一下，自己权衡一下集群，这里我用了3台机器模拟集群，所以填上2。
http.cors.enabled: true 这个参数的设置和下面一个配置就关于ip的访问策略了，如果你发现其他ip地址访问不了就有可以这参数没有配置
http.cors.allow-origin: “*”
如果你碰到错误不要慌下面有解决方法：
[1]"max file descriptor
[2]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
解决方法，请用root权限修改，修改完记得重启elasticsearch和使用配置参数在环境里面生效（重新登录用户）：(如果还有错误请把数字再设置大一点)
vi /etc/security/limits.conf 
 
*  soft  nofile  65536
 
*  hard  nofile  131072
 
*  soft  nproc  2048
 
*  hard  nproc 4096
2、max number of threads
max number of threads [1024] for user [user] is too low, increase to at least [2048]
解决方法： (* 表示对所有用户有效)
vi /etc/security/limits.d/90-nproc.conf 
#修改内容如下
*  soft  nproc  1024
3、在elasticsearch.yml中配置bootstrap.system_call_filter为false，注意要在Memory下面:
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
4、错误：max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
临时设置：sudo sysctl -w vm.max_map_count=262144
永久修改：
修改/etc/sysctl.conf 文件，添加 “vm.max_map_count”设置
并执行：sysctl -p

更多错误参考：http://blog.csdn.net/satiling/article/details/59697916

7）测试集群
[atguigu@hadoop102 elasticsearch-5.2.2]$ curl http://xxxxx:9200
{
  "name" : "node-102",
  "cluster_name" : "my-application",
  "cluster_uuid" : "v-nwhc7ITsmVHECpNQYzHw",
  "version" : {
    "number" : "6.2.2",
    "build_hash" : "57e20f3",
    "build_date" : "2017-09-23T13:16:45.703Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.1"
  },
  "tagline" : "You Know, for Search"
}

 二.head插件的安装

es5.0以下安装： 安装head 插件  sudo bin/plugin install mobz/elasticsearch-head
https://github.com/mobz/elasticsearch-head
https://www.cnblogs.com/xuwenjin/p/8792919.html head插件使用
访问head插件地址：http://xxx.xxx.xxx.xx:9100/

head连不上es，集群健康值: 未连接
（1）elasticsearch-5x下的 config/elasticsearch.yml
http.cors.enabled: true
http.cors.allow-origin: "*"
（2）elasticsearch-head下Gruntfile.js
connect: {
        server: {
            options: {
                hostname: '0.0.0.0',
                port: 9100,
                base: '.',
                keepalive: true
            }
        }
    }   
（3）浏览器问题：chrom不能连，换ie试试

三.安装kibana插件

将安装包上次到服务器，然后解压安装包，例如解压到：
wget  https://artifacts.elastic.co/downloads/kibana/kibana-6.5.0-linux-x86_64.tar.gz  
然后到kibana安装目录的config下，编辑kibana.yml配置文件，添加如下配置：

#配置本机ip
server.host: "0.0.0.0"
#配置es集群url
elasticsearch.url: "http://0.0.0.0:9200"

启动：bin/kibana
访问地址：http://xxx.xxx.xxx.xx.5601

插件使用：https://www.cnblogs.com/cjsblog/p/9476813.html

四.安装分词器

分词器hanlp：https://blog.csdn.net/pengcong90/article/details/76843760
当然也可以选择项目符合的分词器。列如ik分词器

 五.spring集成spring-data-elasticsearch

spring-data-elasticsearch github地址：https://github.com/spring-projects/spring-data-elasticsearch
根据elasticsearch选择版本号：3.2.x还没有正式的版本，建议使用6.2.x版本的elasticsearch

版本选择：笔者这里是spring版本是5.1.4.  spring 4.x启动会报错

<dependency>
   <groupId>org.elasticsearch</groupId>
   <artifactId>elasticsearch</artifactId>
   <version>6.2.2</version>
</dependency>

<dependency>
   <groupId>org.springframework.data</groupId>
   <artifactId>spring-data-elasticsearch</artifactId>
   <version>3.1.6.RELEASE</version>
</dependency>

项目结构：

实体：

package com.frame.common.core.esvo;

import org.springframework.data.annotation.Id;
import org.springframework.data.elasticsearch.annotations.Document;

import java.io.Serializable;

/**
 * @Title: TestEmployee
 * @Description:
 * @Auther:wangli
 * @Version: 1.0
 * @create 2019-04-09 15:30
 */
//索引 和类型设置
@Document(indexName = "megacorp", type = "employee")
public class TestEmployee implements Serializable {

    //这里的注解一定要加上，否则启动会出错
    @Id
    private String id;

    private String first_name;

    private String last_name;

    private Integer age;

    private String about;

    private String[] interests;


    public String getFirst_name() {
        return first_name;
    }

    public void setFirst_name(String first_name) {
        this.first_name = first_name;
    }

    public String getLast_name() {
        return last_name;
    }

    public void setLast_name(String last_name) {
        this.last_name = last_name;
    }

    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

    public Integer getAge() {
        return age;
    }

    public void setAge(Integer age) {
        this.age = age;
    }

    public String getAbout() {
        return about;
    }

    public void setAbout(String about) {
        this.about = about;
    }

    public String[] getInterests() {
        return interests;
    }

    public void setInterests(String[] interests) {
        this.interests = interests;
    }
}

仓库：

package com.frame.common.core.esrepository;

import com.frame.common.core.esvo.TestEmployee;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;
import org.springframework.stereotype.Repository;

/**
 * @Title: EmployeeRepository
 * @Description:
 * @Auther:wangli
 * @Version: 1.0
 * @create 2019-04-09 15:39
 */
@Repository
public interface EmployeeRepository extends ElasticsearchRepository<TestEmployee, Long> {

}

接口：

package com.frame.common.core.esdao;

import com.frame.common.core.esvo.TestEmployee;

import java.util.List;

/**
 * @Title: EmployeeService
 * @Description:
 * @Auther:wangli
 * @Version: 1.0
 * @create 2019-04-09 16:08
 */
public interface EmployeeService {

    List<TestEmployee> getEmployeeTest(TestEmployee testEmployee);

    void saveEmployeeTest(TestEmployee testEmployee);

    TestEmployee getEmployeeById(TestEmployee testEmployee);
}

实现：

package com.frame.common.core.esdao.impl;

import com.frame.common.core.esdao.EmployeeService;
import com.frame.common.core.esrepository.EmployeeRepository;
import com.frame.common.core.esvo.TestEmployee;
import com.google.common.collect.Lists;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import org.springframework.stereotype.Service;

import java.util.List;
import java.util.Optional;

/**
 * @Title: EmployeeRepositoryImpl
 * @Description:
 * @Auther:wangli
 * @Version: 1.0
 * @create 2019-04-09 15:53
 */
@Service
public class EmployeeRepositoryImpl implements EmployeeService {
    @Autowired
    private EmployeeRepository employeeRepository;

    @Autowired
    private ElasticsearchTemplate elasticsearchTemplate;

    @Override
    public List<TestEmployee> getEmployeeTest(TestEmployee testEmployee) {
        String id = testEmployee.getId();
        BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
        queryBuilder.must(QueryBuilders.termQuery("id", id));


        Iterable<TestEmployee> search = employeeRepository.search(queryBuilder);

        List<TestEmployee> employeeList = Lists.newArrayList(search);

        return employeeList;


    }

    @Override
    public void saveEmployeeTest(TestEmployee testEmployee) {
        employeeRepository.save(testEmployee);
    }

    @Override
    public TestEmployee getEmployeeById(TestEmployee testEmployee) {
        String id = testEmployee.getId();
        Optional<TestEmployee> employeeRepositoryById = employeeRepository.findById(Long.valueOf(id));
        TestEmployee employee = employeeRepositoryById.get();

        return employee;
    }
}

xml文件配置：

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:elasticsearch="http://www.springframework.org/schema/data/elasticsearch"
       xsi:schemaLocation="
      http://www.springframework.org/schema/beans
      http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
      http://www.springframework.org/schema/data/elasticsearch
      http://www.springframework.org/schema/data/elasticsearch/spring-elasticsearch-1.0.xsd">

    <!-- 搜索仓库 扫描  包名扫描到最外层 -->
    <elasticsearch:repositories base-package="com.frame.xx" />


    <!-- 配置Client client-transport-sniff 非本地连接的时候用false-->
    <elasticsearch:transport-client id="client" cluster-nodes="xxx.xxx.xx.xx:9300" cluster-name="${cluster.name}" client-transport-sniff="false"/>

    <!-- 配置搜索模板  -->
    <bean id="elasticsearchTemplate"
          class="org.springframework.data.elasticsearch.core.ElasticsearchTemplate">
        <constructor-arg name="client" ref="client" />
    </bean>
</beans>

 

 

5.报错解决

1. [main] ERROR org.springframework.data.elasticsearch.repository.support.AbstractElasticsearchRepository - failed to load elasticsearch nodes : org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: [{#transport#-1}{lvvAmVkbSimt99eaOeEfEg}{xx.xx.xx.xx}{xx.xx.xx.xx:9300}]

确认tcp端口号是否是：9300 不是9200

设置client-transport-sniff flase

 

2.ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]

PUT /twitter/_settings 

{ 

“index.blocks.read_only_allow_delete”: null 

}

curl -XPUT -H “Content-Type: application/json” 

http://127.0.0.1:9200/_all/_settings -d ‘{“index.blocks.read_only_allow_delete”: null}’

其中 _all 这个可以更改为自己在创建 Eleastisearch 索引的时候的name，用来修改单个索引只读状态，当然用 _all 也可以, _all 是修改了所有的索引只读状态

 

 六.spring-boot集成spring-data-elasticsearch

pom文件：

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.1.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>cn.hadron</groupId>
    <artifactId>demo</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>demo</name>
    <description>Demo project for Spring Boot</description>

    <properties>
        <java.version>1.8</java.version>
    </properties>

    <repositories>
        <repository>
            <id>nexus-aliyun</id>
            <name>Nexus aliyun</name>
            <url>http://maven.aliyun.com/nexus/content/groups/public</url>
        </repository>
    </repositories>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

配置文件:

# ELASTICSEARCH (ElasticsearchProperties)
# Elasticsearch cluster name.
spring.data.elasticsearch.cluster-name=my-application
# Comma-separated list of cluster node addresses.
spring.data.elasticsearch.cluster-nodes=xx.xx.xx.xx:9300
# Whether to enable Elasticsearch repositories.
spring.data.elasticsearch.repositories.enabled=true

此处笔者就不多说了直接放上github源码：