SpringBoot集成日志搜索引擎Elasticsearch

一.ES简介

elasticsearch是一个高度可扩展的开源全文搜索和分析引擎，可以快速的、近实时地对大数据进行存储、搜索和分析，用来支撑复杂的数据搜索需求和企业级应用，主要用于海量系统日志的数据查询与分析。

二.ES安装

1. 下载地址：https://www.elastic.co/downloads/elasticsearch。

2. 下载ES压缩包解压后，进入bin目录，双击elasticsearch.bat即可启动es

3. 启动es服务后，访问localhost:9200,浏览器出现如下信息说明启动成功.

   {
   “name” : “YNFljfo”,
   “cluster_name” : “my-application”,
   “cluster_uuid” : “iuLpWK05SJGSqp5evk7Gpg”,
   “version” : {
   “number” : “6.2.4”,
   “build_hash” : “ccec39f”,
   “build_date” : “2018-04-12T20:37:28.497551Z”,
   “build_snapshot” : false,
   “lucene_version” : “7.2.1”,
   “minimum_wire_compatibility_version” : “5.6.0”,
   “minimum_index_compatibility_version” : “5.0.0”
   },
   “tagline” : “You Know, for Search”
   }

三.head插件的安装

head插件可以很方便的操作es，head插件与es的关系就像navicat与MySQL数据库的关系，不过head插件的界面是通过访问网址浏览的。

1. 下载head插件：https://github.com/search?q=elasticsearch-head

2. 因为head插件依赖node环境，所以需要安装node.js:https://nodejs.org/en/download/

3. 运行head需要借助grunt命令，所以要安装。
   在cmd窗口进入node.js安装的根目录，然后执行npm install -g grunt -cli。

4. 安装pathomjs:
   在cmd命令窗口进入head插件解压后的根目录，然后执行npm install

5. 连接es: 设置跨域访问，在es的安装根目录的config目录下有elasticsearch.yml，在此文件中添加如下内容：

   http.cors.enabled: true
   http.cors.allow-origin: “*”

   将cluster.name: my-application的注释取消

6. 运行head：
   在cmd命令窗口进入head插件解压后的根目录，然后执行grunt server，如下图就启动成功。

7. 然后访问localhost:9100看到如下画面就安装成功

四.Springboot集成ES

1. 添加依赖及配置application.properties文件

   org.springframework.boot spring-boot-starter-data-elasticsearch

   spring.data.elasticsearch.cluster-name=my-application
   spring.data.elasticsearch.cluster-nodes=127.0.0.1:9300

2. 创建实体类及响应体

   @Data
   @Document(indexName = “person”)
   @AllArgsConstructor
   @NoArgsConstructor
   public class Person implements Serializable {
   @Id
   private String skuId;
   private String name;
   private String address;
   private Integer age;
   private String sex;
   private String city;
   }

   @Data
   @AllArgsConstructor
   @NoArgsConstructor
   public class Response {

   private Integer code;
   private String message;
   private Object data;
   
   
   public static Response successMsg(){
       Response response=new Response();
       response.setCode(200);
       response.setMessage("请求成功");
       return response;
   }
   
   public static Response failed404(){
       Response response=new Response();
       response.setCode(404);
       response.setMessage("请求失败");
       return response;
   }
   

   }

3. 创建es的Service

自定义接口实现ElasticsearchRepository，类似于与数据库交互的dao层，提供了对es的基本操作。

public interface  PersonRepository extends ElasticsearchRepository<Person,String> {


}

@NoRepositoryBean
public interface ElasticsearchRepository<T, ID extends Serializable> extends ElasticsearchCrudRepository<T, ID> {
    <S extends T> S index(S var1);

    Iterable<T> search(QueryBuilder var1);

    Page<T> search(QueryBuilder var1, Pageable var2);

    Page<T> search(SearchQuery var1);

    Page<T> searchSimilar(T var1, String[] var2, Pageable var3);

    void refresh();

    Class<T> getEntityClass();
}


public interface PersonService {

    long count();

    Person save(Person person);

    void delete(Person person);

    Iterable<Person> getAll();

    List<Person> getByName(String name);

    Page<Person> pageQuery(Integer pageNo, Integer pageSize, String kw);
}


@Service
public class PersonServiceImpl implements PersonService {

    @Autowired
    private PersonRepository personRepository;

    @Override
    public long count() {
        return personRepository.count();
    }

    @Override
    public Person save(Person person) {
        return personRepository.save(person);
    }

    @Override
    public void delete(Person person) {
        personRepository.delete(person);
    }

    @Override
    public Iterable<Person> getAll() {
        return personRepository.findAll();
    }

    @Override
    public List<Person> getByName(String name) {
        List<Person> personList=new ArrayList<>();
        MatchQueryBuilder queryBuilder=new MatchQueryBuilder("name",name);
        Iterable<Person> personIterable=personRepository.search(queryBuilder);
        personIterable.forEach(personList::add);
        return personList;
    }

    @Override
    public Page<Person> pageQuery(Integer pageNo, Integer pageSize, String kw) {
        SearchQuery searchQuery=new NativeSearchQueryBuilder().withQuery(QueryBuilders.matchPhraseQuery("name",kw)).withPageable(PageRequest.of(pageNo,pageSize)).build();
        return personRepository.search(searchQuery);
    }
}

4. Junit测试

Springboot提供了ElasticsearchTemplate,也可实现的ES的数据操作。

@RunWith(SpringRunner.class)
@SpringBootTest
public class SpringEsApplicationTests {


    @Autowired
    public PersonService personService;

    @Autowired
    ElasticsearchTemplate elasticsearchTemplate;
    @Test
    public void contextLoads() {
        Person person=new Person("123456","zlc","西湖区",18,"男","hangzhou");
        personService.save(person);
        Person person1=new Person("98712","张三","余杭区",18,"男","hangzhou");
        IndexQuery indexQuery=new IndexQueryBuilder().withObject(person1).build();
        elasticsearchTemplate.index(indexQuery);

    }

}

5. 创建controller

   @RestController
   @RequestMapping(“/es”)
   public class EsController {

   @Autowired
   PersonService personService;
   
   
   /**
    * 向es添加数据
    */
   @RequestMapping("/insert")
   public Object insert(@RequestBody Person person){
       JSONObject object=null;
       try {
         Person savePerson=  personService.save(person);
         object= (JSONObject) JSON.toJSON(savePerson);
       } catch (Exception e) {
           e.printStackTrace();
       }
   
       return object;
   }
   
   
   /**
    * 从es删除数据
    * @param person
    * @return
    */
   @RequestMapping("/delete")
   public Object delete(@RequestBody Person person){
   
       try {
           personService.delete(person);
       } catch (Exception e) {
           e.printStackTrace();
           return Response.failed404();
       }
       return Response.successMsg();
   
   }
   
   
   /**
    * 查询操作
    * @param pageNo
    * @param pageSize
    * @param key
    * @return
    */
   @RequestMapping("/query")
   public Object query(@RequestParam(value = "pageNo",required = false)Integer pageNo,@RequestParam(value = "pageSize",required = false)Integer pageSize,
   @RequestParam(value = "key",required = false)String key){
   
       Response response=new Response();
       try {
           Page<Person> personPage= personService.pageQuery(pageNo,pageSize,key);
           response.setData(personPage);
       } catch (Exception e) {
           e.printStackTrace();
           return Response.failed404();
       }
       return response;
   
   }
   

   }

通过Postman测试查询结果如下：

五.错误解决

1. SpringBoot加载elasticsearch节点失败

   failed to load elasticsearch nodes : org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available:

初步判定，应该是spring-boot-starter-data-elasticsearch与Elasticsearch 7.x版本不兼容造成的。
通过测试发现，将Elasticsearch降级到6.x版本，即可解决。
2.修改elasticsearch.yml后双击elasticsearch.bat闪退
原因是elasticsearch.yml默认是utf-8编码，所以添加的属性保存时也得是utf-8编码的。

六.源码下载地址

https://github.com/liankaizhao/springboot-es

---

参考文献：

1.https://cloud.tencent.com/developer/article/1461537