引用地址:https://yejingtao.blog.csdn.net/article/details/78414874
ElasticSearch作为搜索引擎,我们需要解决2大问题:
1, 如何将被搜索的数据在ES上创建反向索引
2, Java代码如何与ES交互
其中第一个大问题又分为两个小问题
1.1,如何初始化已有的数据
1.2,如何同步增量数据
第二个大问题也有两种集成方式
2.1 Spring Data 9300端口集成
2.2 Restful API 9200端口集成
本篇先解决第二大问题。
第一种方式,利用RestAPI方式,也叫Jest方式:
示例代码:https://github.com/yejingtao/forblog/tree/master/demo-jest-elasticsearch
Pom.xml:
-
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
-
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
-
<modelVersion>4.0.0</modelVersion>
-
<groupId>yejingtao.demo.springcloud</groupId>
-
<artifactId>demo-jest-elasticsearch</artifactId>
-
<version>0.0.1-SNAPSHOT</version>
-
<packaging>jar</packaging>
-
<name>demo-jest-elasticsearch</name>
-
<url>http://maven.apache.org</url>
-
<properties>
-
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-
</properties>
-
<parent>
-
<groupId>org.springframework.boot</groupId>
-
<artifactId>spring-boot-starter-parent</artifactId>
-
<version>1.5.6.RELEASE</version>
-
</parent>
-
<dependencies>
-
<dependency>
-
<groupId>org.springframework.boot</groupId>
-
<artifactId>spring-boot-starter-web</artifactId>
-
</dependency>
-
<dependency>
-
<groupId>org.springframework.boot</groupId>
-
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
-
</dependency>
-
<dependency>
-
<groupId>io.searchbox</groupId>
-
<artifactId>jest</artifactId>
-
</dependency>
-
<dependency>
-
<groupId>net.java.dev.jna</groupId>
-
<artifactId>jna</artifactId>
-
</dependency>
-
</dependencies>
-
</project>
Application.yml:
server:
port: 7081
spring:
elasticsearch:
jest:
uris:
- http://192.168.226.133:9200
read-timeout: 5000
注意这里是9200端口
主程序:最简单的Spring boot启动程序:
-
@SpringBootApplication
-
public class ESApplication {
-
public static void main(String[] args) {
-
SpringApplication.run(ESApplication.class);
-
}
-
}
定义好ES中的实体类和对ES操作的接口:
-
public class Entity implements Serializable{
-
private static final long serialVersionUID = -763638353551774166L;
-
public static final String INDEX_NAME = "index_entity";
-
public static final String TYPE = "tstype";
-
private Long id;
-
private String name;
-
public Entity() {
-
super();
-
}
-
public Entity(Long id, String name) {
-
this.id = id;
-
this.name = name;
-
}
-
public Long getId() {
-
return id;
-
}
-
public void setId(Long id) {
-
this.id = id;
-
}
-
public String getName() {
-
return name;
-
}
-
public void setName(String name) {
-
this.name = name;
-
}
-
}
-
public interface CityESService {
-
void saveEntity(Entity entity);
-
void saveEntity(List<Entity> entityList);
-
List<Entity> searchEntity(String searchContent);
-
}
接口实现:
-
@Service
-
public class CityESServiceImpl implements CityESService{
-
private static final Logger LOGGER = LoggerFactory.getLogger(CityESServiceImpl.class);
-
@Autowired
-
private JestClient jestClient;
-
@Override
-
public void saveEntity(Entity entity) {
-
Index index = new Index.Builder(entity).index(Entity.INDEX_NAME).type(Entity.TYPE).build();
-
try {
-
jestClient.execute(index);
-
LOGGER.info("ES 插入完成");
-
} catch (IOException e) {
-
e.printStackTrace();
-
LOGGER.error(e.getMessage());
-
}
-
}
-
/**
-
* 批量保存内容到ES
-
*/
-
@Override
-
public void saveEntity(List<Entity> entityList) {
-
Bulk.Builder bulk = new Bulk.Builder();
-
for(Entity entity : entityList) {
-
Index index = new Index.Builder(entity).index(Entity.INDEX_NAME).type(Entity.TYPE).build();
-
bulk.addAction(index);
-
}
-
try {
-
jestClient.execute(bulk.build());
-
LOGGER.info("ES 插入完成");
-
} catch (IOException e) {
-
e.printStackTrace();
-
LOGGER.error(e.getMessage());
-
}
-
}
-
/**
-
* 在ES中搜索内容
-
*/
-
@Override
-
public List<Entity> searchEntity(String searchContent){
-
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
-
//searchSourceBuilder.query(QueryBuilders.queryStringQuery(searchContent));
-
//searchSourceBuilder.field("name");
-
searchSourceBuilder.query(QueryBuilders.matchQuery("name",searchContent));
-
Search search = new Search.Builder(searchSourceBuilder.toString())
-
.addIndex(Entity.INDEX_NAME).addType(Entity.TYPE).build();
-
try {
-
JestResult result = jestClient.execute(search);
-
return result.getSourceAsObjectList(Entity.class);
-
} catch (IOException e) {
-
LOGGER.error(e.getMessage());
-
e.printStackTrace();
-
}
-
return null;
-
}
-
}
这里插入数据的方式给了两种,一种是单次API直接插入,一种是利用ES的bulk批量插入。
做一个controller方面我们测试:
启动后在浏览器中请求http://localhost:7081/entityController/search?name=%E4%BA%BA%E6%89%8B%E4%BA%95
得到结果:
这里只返回了9条记录,而理论上ES默认的size是10,应该不是分页的问题,而是只能检索出9条匹配记录,用Kibana连上相同的搜索确认下:
这里用的是standard分词方式,将每个中文都作为了一个term,凡是包含“人”“手”“井”的都被搜索了出来,只是评分不同,如果想支持只能中文索引需要依赖ik插件,对于ik的详细介绍请见《ElasticSearch中文检索支持-ik插件》
OK,RestFul方式对ElasticSearch的检索已经搞定了,更多的扩展可以慢慢研究下QueryBuilders里的源码和批注。
第二种方式,利用Spring Data客户端方式:
事先说明此方式有个弊端,让我掉了坑里好久才爬上来,Spring Data ElasticSearch必须与ElasticSearch版本相匹配,否则在对接时ES端会报版本不匹配错误,例如我ES是5.6.1版本,Spring boot是1.5.6版本,错误如下:
为解决这个问题我查找了一些资料,Spring Data与elasticsearch版本对应关系如下:
spring data elasticsearch | elasticsearch |
3.0.0.RC2 | 5.5.0 |
3.0.0.M4 | 5.4.0 |
2.0.4.RELEASE | 2.4.0 |
2.0.0.RELEASE | 2.2.0 |
1.4.0.M1 | 1.7.3 |
1.3.0.RELEASE | 1.5.2 |
1.2.0.RELEASE | 1.4.4 |
1.1.0.RELEASE | 1.3.2 |
1.0.0.RELEASE | 1.1.1 |
而我用的Spring Boot 1.5.6版本对应的Spring Data ElasticSearch是2.1.6版本,不支持5.X的ES,所以报错。到本博文撰写为止,Spring Boot的RELEASE版本最新的是1.5.8,对应的Spring Data ElasticSearch是2.1.8,仍不支持5.X的ES,所以如果一定要使用Java客户端方式集成ES只能放弃Spring Boot直接使用Spring Data和Spring MVC,或者降低ES的版本使之与Spring boot匹配。
示例代码:https://github.com/yejingtao/forblog/tree/master/demo-data-elasticsearch
pom.xml依赖:
-
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
-
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
-
<modelVersion>4.0.0</modelVersion>
-
<groupId>yejingtao.demo.springcloud</groupId>
-
<artifactId>demo-data-elasticsearch</artifactId>
-
<version>0.0.1-SNAPSHOT</version>
-
<packaging>jar</packaging>
-
<name>demo-data-elasticsearch</name>
-
<url>http://maven.apache.org</url>
-
<properties>
-
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-
</properties>
-
<parent>
-
<groupId>org.springframework.boot</groupId>
-
<artifactId>spring-boot-starter-parent</artifactId>
-
<version>1.5.8.RELEASE</version>
-
</parent>
-
<dependencies>
-
<dependency>
-
<groupId>org.springframework.boot</groupId>
-
<artifactId>spring-boot-starter-web</artifactId>
-
</dependency>
-
<dependency>
-
<groupId>org.springframework.boot</groupId>
-
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
-
</dependency>
-
</dependencies>
-
</project>
不再引用Jest。
application.yml:
server:
port: 7081
spring:
data:
elasticsearch:
cluster-nodes: 192.168.226.133:9300
cluster-name: my-es
repositories:
enabled: true
注意这里是9300端口
Controller、主程序、Service接口同Jest项目不变,不再罗列
实体类稍作变化,指定ES中的index和type:
@Document(indexName="index_entity", type="tstype")
多一个Repository接口,无需实现类,spring data标准用法:
-
/**
-
* Entity ES操作类
-
* @author yejingtao
-
*
-
*/
-
public interface EntityRepository extends ElasticsearchRepository<Entity,Long>{
-
}
Service实现类与Jest的天壤之别了,从语法上可以看出更像是对数据库层的操作:
-
@Service
-
public class CityESServiceImpl implements CityESService{
-
private static final Logger LOGGER = LoggerFactory.getLogger(CityESServiceImpl.class);
-
int PAGE_SIZE = 15; //默认分页大小
-
int PAGE_NUMBER = 0; //默认当前分页
-
String SCORE_MODE_SUM = "sum"; //权重分求和模式
-
Float MIN_SCORE = 10.0F; //由于无相关性的分值默认为1, 设置权重分最小值为10
-
@Autowired
-
EntityRepository entityRepository;
-
/**
-
* 保存内容到ES
-
*/
-
@Override
-
public Long saveEntity(Entity entity) {
-
Entity entityResult = entityRepository.save(entity);
-
return entityResult.getId();
-
}
-
/**
-
* 在ES中搜索内容
-
*/
-
@Override
-
public List<Entity> searchEntity(int pageNumber, int pageSize, String searchContent){
-
if(pageSize==0) {
-
pageSize = PAGE_SIZE;
-
}
-
if(pageNumber<0) {
-
pageNumber = PAGE_NUMBER;
-
}
-
SearchQuery searchQuery = getEntitySearchQuery(pageNumber,pageSize,searchContent);
-
LOGGER.info("\n searchCity: searchContent [" + searchContent + "] \n DSL = \n "
-
+ searchQuery.getQuery().toString());
-
Page<Entity> cityPage = entityRepository.search(searchQuery);
-
return cityPage.getContent();
-
}
-
/**
-
* 组装搜索Query对象
-
* @param pageNumber
-
* @param pageSize
-
* @param searchContent
-
* @return
-
*/
-
private SearchQuery getEntitySearchQuery(int pageNumber, int pageSize, String searchContent) {
-
FunctionScoreQueryBuilder functionScoreQueryBuilder = QueryBuilders.functionScoreQuery()
-
.add(QueryBuilders.matchPhraseQuery("name", searchContent),
-
ScoreFunctionBuilders.weightFactorFunction(1000))
-
//.add(QueryBuilders.matchPhraseQuery("other", searchContent),
-
//ScoreFunctionBuilders.weightFactorFunction(1000))
-
.scoreMode(SCORE_MODE_SUM).setMinScore(MIN_SCORE);
-
//设置分页,否则只能按照ES默认的分页给
-
Pageable pageable = new PageRequest(pageNumber, pageSize);
-
return new NativeSearchQueryBuilder().withPageable(pageable).withQuery(functionScoreQueryBuilder).build();
-
}
-
}
测试方式同Jest。
这两种方式,从设计上来讲属于两种思路,Spring Data的思路就是将ElasticSearch当自家的数据仓库来管理,直接通过Java客户端代码操作ES;Jest的思路是将ElasticSearch当为独立的服务端,自己作为客户端用兼容性最强的RestFul格式来与之交互。
个人比较倾向于Jest方式,第一兼容性好,不需要考虑版本的问题。第二,从ElasticSearch本身的设计上来分析,9200是对外服务端口,9300是内部管理和集群通信端口,请求9200获取搜索服务更符合ES的设计初衷,不会影响集群内部的通信。
以上比较分析仅代表个人观点,欢迎大神么交流批评。