ElasticSearch官方文档:https://www.elastic.co/guide/cn/elasticsearch/guide/current/index.html
目录
1)、JestAutoConfiguration(默认不生效)
2)、ElasticsearchAutoConfiguration
3)、ElasticsearchDataAutoConfiguration
4)、ElasticsearchRepositoriesAutoConfiguration
2)、SpringBoot整合的ES的版本要和docker的ES镜像版本匹配
-
简介
ElasticSearch是一个基于Lucene的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎,基于RESTful web接口。Elasticsearch是用Java语言开发的,并作为Apache许可条款下的开放源码发布,是一种流行的企业级搜索引擎。ElasticSearch用于云计算中,能够达到实时搜索,稳定,可靠,快速,安装使用方便。官方客户端在Java、.NET(C#)、PHP、Python、Apache Groovy、Ruby和许多其他语言中都是可用的。根据DB-Engines的排名显示,Elasticsearch是最受欢迎的企业搜索引擎,其次是Apache Solr,也是基于Lucene。
-
概念
- 以 员工文档 的形式存储为例:一个文档代表一个员工数据。存储数据到 ElasticSearch 的行为叫做 索引 ,但在索引一个文档之前,需要确定将文档存储在哪里。
- 一个 ElasticSearch 集群可以 包含多个 索引 ,相应的每个索引可以包含多个 类型 。 这些不同的类型存储着多个 文档 ,每个文档又有 多个 属性 。
- 类似关系:
索引-数据库
类型-表
文档-表中的记录
属性-列
索引相当于数据实例、类型相当于表、文档相当于表里面每一个行数据
-
ES的安装
1、docker安装ES
[root@localhost ~]# docker pull elasticsearch
2、docker启动ES
ES启动默认初始会占用2G的内存空间,虚拟机本机内存不够,因此限制的内存大小不然启动会报错
9200是web通信端口,9300ES各个节点之前的通信端口
[root@localhost ~]# docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -d -p 9200:9200 -p 9300:9300 --name es 5acf0e8da90b
-
ES基本使用
使用postman发送Restful请求
Elasticsearch 使用 JavaScript Object Notation(或者 JSON)作为文档的序列化格式。JSON 序列化为大多数编程语言所支持,并且已经成为 NoSQL 领域的标准格式。 它简单、简洁、易于阅读。
1)、添加数据
对于员工目录,我们将做如下操作:
- 每个员工索引一个文档,文档包含该员工的所有信息。
- 每个文档都将是
employee
类型 。 - 该类型位于 索引
megacorp
内。 - 该索引保存在我们的 Elasticsearch 集群中。
实践中这非常简单(尽管看起来有很多步骤),我们可以通过一条命令完成所有这些动作:
PUT /megacorp/employee/1
{
"first_name" : "John",
"last_name" : "Smith",
"age" : 25,
"about" : "I love to go rock climbing",
"interests": [ "sports", "music" ]
}
注意,路径 /megacorp/employee/1
包含了三部分的信息:
megacorp
:索引名称、employee
:类型名称、1
:特定雇员的ID
2)、获取数据
GET请求/megacorp/employee/1
3)、更多操作
https://www.elastic.co/guide/cn/elasticsearch/guide/current/_indexing_employee_documents.html
-
SpringBoot整合ElasticSearch
1、ES的自动配置
SpringBoot默认支持两种技术和ES进行交互,Jest和SpringBoot-data-elasticSearch
1)、JestAutoConfiguration(默认不生效)
如需要Jest生效需要导入io.searchbox.client.JestClient的jar包
import com.google.gson.Gson;
import io.searchbox.client.JestClient;
import io.searchbox.client.JestClientFactory;
import io.searchbox.client.config.HttpClientConfig;
import org.apache.http.HttpHost;
import org.springframework.beans.factory.ObjectProvider;
import org.springframework.boot.autoconfigure.AutoConfigureAfter;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.autoconfigure.elasticsearch.jest.JestProperties.Proxy;
import org.springframework.boot.autoconfigure.gson.GsonAutoConfiguration;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.boot.context.properties.PropertyMapper;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.util.Assert;
/**
* {@link EnableAutoConfiguration Auto-configuration} for Jest.
*
* @author Stephane Nicoll
* @since 1.4.0
* @deprecated since 2.2.0 in favor of other auto-configured Elasticsearch clients
*/
@Configuration(proxyBeanMethods = false)
@ConditionalOnClass(JestClient.class)
@EnableConfigurationProperties(JestProperties.class)
@AutoConfigureAfter(GsonAutoConfiguration.class)
@Deprecated
public class JestAutoConfiguration {
2)、ElasticsearchAutoConfiguration
1.TransportClient需要配置节点信息clusterNodes和clusterName
3)、ElasticsearchDataAutoConfiguration
会给容器中添加一个ElasticsearchTemplate来操作ES
4)、ElasticsearchRepositoriesAutoConfiguration
启用了ElasticsearchRepository接口可以对ES进行操作,类似于JPA的接口编程一样。即编写一个ElasticsearchRepository的子接口就可以对es进行增删改查的操作
@Configuration(proxyBeanMethods = false)
@ConditionalOnClass({ Client.class, ElasticsearchRepository.class })
@ConditionalOnProperty(prefix = "spring.data.elasticsearch.repositories", name = "enabled", havingValue = "true",
matchIfMissing = true)
@ConditionalOnMissingBean(ElasticsearchRepositoryFactoryBean.class)
@Import(ElasticsearchRepositoriesRegistrar.class)
public class ElasticsearchRepositoriesAutoConfiguration {
}
2、Jest怎么操作ES
@Configuration(proxyBeanMethods = false)
@ConditionalOnClass(JestClient.class)
@EnableConfigurationProperties(JestProperties.class)
@AutoConfigureAfter(GsonAutoConfiguration.class)
@Deprecated
public class JestAutoConfiguration {
@Bean(destroyMethod = "shutdownClient")
@ConditionalOnMissingBean
public JestClient jestClient(JestProperties properties, ObjectProvider<Gson> gson,
ObjectProvider<HttpClientConfigBuilderCustomizer> builderCustomizers) {
JestClientFactory factory = new JestClientFactory();
factory.setHttpClientConfig(createHttpClientConfig(properties, gson, builderCustomizers));
return factory.getObject();
}
protected HttpClientConfig createHttpClientConfig(JestProperties properties, ObjectProvider<Gson> gson,
ObjectProvider<HttpClientConfigBuilderCustomizer> builderCustomizers) {
HttpClientConfig.Builder builder = new HttpClientConfig.Builder(properties.getUris());
PropertyMapper map = PropertyMapper.get();
map.from(properties::getUsername).whenHasText()
.to((username) -> builder.defaultCredentials(username, properties.getPassword()));
Proxy proxy = properties.getProxy();
map.from(proxy::getHost).whenHasText().to((host) -> {
Assert.notNull(proxy.getPort(), "Proxy port must not be null");
builder.proxy(new HttpHost(host, proxy.getPort()));
});
map.from(gson::getIfUnique).whenNonNull().to(builder::gson);
map.from(properties::isMultiThreaded).to(builder::multiThreaded);
map.from(properties::getConnectionTimeout).whenNonNull().asInt(Duration::toMillis).to(builder::connTimeout);
map.from(properties::getReadTimeout).whenNonNull().asInt(Duration::toMillis).to(builder::readTimeout);
builderCustomizers.orderedStream().forEach((customizer) -> customizer.customize(builder));
return builder.build();
}
}
1)、引入jest的jar包
<dependency>
<groupId>io.searchbox</groupId>
<artifactId>jest</artifactId>
<version>6.3.1</version>
<type>pom</type>
</dependency>
2)、配置属性
#配置jest的uris
spring.elasticsearch.jest.uris=http://192.168.0.113:9200/
@ConfigurationProperties(prefix = "spring.elasticsearch.jest")
public class JestProperties {
/**
* Comma-separated list of the Elasticsearch instances to use.
*/
private List<String> uris = new ArrayList<>(Collections.singletonList("http://localhost:9200"));
/**
* Login username.
*/
private String username;
/**
* Login password.
*/
private String password;
3)、给ES中索引保存一个文档
@SpringBootTest
class SpringbootIntegrateElasticsearchApplicationTests {
@Autowired
JestClient jestClient;
/**
* 给ES中索引保存一个文档
*/
@Test
void contextLoads() {
Book book = new Book();
book.setId(1);
book.setBookName("西游记");
book.setAuthor("吴承恩");
//建立索引
Index index = new Index.Builder(book).index("es").type("book").build();
try {
jestClient.execute(index);
} catch (IOException e) {
e.printStackTrace();
}
}
}
4)、搜索ES
@SpringBootTest
class SpringbootIntegrateElasticsearchApplicationTests {
@Autowired
JestClient jestClient;
/**
* 测试ES搜索
*/
@Test
public void searchEs(){
//查询表达式
String json = "{\n" +
" \"query\" : {\n" +
" \"match\" : {\n" +
" \"bookName\" : \"西游记\"\n" +
" }\n" +
" }\n" +
"}";
//构建搜索功能
Search build = new Search.Builder(json).addIndex("es").addType("book").build();
//执行
try {
SearchResult execute = jestClient.execute(build);
System.out.println(execute.getJsonString());
} catch (IOException e) {
e.printStackTrace();
}
}
}
3、SpringData操作ES
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
1)、配置属性
#Springdata操作es配置
spring.data.elasticsearch.cluster-nodes=192.168.0.113:9300
#这里可以不用设置默认elasticsearch
spring.data.elasticsearch.cluster-name=elasticsearch
2)、SpringBoot整合的ES的版本要和docker的ES镜像版本匹配
1.升级SpringBoot版本或者安装对应的elasticSearch
https://docs.spring.io/spring-data/elasticsearch/docs/3.2.5.RELEASE/reference/html/#reference
Spring Data Release Train | Spring Data Elasticsearch | Elasticsearch | Spring Boot |
---|---|---|---|
Moore | 3.2.x | 6.8.4 | 2.2.x |
Lovelace | 3.1.x | 6.2.2 | 2.1.x |
Kay[1] | 3.0.x[1] | 5.5.0 | 2.0.x[1] |
Ingalls[1] | 2.1.x[1] | 2.4.0 | 1.5.x[1] |
3)、ElasticsearchRepository子接口
public interface BookRepository extends ElasticsearchRepository<Book,Integer> {
public List<Book> findBookByBookNameLike(String bookName);
}
4)、需要标注索引和类型
@Document(indexName = "es",type = "book")
public class Book {
@JestId//表示是一个主键
private Integer id;
private String bookName;
private String author;
@Override
public String toString() {
return "Book{" +
"id=" + id +
", bookName='" + bookName + '\'' +
", author='" + author + '\'' +
'}';
}
public Integer getId() {
return id;
}
public void setId(Integer id) {
this.id = id;
}
public String getBookName() {
return bookName;
}
public void setBookName(String bookName) {
this.bookName = bookName;
}
public String getAuthor() {
return author;
}
public void setAuthor(String author) {
this.author = author;
}
}
5)、添加索引文档
@SpringBootTest
class SpringbootIntegrateElasticsearchApplicationTests {
/**
* ElasticsearchRepository子接口
*/
@Autowired
BookRepository bookRepository;
/**
* 使用ElasticsearchRepository子接口操作es
*/
@Test
public void testEs01(){
Book book = new Book();
book.setId(3);
book.setBookName("水浒传");
book.setAuthor("施耐庵");
bookRepository.index(book);
}
6)、自定义接口方法查询ES索引
@SpringBootTest
class SpringbootIntegrateElasticsearchApplicationTests {
@Autowired
JestClient jestClient;
/**
* ElasticsearchRepository子接口
*/
@Autowired
BookRepository bookRepository;
/**
* 使用自定方法查询
*/
@Test
public void testEs02(){
List<Book> books = bookRepository.findBookByBookNameLike("水");
books.stream().forEach(System.out::println);
}
/**
* 使用ElasticsearchRepository子接口操作es
*/
@Test
public void testEs01(){
Book book = new Book();
book.setId(1);
book.setBookName("水月洞天");
book.setAuthor("张三");
bookRepository.index(book);
}
7)、执行异常
docker启动es后9300访问不到es容器并且报错:
NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{PfqWva2-TWuAF-bmcr1jRg}{192.168.0.113}{192.168.0.113:9300}]
]
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:352)
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:248)
at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60)
at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:388)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:403)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:391)
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:46)
at org.springframework.data.elasticsearch.core.ElasticsearchTemplate.index(ElasticsearchTemplate.java:589)
at
解决方案:
1.进入es容器
[root@localhost ~]# docker exec -it 1fb31461934c /bin/bash
2.由于容器内不允许编辑所以要将文件拷贝出来
[root@localhost ~]# docker cp 1fb31461934c:/usr/share/elasticsearch/config/elasticsearch.yml /usr/share/elasticsearch.yml
3.拷贝完成后停止并删除原来启动的容器
4.修改/usr/share/elasticsearch.yml 将transport.host:0.0.0.0前的#去掉后保 存文件退出。其作用是允许任何ip地址访问elasticsearch .开发测试阶段可以这么做,生 产环境下指定具体的IP。
这里ES5.x版本和6.X版本不同,6.x版本里面没有transport.host:0.0.0.0 自己添加即可
5.然后启动容器
将外部的配置文件映射到容器里面的配置文件
[root@localhost ~]# docker run -d -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -p 9200:9200 -p 9300:9300 --name es01 -v /usr/share/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml 5acf0e
8da90b