2019/03/06 09:00
ES文档官方:https://support.huaweicloud.com/usermanual-es/es_01_0024.html
华为云区域:华北北京1
ES集群控制台:地址:
ECS控制台:地址:
ps:操作环境:华为云服务器上命令操作,使用bulk API通过cURL命令导入数据文件,以JSON数据文件为例。
华为云ES集群内网ip:xx.xx.xx.xx:9200
★查询索引
curl 'http://xx.xx.xx.xx:9200/_cat/indices'
查询索引下的数据
curl -XGET 'http://xx.xx.xx.xx:9200/poc_index/_search?pretty'
删除索引
curl -XDELETE 'http://xx.xx.xx.xx:9200/ddddd_index'
查询某一个索引库中的数据
查询整个索引库:curl -XGET http://master:9200/bigdata_p/_search?pretty
在url后面加上一个pretty则会对返回结果进行格式化,
查询某一个type:curl -XGET http://master:9200/bigdata_p/product/_search?pretty
查询具体的一条记录:curl -XGET http://master:9200/bigdata_p/product/1?pretty
查询一条索引文档中的具体的字段:curl -XGET http://master:9200/bigdata_p/product/1?_source=name&pretty
如果要查询多个字段,使用","进行隔开。eg.
curl -XGET http://master:9200/bigdata_p/product/1?_source=name,author&pretty
获取source所有数据
curl -XGET http://master:9200/bigdata_p/product/1?_source&pretty
根据条件进行查询
curl -XGET http://master:9200/bigdata_p/product/_search?q=name:hbase,hive&pretty
删除索引下的数据
POST /poc_index/_delete_by_query
{
"query": {
"match_all": {
}
}
}
★创建索引:
curl -X PUT http://xx.xx.xx.xx:9200/my_store -d '
{
"settings": {
"number_of_shards": 1
},
"mappings": {
"products": {
"properties": {
"productName": {
"type": "text"
},
"size": {
"type": "keyword"
}
}
}
}
}' -H 'Content-Type: application/json'
ps: 添加 -H 'Content-Type: application/json' ,否则报错不支持json格式。
★本地创建test.json文件。添加内容:
{"index": {"_index":"my_store","_type":"products"}} {"productName": "2019秋装新款文艺衬衫女装","size": "M"}
-------------------------------------------------------------------------------------
ps:一定要记得在数据最后,添加换行,否则导入数据报错。!!!
需要注意的是{"index":{"_id":"1"}}和文件末尾另起一行换行是不可少的
参见文章:https://www.cnblogs.com/tonglin0325/p/8446975.html
★上传test.json文件到服务器,路径下执行命令导入数据。使用time命令打印命令的执行过程消耗的时间:
命令:
time curl -X PUT "http://xx.xx.xx.xx:9200/_bulk" -H 'Content-Type: application/json' --data-binary @test.json
以下为测试不同数量的数据导入结果:
1条:0.012s
1万条:3.013s
6万条:23.073s
=======================================================
使用springboot整合ES进行数据插入demo:
新建springboot项目:yml配置:
spring: #elasticsearch配置 data: elasticsearch: cluster-name: Es-23b6 cluster-nodes: 192.168.0.166:9300 repositories: enabled: true
ps:华为云ES控制台可以看到cluster-name和cluster-nodes,注意端口一定要改为9300,因为是集群的原因。否则异常:
org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: [{#transport#-1}{S428skP0SrqzcOn8W7vhHg}{192.168.0.166}{192.168.0.166:9200}] at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:349) ~[elasticsearch-6.4.3.jar!/:6.4.3] at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:247) ~[elasticsearch-6.4.3.jar!/:6.4.3] at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60) ~[elasticsearch-6.4.3.jar!/:6.4.3] at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:381) ~[elasticsearch-6.4.3.jar!/:6.4.3] at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:407) ~[elasticsearch-6.4.3.jar!/:6.4.3] at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:396) ~[elasticsearch-6.4.3.jar!/:6.4.3] at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:46) ~[elasticsearch-6.4.3.jar!/:6.4.3] at org.springframework.data.elasticsearch.core.ElasticsearchTemplate.index(ElasticsearchTemplate.java:577) ~[spring-data-elasticsearch-3.1.5.RELEASE.jar!/:3.1.5.RELEASE] at org.springframework.data.elasticsearch.repository.support.AbstractElasticsearchRepository.save(AbstractElasticsearchRepository.java:156) ~[spring
参见文章:https://blog.csdn.net/adsl624153/article/details/78935796
pom中引入依赖:
<!-- elasticsearch --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch </artifactId> <version>2.0.2.RELEASE</version> </dependency>
先对ES创建索引:【linux命令行执行】
创建索引
curl -X PUT http://192.168.0.166:9200/my_store -d '
{
"settings": {
"number_of_shards": 1
},
"mappings": {
"products": {
"properties": {
"id": {
"type": "long"
},
"productName": {
"type": "text"
},
"size": {
"type": "keyword"
}
}
}
}
}' -H 'Content-Type: application/json'
创建实体类:
package com.huawei.Bean; import org.springframework.data.elasticsearch.annotations.Document; import com.fasterxml.jackson.annotation.JsonProperty; import lombok.AllArgsConstructor; import lombok.Data; import lombok.NoArgsConstructor; /** * * @author :ayfei * @createTime :2019年3月5日下午2:13:31 * @description : */ @Data @AllArgsConstructor @NoArgsConstructor //indexName代表索引 名 称 ,type代表表名称 @Document(indexName = "my_store", type = "products") public class Notice { //id @JsonProperty("id") private Long id; // @JsonProperty("productName") private String productName; @JsonProperty("size") private String size; }
创建Dao层接口:
package com.huawei.Dao; /** * * @author :ayfei * @createTime :2019年3月5日下午2:15:26 * @description : */ import org.springframework.data.elasticsearch.repository.ElasticsearchRepository; import org.springframework.stereotype.Component; import com.huawei.Bean.Notice; @Component public interface NoticeDao extends ElasticsearchRepository<Notice, Long> { }
controller层部分代码,直接调用ES的相关API:
@RequestMapping(value = "/saveOne", method = RequestMethod.POST) public String saveOne(@RequestBody @Valid Notice article){ long start = System.currentTimeMillis(); log.info("=====start System.currentTimeMillis()====="+start); noticeDao.save(article); long end = System.currentTimeMillis(); log.info("=====end System.currentTimeMillis()====="+end); long timeRes = end-start; log.info("存储一条数据到ES的执行时间"+ timeRes + "ms"); stringRedisTemplate.opsForValue().set("timeOneToES", timeRes+"ms"); return "存储一条数据到ES的执行时间"+ timeRes + "ms"; }