SpringBoot+elasticsearchTemplate实现es索引数据的创建查询删除

通过elasticsearch实现创建索引、创建mapping映射、插入数据、模糊查询索引、查询index所对应的mapping、数据的滚动条件查询、删除指定数据

目录

elasticsearchTemplate创建索引、创建mapping映射（默认创建类型为doc）

elasticsearchTemplate添加数据

模糊查询index(查询以xxx开头的所有索引）并查询其对应的mapping包含的字段

elasticsearchTemplate滚动条件查询

elasticsearchTemplate查询指定索引类型下的所有数据

elasticsearchTemplate删除前缀为channel_channelId索引下taskId为xx的所有数据

---

elasticsearchTemplate创建索引、创建mapping映射（默认创建类型为doc）

// mapping对象转为json字符串
String s = objectMapper.writeValueAsString(channelOutput.getMappings());
// 拼接索引名称index
String index = "channel_" + channelAdd.getChannelId() + "_" + channelOutput.getType();
// 判断索引是否已存在
boolean b = elasticsearchTemplate.indexExists(index);
// es索引不存在創建索引，mapping
if (!b){
   elasticsearchTemplate.createIndex(index);
   elasticsearchTemplate.putMapping(index, "doc", s);
}

elasticsearchTemplate添加数据

List<IndexQuery> list=new ArrayList<>();
IndexQuery indexQuery = new IndexQuery();
// 设置索引名称       indexQuery.setIndexName("channel_"+taskDataAmount.getChannelId()+"_"+taskDataAmount.getType());
// 设置索引下的类型
indexQuery.setType("doc");
 // 需要添加的数据           indexQuery.setSource(objectMapper.writeValueAsString(taskDataAmount.getData()));
list.add(indexQuery);
elasticsearchTemplate.bulkIndex(list);

 特别说明：添加的数据结构和所设置的mapping结构不一样也可以添加成功

例如：mapping如下

{
  "task_log": {
    "mappings": {
      "data": {
        "properties": {
          "className": {
            "type": "keyword"
          },
          "dataTypeId": {
            "type": "keyword"
          },
          "engineId": {
            "type": "keyword"
          },
          "engineName": {
            "type": "keyword"
          },
          "error": {
            "type": "keyword"
          },
          "extendsInfo": {
            "type": "keyword"
          },
          "method": {
            "type": "keyword"
          },
          "taskId": {
            "type": "keyword"
          }
        }
      }
    }
  }
}

添加数据时，除了className这些字段还可以添加一些mapping中没有的字段

模糊查询index(查询以xxx开头的所有索引）并查询其对应的mapping包含的字段

// 封装类
@Data
public class FieldVo {
    private String indexName;

    private Set<String> fieldNames;
}

List<FieldVo> fieldVos=new ArrayList<>();
Client client = elasticsearchTemplate.getClient();
IndicesStatsRequest indicesStatsRequest = new IndicesStatsRequest();
// 前缀+*
indicesStatsRequest.indices("channel_"+taskLogs.getChannelId()+"*");

Map<String,IndexStats>stats=client.admin().indices().stats(indicesStatsRequest).actionGet().getIndices();

for(IndexStats stat:stats.values()) {
   FieldVo fieldVo=new FieldVo();
   String in = stat.getIndex();//获取索引值
   fieldVo.setIndexName(in);
//获取制定index下的mapping
   Map<String,String> o = (Map<String, String>) elasticsearchTemplate.getMapping(in, "doc").get("properties");
//获取所有的字段
Set<String> fields = o.keySet();
fieldVo.setFieldNames(fields);

 以上两张图分别是Map stats和Map o的数据结构

elasticsearchTemplate滚动条件查询

 public  ResultModel getDataDetail(String index,String scrollId,Long taskLogId,Integer pageSize){
        if (StringUtils.isEmpty(index)){
            return ResultModel.fail("索引为空");
        }
        if (null==taskLogId){
            return ResultModel.fail("taskLogId不能为空");
        }
        if (null==pageSize){
            pageSize=20;
        }
        TaskLogs taskLogs = taskLogsMapper.selectByPrimaryKey(taskLogId);
        // 创建一个查询条件对象
        BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
        // 拼接查询条件
        queryBuilder.must(QueryBuilders.termQuery("taskId",taskLogs.getTaskId()));
        queryBuilder.must(QueryBuilders.termQuery("count",taskLogs.getCount()));
        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withIndices(index)//索引名
                .withTypes("doc")//类型名
                .withPageable(PageRequest.of(0, pageSize))//从0页开始查，每页1000个结果
                .withQuery(queryBuilder)
                .build();
        Page<Map> maps =null;
        if (StringUtils.isEmpty(scrollId)){
            maps=elasticsearchTemplate.startScroll(300000L, searchQuery, Map.class);
        }else {
            maps =elasticsearchTemplate.continueScroll(scrollId, 300000L, Map.class);

        }
        return ResultModel.success(maps);
}

第一次查询时设置的pageSize，后续的查询中不能进行修改，因为是用的一个query

300000L是设置的过期时间

第一次请求是会返回一个scrollId，后续往下查询都需要把这个作为参数去请求

请求结果示例：

{
    "code": 200,
    "message": "ok",
    "data": {
        "content": [
            {
                "id": "324",
                "name": "testONe",
                "username": "test",
                "count": 3,
                "taskId": 10
            }
        ],
        "pageable": "INSTANCE",
        "facets": [],
        "aggregations": null,
        "scrollId": "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAyiXFkdkbm91M0hYUU9xSG13R3A0UHdoWkEAAAAAAAMolhZHZG5vdTNIWFFPcUhtd0dwNFB3aFpBAAAAAAAG2RcWdDJxTUx6QllUbmVmbjM2b0UzSmlFdwAAAAAABtgcFkpvMlZ0X04yVGVxdFE2ZWNOWEZJOWcAAAAAAAbYHRZKbzJWdF9OMlRlcXRRNmVjTlhGSTln",
        "totalElements": 1,
        "totalPages": 1,
        "number": 0,
        "size": 0,
        "sort": {
            "sorted": false,
            "unsorted": true
        },
        "numberOfElements": 1,
        "first": true,
        "last": true
    }
}

elasticsearchTemplate查询指定索引类型下的所有数据

public List<Map<String, Object>> test(){
        Client client = elasticsearchTemplate.getClient();
        SearchRequestBuilder srb = client
                .prepareSearch("channel_weibo_follow")
                .setTypes("doc");

        SearchResponse sr = srb.setQuery(QueryBuilders.matchAllQuery()).execute().actionGet();

        SearchHits hits = sr.getHits();
        List<Map<String, Object>> list = new ArrayList<Map<String, Object>>();
        for (SearchHit hit : hits) {
            Map<String, Object> source = hit.getSourceAsMap();
            list.add(source);
            System.out.println(hit.getSourceAsString());
        }
        return list;
}

 返回的数据：

{
    "code": 200,
    "message": "ok",
    "data": [
        {
            "uid": 3800468188,
            "fansUid": 1730077315,
            "count": 2,
            "taskId": 1
        },
        {
            "uid": 2634488643,
            "fansUid": 1730077315,
            "count": 2,
            "taskId": 1
        },
        {
            "uid": 2003394681,
            "fansUid": 1730077315,
            "count": 2,
            "taskId": 1
        },
        {
            "uid": 1782432341,
            "fansUid": 1730077315,
            "count": 2,
            "taskId": 1
        },
        {
            "uid": 1821525001,
            "fansUid": 1730077315,
            "count": 2,
            "taskId": 1
        },
        {
            "uid": 1709157165,
            "fansUid": 1730077315,
            "count": 2,
            "taskId": 1
        }
    ]
}

elasticsearchTemplate删除前缀为channel_channelId索引下taskId为xx的所有数据

DeleteQuery deleteQuery=new DeleteQuery();
        BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
//        // 拼接查询条件
        queryBuilder.must(QueryBuilders.termQuery("taskId",taskLogs.getTaskId()));
// 查询出所有的索引
        IndicesStatsRequest indicesStatsRequest = new IndicesStatsRequest();
        indicesStatsRequest.indices("channel_"+channelId+"*");
        Map<String, IndexStats> stats=elasticsearchTemplate.getClient().admin().indices().stats(indicesStatsRequest).actionGet().getIndices();
// 根据索引遍历删除
        for(IndexStats stat:stats.values()) {
            deleteQuery.setIndex(stat.getIndex());
            deleteQuery.setQuery(queryBuilder);
            deleteQuery.setType("doc");
            elasticsearchTemplate.delete(deleteQuery);
        }