High Level Rest Client不可用？试试Java API Client操作ES

Java API Client操作ES

---

前言

之前在学习ES7.16的时候，官方已经不推荐使用High Level Rest Client，并且在ES8的时候需要使用Java API Client来操作，所以干脆直接研究一下总结一下。

---

一、使用Java API Client需要引入什么？

官方建议引入方案 官方建议方案，但是实际在测试中会有一些报错。所以我的引入方式是

		<dependency>
            <groupId>co.elastic.clients</groupId>
            <artifactId>elasticsearch-java</artifactId>
            <version>7.16.3</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-client</artifactId>
            <version>7.16.3</version>
        </dependency>
        <dependency>
            <groupId>org.glassfish</groupId>
            <artifactId>jakarta.json</artifactId>
            <version>2.0.1</version>
        </dependency>

二、使用步骤

1.创建index

我们要创建成如下索引
1.test前缀+时间戳：这是因为索引随着使用会逐渐变大，所以会定期重构索引，为了避免定时重建对业务的影响，也为了对业务隐藏具体索引，后面会再创建一个别名
2.创建的mappings和settings如下

{
    "settings": {
        "number_of_shards": 3,
        "number_of_replicas": 2
    },
    "mappings": {
        "properties": {
            "title": {
                "type": "text",
                "analyzer": "ik_max_word"
            },
            "status": {
                "type": "short"
            },
            "updateTime": {
                "type": "date",
                "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
            },
            "updateTimeDay": {
                "type": "keyword"
            }
        }
    }
}

代码如下

public String buildIndex() throws IOException {
        String index = "test_" + System.currentTimeMillis();
        CreateIndexResponse createIndexResponse = elasticsearchClient.indices()
                .create(builder -> builder.index(index)
                        .settings(getSettings())
                        .mappings(getMapping()));
        if (!createIndexResponse.acknowledged()) {
            log.error("createIndexResponse.acknowledged() is not true");
        }
        return index;
    }

    private IndexSettings getSettings() {
        return new IndexSettings.Builder()
                .numberOfShards("3")
                .numberOfReplicas("2")
                .build();
    }

    private TypeMapping getMapping() {
        return new TypeMapping.Builder()
                .properties(MapBuilder.<String, Property>builder(HashMap.class)
                        .put("title", new Property.Builder().text(builder -> builder.analyzer("ik_max_word")).build())
                        .put("status", new Property.Builder().short_(builder -> builder).build())
                        .put("updateTime", new Property.Builder().date(builder -> builder.format("yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis")).build())
                        .put("updateTimeDay", new Property.Builder().keyword(builder -> builder).build())
                        .build())
                .build();
    }

1.别名操作

1. 检查别名&获取别名对应的真实索引名

	@Override
    public String getAliasRealIndex(String alias) throws IOException {
        if (checkAliasExists(alias)) {
            GetAliasResponse aliasResponse = elasticsearchClient.indices().getAlias(builder -> builder.name(alias));
            Map<String, IndexAliases> result = aliasResponse.result();
            Optional<Map.Entry<String, IndexAliases>> first = result.entrySet().stream().findFirst();
            return first.get().getKey();
        }
        return null;
    }

    @Override
    public Boolean checkAliasExists(String alias) throws IOException {
        ExistsAliasRequest request = new ExistsAliasRequest.Builder().name(alias).build();
        BooleanResponse booleanResponse = elasticsearchClient.indices().existsAlias(request);
        return booleanResponse.value();
    }

2. 如果有索引则切换索引，没有则新建索引。

	@Override
    public void createOrChangeAlias(String alias, String index) throws IOException {
        String aliasRealIndex = getAliasRealIndex(alias);
        System.out.println("aliasRealIndex:" + aliasRealIndex);
        doSwitchAliases(alias, aliasRealIndex, index);
    }

    @Override
    public void doSwitchAliases(String alias, String oldIndex, String newIndex) throws IOException {
        //如果别名存在
        if (Objects.nonNull(oldIndex)) {
            UpdateAliasesRequest request = new UpdateAliasesRequest.Builder().actions(actions(alias, oldIndex, newIndex)).build();
            System.out.println("+++++++++++++++++++++++++++" + ESUtil.printEsBySearchRequest(request));
            UpdateAliasesResponse updateAliasesResponse = elasticsearchClient.indices().updateAliases(request);
            if (!updateAliasesResponse.acknowledged()) {
                log.error("updateAliasesResponse.acknowledged() is not true");
            }
        } else {
            //如果不存在创建别名
            PutAliasRequest request = new PutAliasRequest.Builder().index(newIndex).name(alias).build();
            PutAliasResponse putAliasResponse = elasticsearchClient.indices().putAlias(request);
            if (!putAliasResponse.acknowledged()) {
                log.error("putAliasResponse.acknowledged() is not true");
            }
        }
    }

    /**
     * 构建别名切换的原子操作
     * @param alias
     * @param oldIndex
     * @param newIndex
     * @return
     */
    private List<Action> actions(String alias, String oldIndex, String newIndex) {
        return ListBuilder.<Action>builder(ArrayList.class)
                .add(new Action.Builder()
                        .remove(builder -> builder.index(oldIndex).alias(alias))
                        .build())
                .add(new Action.Builder()
                        .add(builder -> builder.index(newIndex).alias(alias))
                        .build())
                .build();
    }

3.数据写入

1. 批量写入不包含更新

@Override
    public void batchSyncInfoToES(List<TestEntity> testEntities, String index) throws IOException {
        BulkRequest bulkRequest = new BulkRequest.Builder()
                .operations(testEntities.stream().map(this::convertByTestEntity).collect(Collectors.toList()))
                .index(index)
                .build();
        System.out.println("+++++++++++++++++++++++++++" + ESUtil.printEsBySearchRequest(bulkRequest));
        BulkResponse bulkResponse = elasticsearchClient.bulk(bulkRequest);
//        List<BulkResponseItem> items = bulkResponse.items();
//        System.out.println("+++++++++++++++++++++++++++" + JSON.toJSONString(items));
        // Log errors, if any
        if (bulkResponse.errors()) {
            log.error("Bulk had errors");
            for (BulkResponseItem item : bulkResponse.items()) {
                if (item.error() != null) {
                    log.error(item.error().reason());
                }
            }
        }
    }

    private BulkOperation convertByTestEntity(TestEntity entity) {
        return new BulkOperation.Builder()
                .create(new CreateOperation.Builder<TestDTO>()
                        .id(String.valueOf(entity.getId()))
                        .document(converByTestEntity(entity))
                        .build())
                .build();
    }

    DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd");

    private TestDTO converByTestEntity(TestEntity entity) {
        return TestDTO.builder()
                .title(entity.getTitle())
                .status(entity.getStatus())
                .updateTime(entity.getUpdateTime())
                .updateTimeDay(formatter.format(Instant.ofEpochMilli(entity.getUpdateTime()).atZone(ZoneId.systemDefault()).toLocalDateTime()))
                .build();
    }

2. 增量更新（包括新增和更新）

@Override
    public void batchIncreaseSyncInfoToES(List<TestEntity> testEntities, String index) throws IOException {
        BulkRequest bulkRequest = new BulkRequest.Builder()
                .operations(testEntities.stream().map(testEntitie -> convertByLianV1ArticleCheck(testEntitie, index)).filter(Objects::nonNull).collect(Collectors.toList()))
                .index(index)
                .build();
        System.out.println("+++++++++++++++++++++++++++" + ESUtil.printEsBySearchRequest(bulkRequest));
        BulkResponse bulkResponse = elasticsearchClient.bulk(bulkRequest);
        // Log errors, if any
        if (bulkResponse.errors()) {
            log.error("Bulk had errors");
            for (BulkResponseItem item : bulkResponse.items()) {
                if (item.error() != null) {
                    log.error(item.error().reason());
                }
            }
        }
    }

    private BulkOperation convertByLianV1ArticleCheck(TestEntity testEntity, String index) {
        TestDTO testDTO = converByTestEntity(testEntity);
        try {
            if (checkExists(index, String.valueOf(testEntity.getId()))) {
                return new BulkOperation.Builder()
                        .update(new UpdateOperation.Builder<Map<String, Object>>()
                                .id(String.valueOf(testEntity.getId()))
                                .document(MapBuilder.<String, Object>builder(HashMap.class).put("doc", testDTO).build())//Java client bug，update语句没有生成doc的属性，直接和create语法一样了,这里手动添加一下解决问题，ref:https://github.com/elastic/elasticsearch-py/issues/1042
                                .build())
                        .build();
            } else {
                return new BulkOperation.Builder()
                        .create(new CreateOperation.Builder<TestDTO>()
                                .id(String.valueOf(testEntity.getId()))
                                .document(testDTO)
                                .build())
                        .build();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return null;
    }

    @Override
    public Boolean checkExists(String index, String id) throws IOException {
        co.elastic.clients.elasticsearch.core.ExistsRequest existsRequest = new ExistsRequest.Builder().id(id).index(index).type("_doc").build();
//        System.out.println("+++++++++++++++++++++++++++" + ESUtil.printEsBySearchRequest(existsRequest));
        BooleanResponse exists = elasticsearchClient.exists(existsRequest);
        System.out.println("exists " + exists.value());
        return exists.value();
    }

4.查询

查询title包含test且高亮并且status是1的记录，理论上有ID是2，3，4，5的记录
ES的DSL语法如下

{
    "from": 0,
    "highlight": {
        "fields": {
            "title": {}
        },
        "fragment_size": 800000,
        "number_of_fragments": 0,
        "post_tags": [
            "</font>"
        ],
        "pre_tags": [
            "<font color='red'>"
        ],
        "require_field_match": false
    },
    "query": {
        "bool": {
            "must": [
                {
                    "term": {
                        "status": 1
                    }
                }
            ],
            "should": [
                {
                    "match_phrase": {
                        "title": {
                            "query": "test"
                        }
                    }
                }
            ],
             "minimum_should_match": 1
        }
    },
    "size": 10,
    "sort": [
        {
            "updateTime": {
                "order": "desc"
            }
        }
    ]
}

对应java代码如下

	@Override
    public List<Map<String, Object>> search(String keyword) throws IOException {
        SearchRequest searchRequest = new SearchRequest.Builder()
                .highlight(new Highlight.Builder()//构建高亮字段
                        .fields(MapBuilder.<String, HighlightField>builder(HashMap.class)
                                .put("title", new HighlightField.Builder().build())
                                .build())
                        .preTags("<font color='red'>")
                        .postTags("</font>")
                        .fragmentSize(800000)//最大高亮分片数
                        .numberOfFragments(0)//从第一个分片开始获取高亮片段
                        .requireFieldMatch(false)//fragmentSize、numberOfFragments、requireFieldMatch三个字段解决高亮字段展示不全的问题 https://elasticsearch.cn/question/2475
                        .build())
                .query(QueryBuilders.bool()//布尔查询构造器
                        .should(ListBuilder.<Query>builder(ArrayList.class)
                                .add(QueryBuilders.matchPhrase()
                                        .field("title")
                                        .query(keyword)
                                        .analyzer("ik_smart")
                                        .build()
                                        ._toQuery())
                                .build())
                        .must(QueryBuilders.term()
                                .field("status")
                                .value(builder -> builder.longValue(1))
                                .build()
                                ._toQuery())
                        .minimumShouldMatch(String.valueOf(1))//bool字句里既有should，又有must需要显示指定此指，不然should不命中也会返回结果 https://blog.csdn.net/wlei0618/article/details/127577614
                        .build()
                        ._toQuery())
                .sort(builder -> builder.field(sortBuilder -> sortBuilder.field("updateTime").order(SortOrder.Desc)))
                .from(0)
                .size(10)
                .index(Constans.ALIAS)
                .type("_doc")
                .build();
        System.out.println("+++++++++++++++++++++++++++" + ESUtil.printEsBySearchRequest(searchRequest));
        SearchResponse searchResponse = elasticsearchClient.search(searchRequest, Map.class);
        //命中list
        List<Hit> list = searchResponse.hits().hits();
        //处理结果list
        List<Map<String, Object>> resList = new ArrayList<>();
        for (Hit hit : list) {
            Map<String, Object> resMap = new HashMap<>();
            Map highLight = hit.highlight();
            Double score = hit.score();
            Map map = (Map) hit.source();
            resMap.put("highLight", highLight);
            resMap.put("score", score);
            resMap.put("source", map);
            resList.add(resMap);
        }
        return resList;
    }

---

总结

以上是一些简单的操作，仅供参考，如果有不对的地方还请各位指正。源码可以参考 源码