03-Elasticsearch的 TransportClient

最新推荐文章于 2024-04-23 10:00:56 发布

yk10010

最新推荐文章于 2024-04-23 10:00:56 发布

阅读量423

点赞数 1

分类专栏： Elasticsearch 文章标签： elasticsearch

本文链接：https://blog.csdn.net/yk10010/article/details/103918855

版权

Elasticsearch 专栏收录该内容

4 篇文章 0 订阅

订阅专栏

Elasticsearch的 TransportClient

1. Java Client 说明

java client 使用 TransportClient，各种操作本质上都是异步的(可以用 listener，或返回 Future ）。
注意：ES的发展规划中在7.0版本开始将废弃 TransportClient，8.0版本中将完全移除 TransportClient，取而代之的是High Level REST Client。
High Level REST Client 中的操作API和java client 大多是一样的，除了连接方式InitClient代码不一样。

2.官方学习链接

https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/index.html

3. 兼容性说明

请使用与服务端ES版本一致的客户端版本

4. Java Client maven 集成

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-high-level-client</artifactId>
    <version>6.3.2</version>
</dependency>

5. Transport API使用示例

添加配置

注意TransportClient的端口使用9300

@Configuration
@EnableElasticsearchRepositories(basePackages = "com.xxx.xxx.es.repository")
public class ElasticConfiguration {
    private static final Logger LOGGER = LoggerFactory.getLogger(ElasticConfiguration.class);

    @Value("${elasticsearch.cluster-name:my-application}")
    private String esClusterName;

    @Value("#{'${elasticsearch.cluster-nodes:127.0.0.1:9300}'.split(',')}")
    private List<String> serversList;

    private TransportClient client;

    @PostConstruct
    public void initialize() {
        try {
            Settings esSettings = Settings.builder()
                    .put("cluster.name", esClusterName)
                    .put("client.transport.sniff", true).build();
            client = new PreBuiltTransportClient(esSettings);
            for (String hostPort : serversList) {
                String[] addressPort = hostPort.split(":");
                LOGGER.info("=====> 【ES : ip:{} port:{}】", addressPort[0], addressPort[1]);
                client.addTransportAddress(new TransportAddress(InetAddress.getByName(addressPort[0]),
                        Integer.valueOf(addressPort[1])));
            }
            LOGGER.info("=====> 【ES : TransportClient init Success】");
        } catch (Exception e) {
            LOGGER.error("=====> 【ES : TransportClient init Failed】MSG:{}", e);
        }
    }

    @Bean
    public Client client() {
        return client;
    }

    @Bean
    public ElasticsearchTemplate elasticsearchTemplate() throws Exception {
        if (null == client ) {
            LOGGER.info("=====> 【ES : TransportClient is null】");
            return null;
        } else {
            return new ElasticsearchTemplate(client);
        }
    }

    @PreDestroy
    public void destroy() {
        if (client != null) {
            client.close();
        }
    }
}

定义ES实体

@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
@Accessors(chain = true)
@Document(indexName = "索引名称", type = "索引类型")
public class PubIndivItem extends  BaseItem {

    @Id
    private String id;
    
    @Field(type= FieldType.Keyword)
    private String indivUniCode;

    @Field(type= FieldType.Keyword)
    private String photoPath;

    @Field(type = FieldType.Text, index = true, analyzer = "ik_max_word", searchAnalyzer = "ik_smart")
    private String indivName;

    @Field(type= FieldType.Keyword)
    private  String posiName;

    @Field(type= FieldType.Keyword)
    private String servingEntName;

    @Field(type= FieldType.Keyword)
    private Integer totalServingNum;

    @Field(type= FieldType.Keyword)
    private String servingAllEnt;
}

创建Repository

public interface PubIndivItemRepository extends ElasticsearchRepository<PubIndivItem, String> {

}

注入模板

@Autowired
private ElasticsearchTemplate elasticsearchTemplate;

模板使用

验证索引是否存在

elasticsearchTemplate.indexExists(indexName);

创建索引

elasticsearchTemplate.createIndex(indexName);

删除索引

elasticsearchTemplate.deleteIndex(indexName);

删除索引中的数据

public Long deleteAllIndexData(String indexName) {
        DeleteByQueryRequestBuilder builder = DeleteByQueryAction.INSTANCE
                .newRequestBuilder(client)
                .filter(QueryBuilders.boolQuery()).
                        source(indexName);
        BulkByScrollResponse response = builder.get();
        if (null != response) {
            Long deletedNum = response.getDeleted();
            LOGGER.info("==>[删除{}索引数据，删除条数:{}条！", indexName, response.getDeleted());
            return deletedNum;
        }
        return  Long.valueOf("0");
    }

批量保存数据

    public void saveBatchDeepReport( List<DeepReportItem> deepReportItemList) {
        List<List<DeepReportItem>> itemLists = Lists.partition(deepReportItemList, BATCH_NUM);
        IndexQuery indexItem  = null;
        List<IndexQuery> queries = null;
        for (List<DeepReportItem> itemList : itemLists) {
            queries = Lists.newArrayListWithExpectedSize(itemList.size());
            for(DeepReportItem item : itemList) {
                indexItem = new IndexQuery();
                indexItem.setIndexName(ESIndexEnum.MR_DEEP_REPORT_INDEX.getCode());
                indexItem.setObject(item);
                queries.add(indexItem);
            }
            elasticsearchTemplate.bulkIndex(queries);
        }
        elasticsearchTemplate.refresh(ESIndexEnum.MR_DEEP_REPORT_INDEX.getCode());
    }

分页查询数据(浅分页)

public InfoResponseDto findInfoPageAndStatistics(SearchCustInfoDto custinfoSearch) {
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        if (StringUtils.isNotBlank(custinfoSearch.getKeyword())) {
            boolQuery.must(QueryBuilders.matchQuery("custName", custinfoSearch.getKeyword()));
        } else {
            boolQuery.must(QueryBuilders.matchAllQuery());
        }
        if (StringUtils.isNotBlank(custinfoSearch.getProvince())) {
            boolQuery.must(QueryBuilders.matchPhraseQuery("province", custinfoSearch.getProvince()));
        }
        NativeSearchQuery nativeSearchQuery = new NativeSearchQueryBuilder()
                .withQuery(boolQuery)
                .addAggregation(AggregationBuilders.terms("province_count").field("province"))
                .withPageable(PageRequest.of(custinfoSearch.getPageNo()-1, custinfoSearch.getPageSize()))
                .withSort(SortBuilders.scoreSort())
                .withHighlightFields(buildHighlightField())
                .build();
        AggregatedPage<CustInfoItem> suppliers = elasticsearchTemplate.queryForPage(nativeSearchQuery,
                CustInfoItem.class, new SearchResultMapper() {
                    @Override
                    public <T> AggregatedPage<T> mapResults(SearchResponse response, Class<T> clazz, Pageable pageable) {
                        SearchHits searchHits = response.getHits();
                        Long totalHis = searchHits.getTotalHits() >= TEN_THOUSAND ? TEN_THOUSAND : searchHits.getTotalHits();
                        SearchHit[] hits = searchHits.getHits();
                        ArrayList<CustInfoItem> suppliers = new ArrayList<>();
                        CustInfoItem custInfoItem = null;
                        for (SearchHit hit : hits) {
                            Map<String, Object> sourceAsMap = hit.getSourceAsMap();
                            custInfoItem = CustInfoItem.builder()
                                    .custId(convertString(sourceAsMap, "custId"))
                                    .custName(convertString(sourceAsMap, "custName"))
                                    .build();
                            custInfoItem.setTotalHis(totalHis);
                            custInfoItem.setItemScore(hit.getScore());
                            // 处理高亮
                            Map<String, HighlightField> highlightFields = hit.getHighlightFields();
                            if (highlightFields.get("custName") != null) {
                                String nameHighlight = highlightFields.get("custName").getFragments()[0].toString();
                                custInfoItem.setCustName(nameHighlight);
                            }
                            try {
                                LOGGER.info("========》{}", MAPPER.writeValueAsString(custInfoItem));
                            } catch (JsonProcessingException e) {
                                LOGGER.info("========》【JSON解析异常】");
                            }
                            suppliers.add(custInfoItem);
                        }
                        return new AggregatedPageImpl((List<T>) suppliers, pageable,
                                response.getHits().getTotalHits(), response.getAggregations());
                    }
                });
        // 统计处理
        Terms provinceTerm = suppliers.getAggregations().get("province_count");
        Terms industryTypeTerm = suppliers.getAggregations().get("industryType_count");
        Terms levelNameTerm = suppliers.getAggregations().get("levelName_count");
        return InfoResponseDto.builder()
                .provinceMap(getStatisticsInfo(provinceTerm))
                .custInfoDtoList(suppliers.getContent())
                .totleNum(suppliers.getTotalElements())
                .build();
    }
	
	
	private Map<String, Long> getStatisticsInfo(StringTerms teamAgg){
        Map<String, Long> map = new LinkedHashMap<>();
        List<StringTerms.Bucket> bucketList = teamAgg.getBuckets();
        for(StringTerms.Bucket bucket : bucketList) {
            if (null != bucket.getKey() &&  StringUtils.isNotBlank(bucket.getKey().toString()) ) {
                if (!"null".equalsIgnoreCase(bucket.getKey().toString())) {
                    map.put(bucket.getKey().toString(), bucket.getDocCount());
                }
            }
        }
        return map;
    }

分页查询信息（深分页）

public CustInfoResponseDto findCustInfoPageAndStatisticsBigNum(SearchCustInfoDto custinfoSearch) {
        // 查询条件
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        if (StringUtils.isNotBlank(custinfoSearch.getKeyword())) {
            boolQuery.must(QueryBuilders.matchQuery("custName", custinfoSearch.getKeyword()));
        } else {
            boolQuery.must(QueryBuilders.matchAllQuery());
        }
        if (StringUtils.isNotBlank(custinfoSearch.getProvince())) {
            boolQuery.must(QueryBuilders.matchPhraseQuery("province", custinfoSearch.getProvince()));
        }
        NativeSearchQuery nativeSearchQuery = new NativeSearchQueryBuilder()
                .withQuery(boolQuery)
                .addAggregation(AggregationBuilders.terms("province_count").field("province"))
                .withPageable(PageRequest.of(0, custinfoSearch.getPageSize()))
                .withSort(SortBuilders.scoreSort())
                .withHighlightFields(buildHighlightField())
                .build();
        // 执行语句获取聚合结果
        Aggregations aggregations = elasticsearchTemplate.query(nativeSearchQuery, new ResultsExtractor<Aggregations>() {
            @Override
            public Aggregations extract(SearchResponse response) {
                return response.getAggregations();
            }
        });
        // 封装数据
        SearchResultMapper searchResultMapper = new SearchResultMapper() {
            @Override
            public <T> AggregatedPage<T> mapResults(SearchResponse response, Class<T> clazz, Pageable pageable) {
                SearchHits searchHits = response.getHits();
                Long totalHis = searchHits.getTotalHits() >= TEN_THOUSAND ? TEN_THOUSAND : searchHits.getTotalHits();
                SearchHit[] hits = searchHits.getHits();
                ArrayList<CustInfoItem> suppliers = new ArrayList<>();
                CustInfoItem custInfoItem = null;
                for (SearchHit hit : hits) {
                    Map<String, Object> sourceAsMap = hit.getSourceAsMap();
                    custInfoItem = CustInfoItem.builder()
                            .custId(convertString(sourceAsMap, "custId"))
                            .custName(convertString(sourceAsMap, "custName"))
                            .build();
                    custInfoItem.setTotalHis(totalHis);
                    custInfoItem.setItemScore(hit.getScore());
                    // 处理高亮
                    Map<String, HighlightField> highlightFields = hit.getHighlightFields();
                    if (highlightFields.get("custName") != null) {
                        String nameHighlight = highlightFields.get("custName").getFragments()[0].toString();
                        custInfoItem.setCustName(nameHighlight);
                    }
                    try {
                        LOGGER.info("========》{}", MAPPER.writeValueAsString(custInfoItem));
                    } catch (JsonProcessingException e) {
                        LOGGER.info("========》【JSON解析异常】");
                    }
                    suppliers.add(custInfoItem);
                }
                return new AggregatedPageImpl((List<T>) suppliers, pageable,
                        response.getHits().getTotalHits(), response.getAggregations());
            }
        };
        ScrolledPage<CustInfoItem> scroll = (ScrolledPage<CustInfoItem>) this.elasticsearchTemplate.startScroll(
                SCROLL_TIME_IN_MILLIS,
                nativeSearchQuery,
                CustInfoItem.class,
                searchResultMapper);
        for (int i = 0; i < custinfoSearch.getPageNo()-1; i++) {
            if (scroll.hasContent()) {
                scroll = (ScrolledPage<CustInfoItem>) elasticsearchTemplate.continueScroll(scroll.getScrollId(),
                        SCROLL_TIME_IN_MILLIS, CustInfoItem.class, searchResultMapper);
            }
        }
        return CustInfoResponseDto.builder()
                .provinceMap(getStatisticsInfo((StringTerms) aggregations.asMap().get("provinceTerm") ))
                .custInfoDtoList(scroll.getContent())
                .totleNum(scroll.getTotalElements())
                .build();
    }

yk10010

关注

1
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
03-Elasticsearch的 TransportClient

Elasticsearch的 TransportClient1. Java Client 说明java client 使用 TransportClient，各种操作本质上都是异步的(可以用 listener，或返回 Future ）。注意：ES的发展规划中在7.0版本开始将废弃 TransportClient，8.0版本中将完全移除 TransportClient，取而代之的是High Le...
复制链接

扫一扫