es中springboot的两种连接方式

最新推荐文章于 2024-09-18 11:14:16 发布

名明鸣冥

最新推荐文章于 2024-09-18 11:14:16 发布

阅读量1.6w

点赞数 2

分类专栏： elasticsearch 文章标签： elasticsearch transport

本文链接：https://blog.csdn.net/mingover/article/details/82941506

版权

elasticsearch 专栏收录该内容

6 篇文章 0 订阅

订阅专栏

使用客户端调用es服务的两种方式

有两种方式,一种是9200端口或叫rest 接口，
另一种是用节点的9300端口或叫 TransportClient

说明

es的官网中https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-api.html
We plan on deprecating the TransportClient in Elasticsearch 7.0 and removing it completely in 8.0. Instead, you should be using the Java High Level REST Client, which executes HTTP requests rather than serialized Java requests. The migration guidedescribes all the steps needed to migrate.
The Java High Level REST Client currently has support for the more commonly used APIs, but there are a lot more that still need to be added. You can help us prioritise by telling us which missing APIs you need for your application by adding a comment to this issue: Java high-level REST client completeness.
Any missing APIs can always be implemented today by using the low level Java REST Client with JSON request and response bodies.

简单点说就是 TransportClient 在es7后不支持了,所以你要用 java high level rest client，这个是用http的请求)
所以，后面都是采用9200端口的方式！！！

官网中会说到low-level REST client f High Level REST Client,这两个震得了是rest方式,替换 transportclient方式的!!!

aws 连elasticsearch 的坑

项目中是使用 transport来调用es的，但是在aws中，这种方式是不支持的!!!,要用9200的方式
具体情况:
https://forums.aws.amazon.com/thread.jspa?messageID=683536
在这里插入图片描述

不过这个还好，官网也说了，之后高版本会放弃这个,所以早做改变也是OK的.
org.springframework.data.elasticsearch.core中的ElasticsearchOperations 是一个接口，可以多种实现!

在这里插入图片描述

使用springframework.data的情况(Transport), 不建议使用，请查看下一章.

怎么连es，使用第一种方式

这边是使用spring boot的方式

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>

spring对es地址及相关配置.
spring.data.elasticsearch.cluster-nodes=xxxx:9300,xxx:9301
spring.data.elasticsearch.cluster-name=b_cluster

怎么使用

直接引用即可

聚合,

//聚合操作
TermsAggregationBuilder tbuild = AggregationBuilders.terms("sa_groupid").field("said");
AggregationBuilder sab = AggregationBuilders.stats("a_stat").field("player_count");
tbuild.subAggregation(sab);
tbuild.size(saidlist.size());
SearchQuery searchQuery = new NativeSearchQueryBuilder()
        .withQuery(boolQueryBuilder).withIndices("aaa_online_log").withTypes("doc").withSearchType(SearchType.DEFAULT)
        .addAggregation(tbuild)
        .build();

Aggregations aggregations = elasticsearchTemplate.query(searchQuery, new ResultsExtractor<Aggregations>() {
    @Override
    public Aggregations extract(SearchResponse response) {
        return response.getAggregations();
    }
});

//聚合的内容获取!
MultiBucketsAggregation targetAggregation = aggregations.get("sa_groupid");
Map<Integer, DAggOnlineStat> sa_stat = new HashMap<>();
targetAggregation.getBuckets().forEach(tmpbuck ->{
    DAggOnlineStat dAggOnlineStat = new DAggOnlineStat();
    dAggOnlineStat.setApp_id(((Long)tmpbuck.getKey()).intValue());
    dAggOnlineStat.setDoc_count((int) tmpbuck.getDocCount());

    Aggregation tmpa = tmpbuck.getAggregations().getAsMap().get("a_stat");
    Stats tmpstats = (Stats) tmpa;

    dAggOnlineStat.setAvg(tmpstats.getAvg());
    dAggOnlineStat.setMax(((Double)tmpstats.getMax()).intValue());
    dAggOnlineStat.setMin(((Double)tmpstats.getMin()).intValue());
    sa_stat.put(dAggOnlineStat.getApp_id(), dAggOnlineStat);
});
return sa_stat;

查询单表

        String indexName = String.format("a_ooo_price_%s_log", currency_flag);

        BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
        boolQueryBuilder.must(new TermQueryBuilder("appid", appid));

        //按时间从早到晚排序
        SortBuilder sortBuilder = new FieldSortBuilder("current_time").order(SortOrder.ASC);

        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withQuery(boolQueryBuilder).withIndices(indexName).withTypes("doc").withSort(sortBuilder)
                .build();

        SearchHits searchHits = elasticsearchTemplate.query(searchQuery, new ResultsExtractor<SearchHits>() {
            @Override
            public SearchHits extract(SearchResponse response) {
                return response.getHits();
            }
        });

        List<sfoogamePriceLog> sfoogamePriceLogs = new ArrayList<>();
        for (SearchHit searchHit : searchHits) {
            Map<String, Object> hitMap = searchHit.getSource();
            sfoogamePriceLog sfoogamePriceLog = new sfoogamePriceLog();
            sfoogamePriceLog.setId(searchHit.getId());
            sfoogamePriceLog.setDiscount_percent((Integer) hitMap.get("discount_percent"));
            sfoogamePriceLog.setUpdated((Long) hitMap.get("updated"));
            sfoogamePriceLog.setCdest((Integer) hitMap.get("cdest"));
            sfoogamePriceLog.setInitial((Integer) hitMap.get("initial"));
            sfoogamePriceLog.setCurrent_time((Long) hitMap.get("current_time"));
            sfoogamePriceLog.setCdate((String) hitMap.get("cdate"));
            sfoogamePriceLog.setAppid((Integer) hitMap.get("appid"));
            sfoogamePriceLog.setCurrency_flag((String) hitMap.get("currency_flag"));
            sfoogamePriceLogs.add(sfoogamePriceLog);
        }
        return sfoogamePriceLogs;

查询percent

PercentilesAggregationBuilder tbuild = AggregationBuilders.percentiles("b_sta").field("score");

BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
boolQueryBuilder.must(new TermQueryBuilder("language", "english"));

SearchQuery searchQuery = new NativeSearchQueryBuilder()
        .withQuery(boolQueryBuilder).withIndices("sfoogame_detail").withTypes("doc").withSearchType(SearchType.DEFAULT)
        .addAggregation(tbuild)
        .build();

Aggregations aggregations = elasticsearchTemplate.query(searchQuery, new ResultsExtractor<Aggregations>() {
    @Override
    public Aggregations extract(SearchResponse response) {
        return response.getAggregations();
    }
});

InternalTDigestPercentiles tmp = aggregations.get("b_sta");
Double percentile_val = tmp.percentile(50);
return percentile_val;

怎么连jest与使用,即9200端口.

官网:
https://github.com/VanRoy/spring-data-jest

com.github.vanroy spring-boot-starter-data-jest 3.1.5.RELEASE

官网使用的是xml的配置

    <!-- ElasticSearch Jest Client -->
    <bean id="jestClient" factory-bean="jestClientFactory" factory-method="getObject" destroy-method="shutdownClient" />

    <bean id="jestClientConfigBuilder" class="io.searchbox.client.config.HttpClientConfig.Builder">
        <constructor-arg type="java.lang.String" value="http://localhost:9200"/>
    </bean>

    <bean id="jestClientConfig" factory-bean="jestClientConfigBuilder" factory-method="build"/>

    <bean id="jestClientFactory" class="io.searchbox.client.JestClientFactory">
        <property name="httpClientConfig" ref="jestClientConfig"/>
    </bean>

    <!-- Elasticsearch Jest Template -->
    <bean id="jestElasticsearchTemplate" class="com.github.vanroy.springdata.jest.JestElasticsearchTemplate">
        <constructor-arg ref="jestClient"/>
    </bean>

由于我们是使用springboot的,上面的配置可以不用，
我们使用注解的方式进行配置

    @Value("${project.es.serveruri}")
    public String serverUri ;

    @Bean
    public io.searchbox.client.config.HttpClientConfig.Builder jestClientConfigBuilder() {
        io.searchbox.client.config.HttpClientConfig.Builder builder = new io.searchbox.client.config.HttpClientConfig.Builder(serverUri);
        return builder;
    }
    @Bean
    public HttpClientConfig jestClientConfig(@Qualifier("jestClientConfigBuilder") io.searchbox.client.config.HttpClientConfig.Builder builder) {
        return builder.build();
    }

    @Bean
    public io.searchbox.client.JestClientFactory jestClientFactory(@Qualifier("jestClientConfig") HttpClientConfig httpClientConfig){
        io.searchbox.client.JestClientFactory factory = new io.searchbox.client.JestClientFactory();
        factory.setHttpClientConfig(httpClientConfig);
        return factory;
    }

    @Bean
    public JestClient jestClient(@Qualifier("jestClientFactory")JestClientFactory jestClientFactory) {
        return jestClientFactory.getObject();
    }


    @Bean
    public com.github.vanroy.springdata.jest.JestElasticsearchTemplate jestElasticsearchTemplate(@Qualifier("jestClient")JestClient jestClient){
        JestElasticsearchTemplate jestElasticsearchTemplate = new JestElasticsearchTemplate(jestClient);
        return jestElasticsearchTemplate;
    }

jest怎么使用?

@Autowired
private JestElasticsearchTemplate elasticsearchTemplate;

直接使用是OK的，不过我找了很久都有发现有靠谱的文档，对于使用template的资料很少，只能自己摸索,

使用template查单表数据

        String indexName = "aaa_index"

        BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
        boolQueryBuilder.must(new TermQueryBuilder("aid", aid));

        //按时间从早到晚排序
        SortBuilder sortBuilder = new FieldSortBuilder("current_time").order(SortOrder.ASC);

        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withQuery(boolQueryBuilder).withIndices(indexName).withTypes("doc").withSort(sortBuilder)
                .build();

        List<SearchResult.Hit<AAA, Void>> searchHits = elasticsearchTemplate.query(searchQuery, new JestResultsExtractor<List<SearchResult.Hit<AAA, Void>>>() {
            @Override
            public List<SearchResult.Hit<AAA, Void>> extract(SearchResult searchResult) {
                return  searchResult.getHits(AAA.class);
            }
        });

        List<AAA> aaas = new ArrayList<>();
        for (SearchResult.Hit<AAA, Void> searchHit : searchHits) {
            AAA aaa = searchHit.source;

        }

AAA.class是一个自定义的pojo类,用来装内容的.

查max

        BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
        TermsQueryBuilder queryBuilder1 = new TermsQueryBuilder("aid",String.valueOf(aid));

        boolQueryBuilder.must(queryBuilder1);

        MaxAggregationBuilder tbuild = AggregationBuilders.max("count_max").field("player_count");


        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withQuery(boolQueryBuilder).withIndices("project_online_peak").withTypes("doc").withSearchType(SearchType.DEFAULT)
                .addAggregation(tbuild)
                .build();

        MetricAggregation aggregations = elasticsearchTemplate.query(searchQuery, new JestResultsExtractor<MetricAggregation>() {
            @Override
            public MetricAggregation extract(SearchResult searchResult) {
                return  searchResult.getAggregations();
            }
        });


        Double count_max_double = null;
        try {
//            count_max_double = ((InternalMax)aggregations.get("count_max")).value();
            count_max_double = aggregations.getMaxAggregation("count_max").getMax();
        } catch (Exception e) {
            log.error("count_max获取失败", e);
        }
        return  count_max_double.intValue();

查percentiles数据

PercentilesAggregationBuilder tbuild = AggregationBuilders.percentiles("b_sta").field("score");

BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
boolQueryBuilder.must(new TermQueryBuilder("language", "english"));

SearchQuery searchQuery = new NativeSearchQueryBuilder()
        .withQuery(boolQueryBuilder).withIndices("sfoogame_detail").withTypes("doc").withSearchType(SearchType.DEFAULT)
        .addAggregation(tbuild)
        .build();

MetricAggregation aggregations = elasticsearchTemplate.query(searchQuery, new JestResultsExtractor<MetricAggregation>() {
    @Override
    public MetricAggregation extract(SearchResult searchResult) {
        return  searchResult.getAggregations();
    }
});


//        InternalTDigestPercentiles tmp = aggregations.get("b_sta");
Map<String, Double> tmp = aggregations.getPercentilesAggregation("b_sta").getPercentiles();

Double targetVal = null;
for (String percentilestr : tmp.keySet()) {
    Double tdouble = new Double(percentilestr);
    if(75 == tdouble.intValue()){ // %50以上
        targetVal = tmp.get(percentilestr);
    }
}

terms聚合

QueryBuilder queryBuilder = QueryBuilders.boolQuery()
        .must(QueryBuilders.termQuery("cdatestr", "20170912"));
//聚合操作
TermsAggregationBuilder tbuild = AggregationBuilders.terms("sa_groupid").field("appid");
AggregationBuilder sab = AggregationBuilders.stats("a_stat").field("player_count");
tbuild.subAggregation(sab);
SearchQuery searchQuery = new NativeSearchQueryBuilder()
        .withQuery(queryBuilder).withIndices("sa_online_log").withTypes("doc").withSearchType(SearchType.DEFAULT)
        .addAggregation(tbuild)
        .build();



MetricAggregation aggregations = elasticsearchTemplate.query(searchQuery, new JestResultsExtractor<MetricAggregation>() {
    @Override
    public MetricAggregation extract(SearchResult response) {
        return  response.getAggregations();
    }
});

//聚合的内容获取!
List<TermsAggregation.Entry> tmpList = aggregations.getTermsAggregation("sa_groupid").getBuckets();
Map<Integer, DAggOnlineStat> sa_stat = new HashMap<>();
for (TermsAggregation.Entry entry : tmpList) {
    System.out.println(entry);
    DAggOnlineStat dAggOnStat = new DAggOnlineStat();
    dAggOnStat.setApp_id(Integer.valueOf(entry.getKey()));
    dAggOnStat.setDoc_count(entry.getCount().intValue());
    StatsAggregation statsAggregation = entry.getStatsAggregation("a_stat");
    dAggOnStat.setAvg(statsAggregation.getAvg());
    dAggOnStat.setMax(statsAggregation.getMax().intValue());
    dAggOnStat.setMin(statsAggregation.getMin().intValue());
    sa_stat.put(dAggOnStat.getApp_id(), dAggOnStat);
}
System.out.println("-----------");
System.out.println(sa_stat);

两种的一些使用差异.

在有些方法，会出现unsupportedOpertationException.对这些方法，我们要区别去改代码…【竟然不是在接口层去封装而是改业务代码，真是诡异】,
调这些方法会抛错

java.lang.UnsupportedOperationException
    at com.github.vanroy.springdata.jest.JestElasticsearchTemplate.query(JestElasticsearchTemplate.java:413)

现在来看，换个方法即可

参考:

https://help.aliyun.com/document_detail/69194.html?spm=a2c4g.11186623.6.552.45263075k2X9j5

如果要连aws，可参考:
https://blog.csdn.net/mingover/article/details/82984579