es 聚合统计

已于 2022-01-20 10:52:27 修改
阅读量2.9k 收藏
点赞数
分类专栏： 笔记 文章标签： elasticsearch 大数据 big data
于 2022-01-05 10:49:07 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/he_yanghui/article/details/122316779
版权

最新项目做ES 统计 就分享一下：
直接上代码
修改完善了下，1.统计数据只取10条分页问题，数据查询只取10000条问题
需要将es 修改下：

da/_settings/put
{"max_result_window":200000000}

	// 高级检索，根据字段来搜索
	@Override
	public Page<DaDab> esByParam(EsSearchDto esSearchDto) {

		String index = esSearchDto.getIndex();
		List<Map<String, Object>> list = new ArrayList<>();
		Page<DaDab> DaDabPage = new Page<>();
		try {
			SearchRequest request = new SearchRequest();
			SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
			request.indices(index);

			List<Integer> mlids = esSearchDto.getMlids();
			//保存期限
			String bcqx = esSearchDto.getBcqx();
			//创建时间
			String cjsj = esSearchDto.getCjsj();
			//部门
			Integer deptId = esSearchDto.getDeptId();
			//档案密级
			String damj = esSearchDto.getDamj();
			//归档时间
			String gdsj = esSearchDto.getGdsj();

			List<String> dabhs = esSearchDto.getDabhs();

			// 多字段模糊条件
			BoolQueryBuilder boolQueryBuilder = boolQuery();
			// 必须条件 精准查询
			TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("sfsc", 0);
			boolQueryBuilder.must(termQueryBuilder);

			//保存期限
			if (StringUtils.isNotBlank(bcqx)) {
				boolQueryBuilder.must(QueryBuilders.termQuery("bcqx.keyword", bcqx));
			}
			// cjsj  （2021-01-01）
			if (StringUtils.isNotBlank(cjsj)) {
				String from = cjsj + " 00:00:00";
				String to = cjsj + " 23:59:59";
				RangeQueryBuilder rangequerybuilder = QueryBuilders
						//传入时间，目标格式2020-01-02T03:17:37.638Z
						.rangeQuery("cjsj.keyword").gte(from).lte(to);
				boolQueryBuilder.must(rangequerybuilder);
			}
			// deptId
			if (deptId != null) {
				boolQueryBuilder.must(QueryBuilders.termQuery("deptId", deptId));
			}
			// damj
			if (StringUtils.isNotBlank(damj)) {
				boolQueryBuilder.must(QueryBuilders.termQuery("damj.keyword", damj));
			}
			// gdsj 因为归档时间存的是 long 格式 所以需要转化
			if (StringUtils.isNotBlank(gdsj)) {
//				long time = DateUtil.parse(gdsj).getTime();
//				boolQueryBuilder.must(QueryBuilders.termQuery("gdsj", time));

				String from = gdsj + " 00:00:00";
				String to = gdsj + " 23:59:59";
				long ftime = DateUtil.parse(from).getTime();
				long ttime = DateUtil.parse(to).getTime();
				RangeQueryBuilder rangequerybuilder = QueryBuilders
						//传入时间，目标格式2020-01-02T03:17:37.638Z
						.rangeQuery("gdsj").gte(ftime).lte(ttime);
				boolQueryBuilder.must(rangequerybuilder);
			}

			if (mlids != null && mlids.size() > 0) {
				BoolQueryBuilder mlboolQueryBuilder = boolQuery();
				// 精致查询
				mlids.forEach(mlid ->{
					mlboolQueryBuilder.should(QueryBuilders.termQuery("mlid.keyword", mlid));
				});
				boolQueryBuilder.must(mlboolQueryBuilder);
			}
			if (dabhs != null && dabhs.size() > 0) {
				BoolQueryBuilder dabhboolQB = boolQuery();
				// 精致查询
				dabhs.forEach(dabh ->{
					dabhboolQB.should(QueryBuilders.termQuery("dabh.keyword", dabh));
				});
				boolQueryBuilder.must(dabhboolQB);
			}
			searchSourceBuilder.query(boolQueryBuilder);
			searchSourceBuilder.timeout(new TimeValue(10, TimeUnit.SECONDS));
			Integer offset = (esSearchDto.getPageNo() - 1) * esSearchDto.getPageSize();
			searchSourceBuilder.from(offset);
			searchSourceBuilder.size(esSearchDto.getPageSize());
			searchSourceBuilder.trackTotalHits(true);
			// 执行搜索
			request.source(searchSourceBuilder);
			// client执行
			SearchResponse search = client.search(request, RequestOptions.DEFAULT);
			SearchHits hits = search.getHits();
			// 获取检索结果总数
			int count = (int) hits.getTotalHits().value;

			DaDabPage.setTotal(count);
			if (count != 0) {
				for (SearchHit hit : hits.getHits()) {
					Map<String, Object> sourceAsMap = hit.getSourceAsMap();
					list.add(sourceAsMap);
				}
			}

		} catch (IOException e) {
			e.printStackTrace();
		}
		DaDabPage.setRecords(formatBean(list));
		return DaDabPage;
	}
	
	@Override
	public List<Map> esTjByBcqxAndMl(EsSearchDto esSearchDto) {
		List<Map> list = new ArrayList<>();
		String index = esSearchDto.getIndex();
		try {
			SearchRequest request = new SearchRequest();
			SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
			request.indices(index);
			// 多字段模糊条件
			BoolQueryBuilder boolQueryBuilder = boolQuery();
			// 必须条件 精准查询
			TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("sfsc", 0);
			boolQueryBuilder.must(termQueryBuilder);
			//开始结束时间
			if (StrUtil.isNotEmpty(esSearchDto.getStartTime()) && StrUtil.isNotEmpty(esSearchDto.getEndTime())) {
				RangeQueryBuilder rangequerybuilder = QueryBuilders
						//传入时间，目标格式2020-01-02T03:17:37.638Z
						.rangeQuery("cjsj")
						.from(esSearchDto.getStartTime()).to(esSearchDto.getEndTime());
				boolQueryBuilder.must(rangequerybuilder);
			}
			//保存期限
			if (StringUtils.isNotBlank(esSearchDto.getBcqx())) {
				// 精准查询
				TermQueryBuilder termQueryBuilder2 = QueryBuilders.termQuery("bcqx", esSearchDto.getBcqx());
				boolQueryBuilder.must(termQueryBuilder2);
			}
			searchSourceBuilder.query(boolQueryBuilder);
			searchSourceBuilder.timeout(new TimeValue(10, TimeUnit.SECONDS));
			// 解决统计分页问题，取100条 可以设置更大
			TermsAggregationBuilder termsBuilder2 = AggregationBuilders.terms("bcqxsum").field("bcqxmc.keyword").size(100);
			TermsAggregationBuilder termsBuilder1 = AggregationBuilders.terms("mlsum").field("mlmc.keyword").size(100);
			termsBuilder2.subAggregation(termsBuilder1);

			searchSourceBuilder.aggregation(termsBuilder2);
			// 执行搜索
			request.source(searchSourceBuilder);

			// client执行
			SearchResponse response = client.search(request, RequestOptions.DEFAULT);

			// 获取统计后数据
			Terms terms = response.getAggregations().get("bcqxsum");
			for (Terms.Bucket buck : terms.getBuckets()) {
				String bcqxmc = buck.getKeyAsString();
				Terms mlsum = buck.getAggregations().get("mlsum");
				for (Terms.Bucket mlbuck : mlsum.getBuckets()) {
					HashMap<String, Object> map = new HashMap<>(16);
					String mlmc = mlbuck.getKeyAsString();
					long num = mlbuck.getDocCount();
					map.put("bcqxmc", bcqxmc);
					map.put("mlmc", mlmc);
					map.put("num", num);
					list.add(map);
				}
			}
		} catch (IOException e) {
			e.printStackTrace();
		}
		return list;
	}

	@Override
	public List<Map> esDaTjByGdsjAndMl(EsSearchDto esSearchDto) {
		List<Map> list = new ArrayList<>();
		String index = esSearchDto.getIndex();
		try {
			SearchRequest request = new SearchRequest();
			SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
			request.indices(index);
			// 多字段模糊条件
			BoolQueryBuilder boolQueryBuilder = boolQuery();
			// 必须条件 精准查询
			TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("sfsc", 0);
			boolQueryBuilder.must(termQueryBuilder);
			//开始结束时间
			if (StrUtil.isNotEmpty(esSearchDto.getStartTime()) && StrUtil.isNotEmpty(esSearchDto.getEndTime())) {
				RangeQueryBuilder rangequerybuilder = QueryBuilders
						//传入时间，目标格式2020-01-02T03:17:37.638Z
						.rangeQuery("cjsj")
						.from(esSearchDto.getStartTime()).to(esSearchDto.getEndTime());
				boolQueryBuilder.must(rangequerybuilder);
			}

			searchSourceBuilder.query(boolQueryBuilder);
			searchSourceBuilder.timeout(new TimeValue(10, TimeUnit.SECONDS));

			DateHistogramAggregationBuilder termsBuilder2 = AggregationBuilders.dateHistogram("gdsjsum")
					.field("gdsj").dateHistogramInterval(DateHistogramInterval.YEAR);
			//termsBuilder2.format("yyyy");
			TermsAggregationBuilder termsBuilder1 = AggregationBuilders.terms("mlsum").field("mlmc.keyword").size(100);
			termsBuilder2.subAggregation(termsBuilder1);

			searchSourceBuilder.aggregation(termsBuilder2);
			// 执行搜索
			request.source(searchSourceBuilder);

			// client执行
			SearchResponse response = client.search(request, RequestOptions.DEFAULT);

			// 获取统计后数据
			Histogram terms = response.getAggregations().get("gdsjsum");
			for (Histogram.Bucket buck : terms.getBuckets()) {
				String gdsj = buck.getKeyAsString();
				DateTime time = DateUtil.date(new Long(gdsj));
				Terms mlsum = buck.getAggregations().get("mlsum");
				for (Terms.Bucket mlbuck : mlsum.getBuckets()) {
					HashMap<String, Object> map = new HashMap<>(16);
					String mlmc = mlbuck.getKeyAsString();
					long num = mlbuck.getDocCount();
					map.put("year", time.year());
					map.put("mlmc", mlmc);
					map.put("num", num);
					list.add(map);
				}
			}
		} catch (IOException e) {
			e.printStackTrace();
		}
		return list;
	}

	@Override
	public List<Map> esDaTjByGdsj(EsSearchDto esSearchDto) {
		List<Map> list = new ArrayList<>();
		String index = esSearchDto.getIndex();
		try {
			SearchRequest request = new SearchRequest();
			SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
			request.indices(index);
			// 多字段模糊条件
			BoolQueryBuilder boolQueryBuilder = boolQuery();
			// 必须条件 精准查询
			TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("sfsc", 0);
			boolQueryBuilder.must(termQueryBuilder);
			//开始结束时间
			if (StrUtil.isNotEmpty(esSearchDto.getStartTime()) && StrUtil.isNotEmpty(esSearchDto.getEndTime())) {
				RangeQueryBuilder rangequerybuilder = QueryBuilders
						//传入时间，目标格式2020-01-02T03:17:37.638Z
						.rangeQuery("cjsj")
						.from(esSearchDto.getStartTime()).to(esSearchDto.getEndTime());
				boolQueryBuilder.must(rangequerybuilder);
			}

			searchSourceBuilder.query(boolQueryBuilder);
			searchSourceBuilder.timeout(new TimeValue(10, TimeUnit.SECONDS));

			DateHistogramAggregationBuilder termsBuilder2 = AggregationBuilders.dateHistogram("gdsjsum")
					.field("gdsj").dateHistogramInterval(DateHistogramInterval.YEAR);

			searchSourceBuilder.aggregation(termsBuilder2);
			// 执行搜索
			request.source(searchSourceBuilder);

			// client执行
			SearchResponse response = client.search(request, RequestOptions.DEFAULT);

			// 获取统计后数据
			Histogram terms = (Histogram) response.getAggregations().get("gdsjsum");
			for (Histogram.Bucket buck : terms.getBuckets()) {
				String gdsj = buck.getKeyAsString();
				DateTime time = DateUtil.date(new Long(gdsj));
				HashMap<String, Object> map = new HashMap<>(16);
				long num = buck.getDocCount();
				map.put("year", time.year());
				map.put("num", num);
				list.add(map);
			}
		} catch (IOException e) {
			e.printStackTrace();
		}
		return list;
	}

	@Override
	public List<Map> esYwgsTjByMl(EsSearchDto esSearchDto) {
		List<Map> list = new ArrayList<>();
		String index = esSearchDto.getIndex();
		try {
			SearchRequest request = new SearchRequest();
			SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
			request.indices(index);
			// 多字段模糊条件
			BoolQueryBuilder boolQueryBuilder = boolQuery();
			// 必须条件 精准查询
			TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("sfsc", 0);
			boolQueryBuilder.must(termQueryBuilder);
			//开始结束时间
			if (StrUtil.isNotEmpty(esSearchDto.getStartTime()) && StrUtil.isNotEmpty(esSearchDto.getEndTime())) {
				RangeQueryBuilder rangequerybuilder = QueryBuilders
						//传入时间，目标格式2020-01-02T03:17:37.638Z
						.rangeQuery("cjsj")
						.from(esSearchDto.getStartTime()).to(esSearchDto.getEndTime());
				boolQueryBuilder.must(rangequerybuilder);
			}
			//保存期限
			if (StringUtils.isNotBlank(esSearchDto.getYwgs())) {
				// 精准查询
				TermQueryBuilder termQueryBuilder2 = QueryBuilders.termQuery("ywgs", esSearchDto.getYwgs());
				boolQueryBuilder.must(termQueryBuilder2);
			}
			searchSourceBuilder.query(boolQueryBuilder);
			searchSourceBuilder.timeout(new TimeValue(10, TimeUnit.SECONDS));

			TermsAggregationBuilder termsBuilder2 = AggregationBuilders.terms("ywgssum").field("ywgs.keyword").size(100);
			TermsAggregationBuilder termsBuilder1 = AggregationBuilders.terms("mlsum").field("mlmc.keyword").size(100);
			termsBuilder2.subAggregation(termsBuilder1);

			searchSourceBuilder.aggregation(termsBuilder2);
			// 执行搜索
			request.source(searchSourceBuilder);

			// client执行
			SearchResponse response = client.search(request, RequestOptions.DEFAULT);

			// 获取统计后数据
			Terms terms = response.getAggregations().get("ywgssum");
			for (Terms.Bucket buck : terms.getBuckets()) {
				String ywgs = buck.getKeyAsString();
				Terms mlsum = buck.getAggregations().get("mlsum");
				for (Terms.Bucket mlbuck : mlsum.getBuckets()) {
					HashMap<String, Object> map = new HashMap<>(16);
					String mlmc = mlbuck.getKeyAsString();
					long num = mlbuck.getDocCount();
					map.put("ywgs", ywgs);
					map.put("mlmc", mlmc);
					map.put("num", num);
					list.add(map);
				}
			}
		} catch (IOException e) {
			e.printStackTrace();
		}
		return list;
	}

	@Override
	public List<Map> esdagdTjByMl(EsSearchDto esSearchDto) {
		List<Map> list = new ArrayList<>();
		String index = esSearchDto.getIndex();
		try {
			SearchRequest request = new SearchRequest();
			SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
			request.indices(index);
			// 多字段模糊条件
			BoolQueryBuilder boolQueryBuilder = boolQuery();
			// 必须条件 精准查询
			TermQueryBuilder termQueryBuilder1 = QueryBuilders.termQuery("sfsc", 0);
			TermQueryBuilder termQueryBuilder2 = QueryBuilders.termQuery("dazt", 10);
			boolQueryBuilder.must(termQueryBuilder1);
			boolQueryBuilder.must(termQueryBuilder2);
			//开始结束时间
			if (StrUtil.isNotEmpty(esSearchDto.getStartTime()) && StrUtil.isNotEmpty(esSearchDto.getEndTime())) {
				RangeQueryBuilder rangequerybuilder = QueryBuilders
						//传入时间，目标格式2020-01-02T03:17:37.638Z
						.rangeQuery("cjsj")
						.from(esSearchDto.getStartTime()).to(esSearchDto.getEndTime());
				boolQueryBuilder.must(rangequerybuilder);
			}

			TermsAggregationBuilder termsBuilder = AggregationBuilders.terms("sum").field("mlmc.keyword").size(100);
			searchSourceBuilder.query(boolQueryBuilder);
			searchSourceBuilder.timeout(new TimeValue(10, TimeUnit.SECONDS));
			searchSourceBuilder.aggregation(termsBuilder);
			// 执行搜索
			request.source(searchSourceBuilder);
			// client执行
			SearchResponse response = client.search(request, RequestOptions.DEFAULT);
			// 获取统计后数据
			Terms terms = response.getAggregations().get("sum");

			for (int i = 0; i < terms.getBuckets().size(); i++) {
				HashMap<String, Object> map = new HashMap<>(16);
				// mlmc
				String mlmc = terms.getBuckets().get(i).getKey().toString();
				//数量
				Long sum = terms.getBuckets().get(i).getDocCount();
				map.put("mlmc", mlmc);
				map.put("num", sum.intValue());
				list.add(map);
			}
		} catch (IOException e) {
			e.printStackTrace();
		}
		return list;
	}

	@Override
	public List<Map> esDaTjByCjrAndMl(EsSearchDto esSearchDto) {
		List<Map> list = new ArrayList<>();
		String index = esSearchDto.getIndex();
		try {
			SearchRequest request = new SearchRequest();
			SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
			request.indices(index);
			// 多字段模糊条件
			BoolQueryBuilder boolQueryBuilder = boolQuery();
			// 必须条件 精准查询
			TermQueryBuilder termQueryBuilder1 = QueryBuilders.termQuery("sfsc", 0);
			boolQueryBuilder.must(termQueryBuilder1);
			//开始结束时间
			if (StrUtil.isNotEmpty(esSearchDto.getStartTime()) && StrUtil.isNotEmpty(esSearchDto.getEndTime())) {
				RangeQueryBuilder rangequerybuilder = QueryBuilders
						//传入时间，目标格式2020-01-02T03:17:37.638Z
						.rangeQuery("cjsj")
						.from(esSearchDto.getStartTime()).to(esSearchDto.getEndTime());
				boolQueryBuilder.must(rangequerybuilder);
			}
			if (StringUtils.isNotBlank(esSearchDto.getCjrmc())) {
				boolQueryBuilder.must(QueryBuilders.termQuery("cjrmc.keyword", esSearchDto.getCjrmc()));
			}
			TermsAggregationBuilder termsBuilder2 = AggregationBuilders.terms("cjrsum").field("cjrmc.keyword").size(999);
			TermsAggregationBuilder termsBuilder1 = AggregationBuilders.terms("mlsum").field("mlmc.keyword").size(999);
			termsBuilder2.subAggregation(termsBuilder1);

			searchSourceBuilder.query(boolQueryBuilder);
			searchSourceBuilder.aggregation(termsBuilder2);
			// 执行搜索
			request.source(searchSourceBuilder);

			// client执行
			SearchResponse response = client.search(request, RequestOptions.DEFAULT);

			// 获取统计后数据
			Terms terms = response.getAggregations().get("cjrsum");
			for (Terms.Bucket buck : terms.getBuckets()) {
				String cjr = buck.getKeyAsString();
				Terms mlsum = buck.getAggregations().get("mlsum");
				for (Terms.Bucket mlbuck : mlsum.getBuckets()) {
					HashMap<String, Object> map = new HashMap<>(16);
					String mlmc = mlbuck.getKeyAsString();
					long num = mlbuck.getDocCount();
					map.put("cjr", cjr);
					map.put("mlmc", mlmc);
					map.put("num", num);
					list.add(map);
				}
			}
		} catch (IOException e) {
			e.printStackTrace();
		}
		return list;
	}

	@Override
	public List<Map> esDaTjByCjr(EsSearchDto esSearchDto) {
		List<Map> list = new ArrayList<>();
		String index = esSearchDto.getIndex();
		try {
			SearchRequest request = new SearchRequest();
			SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
			request.indices(index);
			// 多字段模糊条件
			BoolQueryBuilder boolQueryBuilder = boolQuery();
			// 必须条件 精准查询
			TermQueryBuilder termQueryBuilder1 = QueryBuilders.termQuery("sfsc", 0);
			boolQueryBuilder.must(termQueryBuilder1);
			//开始结束时间
			if (StrUtil.isNotEmpty(esSearchDto.getStartTime()) && StrUtil.isNotEmpty(esSearchDto.getEndTime())) {
				RangeQueryBuilder rangequerybuilder = QueryBuilders
						//传入时间，目标格式2020-01-02T03:17:37.638Z
						.rangeQuery("cjsj")
						.from(esSearchDto.getStartTime()).to(esSearchDto.getEndTime());
				boolQueryBuilder.must(rangequerybuilder);
			}
			TermsAggregationBuilder termsBuilder2 = AggregationBuilders.terms("cjrsum").field("cjrmc.keyword").size(999);
			searchSourceBuilder.query(boolQueryBuilder);
			searchSourceBuilder.aggregation(termsBuilder2);
			// 执行搜索
			request.source(searchSourceBuilder);

			// client执行
			SearchResponse response = client.search(request, RequestOptions.DEFAULT);

			// 获取统计后数据
			Terms terms = response.getAggregations().get("cjrsum");

			for (Terms.Bucket buck : terms.getBuckets()) {
				String cjr = buck.getKeyAsString();
				HashMap<String, Object> map = new HashMap<>(16);
				long num = buck.getDocCount();
				map.put("cjr", cjr);
				map.put("num", num);
				list.add(map);
			}
		} catch (IOException e) {
			e.printStackTrace();
		}
		return list;
	}

	@Override
	public List<String> esDaByZt(String dazt) {
		List<String> list = new ArrayList<>();
		try {
			SearchRequest request = new SearchRequest();
			SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
			request.indices("da");
			// 多字段模糊条件
			BoolQueryBuilder boolQueryBuilder = boolQuery();
			// 必须条件 精准查询
			boolQueryBuilder.must(QueryBuilders.termQuery("sfsc", 0))
					.must(QueryBuilders.termQuery("dazt", Integer.parseInt(dazt)));
			searchSourceBuilder.fetchSource("dabh","");
			searchSourceBuilder.query(boolQueryBuilder);
			searchSourceBuilder.trackTotalHits(true);
			searchSourceBuilder.timeout(new TimeValue(10, TimeUnit.SECONDS));
			// 执行搜索
			request.source(searchSourceBuilder);

			SearchResponse search = client.search(request, RequestOptions.DEFAULT);
			SearchHits hits = search.getHits();
			// 获取检索结果总数
			int count = (int) hits.getTotalHits().value;

			if (count != 0) {
				for (SearchHit hit : hits.getHits()) {
					String sourceAsMap = hit.getSourceAsString();
					list.add(sourceAsMap);
				}
			}
		} catch (IOException e) {
			e.printStackTrace();
		}
		return list;
	}

	@Override
	public List<Map> esDaTjByYwgs(EsSearchDto esSearchDto) {
		List<Map> list = new ArrayList<>();
		String index = esSearchDto.getIndex();
		try {
			SearchRequest request = new SearchRequest();
			SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
			request.indices(index);
			// 多字段模糊条件
			BoolQueryBuilder boolQueryBuilder = boolQuery();
			// 必须条件 精准查询
			TermQueryBuilder termQueryBuilder1 = QueryBuilders.termQuery("sfsc", 0);
			boolQueryBuilder.must(termQueryBuilder1);
			//开始结束时间
			if (StrUtil.isNotEmpty(esSearchDto.getStartTime()) && StrUtil.isNotEmpty(esSearchDto.getEndTime())) {
				RangeQueryBuilder rangequerybuilder = QueryBuilders
						//传入时间，目标格式2020-01-02T03:17:37.638Z
						.rangeQuery("cjsj")
						.from(esSearchDto.getStartTime()).to(esSearchDto.getEndTime());
				boolQueryBuilder.must(rangequerybuilder);
			}

			TermsAggregationBuilder termsBuilder = AggregationBuilders.terms("sum").field("ywgs.keyword").size(100);
			searchSourceBuilder.query(boolQueryBuilder);
			searchSourceBuilder.timeout(new TimeValue(10, TimeUnit.SECONDS));
			searchSourceBuilder.aggregation(termsBuilder);
			// 执行搜索
			request.source(searchSourceBuilder);
			// client执行
			SearchResponse response = client.search(request, RequestOptions.DEFAULT);
			// 获取统计后数据
			Terms terms = response.getAggregations().get("sum");

			for (int i = 0; i < terms.getBuckets().size(); i++) {
				HashMap<String, Object> map = new HashMap<>(16);
				// ywgs
				String ywgs = terms.getBuckets().get(i).getKey().toString();
				//数量
				Long sum = terms.getBuckets().get(i).getDocCount();
				map.put("ywgs", ywgs);
				map.put("num", sum.intValue());
				list.add(map);
			}
		} catch (IOException e) {
			e.printStackTrace();
		}
		return list;
	}

	@Override
	public List<Map> esTjByBcqx(EsSearchDto esSearchDto) {
		List<Map> list = new ArrayList<>();
		String index = esSearchDto.getIndex();
		try {
			SearchRequest request = new SearchRequest();
			SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
			request.indices(index);
			// 多字段模糊条件
			BoolQueryBuilder boolQueryBuilder = boolQuery();
			// 必须条件 精准查询
			TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("sfsc", 0);
			boolQueryBuilder.must(termQueryBuilder);
			//开始结束时间
			if (StrUtil.isNotEmpty(esSearchDto.getStartTime()) && StrUtil.isNotEmpty(esSearchDto.getEndTime())) {
				RangeQueryBuilder rangequerybuilder = QueryBuilders
						//传入时间，目标格式2020-01-02T03:17:37.638Z
						.rangeQuery("cjsj")
						.from(esSearchDto.getStartTime()).to(esSearchDto.getEndTime());
				boolQueryBuilder.must(rangequerybuilder);
			}
			searchSourceBuilder.query(boolQueryBuilder);
			searchSourceBuilder.timeout(new TimeValue(10, TimeUnit.SECONDS));

			TermsAggregationBuilder termsBuilder2 = AggregationBuilders.terms("bcqxsum").field("bcqxmc.keyword").size(100);
			searchSourceBuilder.aggregation(termsBuilder2);
			// 执行搜索
			request.source(searchSourceBuilder);

			// client执行
			SearchResponse response = client.search(request, RequestOptions.DEFAULT);

			// 获取统计后数据
			Terms terms = response.getAggregations().get("bcqxsum");
			for (Terms.Bucket buck : terms.getBuckets()) {
				HashMap<String, Object> map = new HashMap<>(16);
				String bcqxmc = buck.getKeyAsString();
				long num = buck.getDocCount();
				map.put("bcqxmc", bcqxmc);
				map.put("num", num);
				list.add(map);
			}
		} catch (IOException e) {
			e.printStackTrace();
		}
		return list;
	}
мг丶one
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
java使用elasticsearch分组进行聚合查询过程解析
08-25
Java 使用 Elasticsearch 进行分组聚合查询过程解析 Java 使用 Elasticsearch 进行分组聚合查询是一个常见的需求，特别是在大数据处理和数据分析领域。Elasticsearch 提供了强大的聚合功能，可以对数据进行分组、...
ES聚合统计group by,sum,max,min,avg,count等聚合统计
絮落锦乡的博客
02-01 7557
ES聚合统计group by,sum,max,min,avg,count等聚合统计
elastic search中实现聚合去重查询统计
a17432025的博客
04-28 1846
在elastic search中使用json实现查询功能，与sql语言不同，理解起来要复杂一些，尤其是一些嵌套关系时。对于一些特殊查询要求，记录下来供方便使用。
es搜索，统计
最新发布
qq_20182781的博客
07-16 131
记得好几年前用es做过标签画像统计，如今再看es时已是很生疏了，再用时已更新到了7.12版本了。以前用TransportClient客户端，现在出了而且是官方推荐用RestHighLevelClient客户端。这几天用RestHighLevelClient时还是觉得比较方便的。现将一些基本常用功能记录一下。
es查询-统计总数以及深度分页
u011250186的博客
06-27 2万+
es查询-统计总数以及深度分页
ES聚合统计
hdn_kb的博客
12-02 3078
1. 以多个字段唯一并去重后统计总数 2. 求近15添内日平均数据 3. 求近15天内平均数据
ES聚合分组后count排序
liaofucheng123的博客
06-09 1万+
ES通过多个字段分组后count排序. 数据库SQL:select count(1) from table group by storeCode,productId order by count(1) desc; { "query": { "bool": { "must": [ { "exists": { "field": "storeCod
elasticsearch聚合后分页
12-16
方法如果传总页数了，es就不用查询总页数，直接通过开始位置到结束位置取数即可
Elasticsearch聚合 之 Terms
06-19
Elasticsearch聚合之Terms Elasticsearch中的Terms聚合是一种常用的聚合方式，用于对文档中的某个字段进行分组统计。Terms聚合可以对文档中的某个字段进行分组，并统计每个组中的文档数量。 在Elasticsearch中，...
elasticsearch聚合值过滤
03-02
而“elasticsearch聚合值过滤”是ES聚合功能的一个高级应用场景，它允许我们根据聚合计算出的结果来进一步筛选文档，即在聚合后的基础上进行过滤操作。这一特性在数据分析和报表生成时非常有用，能够帮助我们得到更...
Elasticsearch聚合探索：桶聚合与度量聚合的区分与应用
07-02
Elasticsearch的聚合功能允许用户对数据进行汇总和分析，其中桶聚合和度量聚合是两种常用的聚合类型。本文将详细探讨这两种聚合的区别及其应用场景。 桶聚合和度量聚合在Elasticsearch中扮演着不同的角色，它们共同...
Elasticsearch 聚合（aggregation）查询返回所有
热门推荐
baidu_37366055的博客
03-02 3万+
Elasticsearch做查询的时候，很多情况下，有一些查询满足不了我们的要求，这个时候，我们用Elasticsearch 的聚合（aggregation ）函数，下面讲讲aggregation 的使用姿势。 Elasticsearch做聚合查询的时候。 { "from": 0, "size": 10, "aggregations": { "types": { "terms": {...
ES系列之聚合统计不准确问题对应解决方案
u011250186的博客
10-20 2106
ES系列之聚合统计不准确问题对应解决方案
Elasticsearch聚合之sum_other_doc_count问题
qq_35526951的博客
05-11 8602
一、问题分析 1）、doc_count_error_upper_bound：表示没有在这次聚合中返回，但是可能存在的潜在聚合结果。 2）、sum_other_doc_count：表示这次聚合中没有统计到的文档数。这个好理解，因为ES统计的时候默认只会根据count显示排名前十的分桶。如果分类（这里是目的地）比较多，自然会有文档没有被统计到。 二、解决办法 1）、不分片 设置主分片为1，也就是不分片了。这个显而易见，上面分析聚合不精确的核心原因就在于分片，所以不分片肯定可以解决问题。但是缺点也是显然的，只适用
es_聚合用法详解
chuige2013的博客
03-18 6046
ES统计分析概念ES中的聚合查询，类似SQL的SUM/AVG/COUNT/GROUP BY分组查询，主要用于统计分析场景。下面先介绍ES聚合查询的核心流程和核心概念。
es使用和常用查询（包含多字段聚合查询，实体类方式保存es）
鲨墨狐的博客
04-12 1524
es使用和查询
es 多字段聚合 in golang
WZP_LOVE的专栏
11-30 3973
前言 es中如何实现多字段聚合统计？类似于mysql的： select a,b from my_table grroup by a,b; golang如何借助第三方类oliver elastic实现es原生DSL（domain special language 特殊领域查询语言）的构造？ 正文 先来看单字段聚合统计的es原生DSL语句的写法： { "query": { "bool": { "must": [ { "query_stri
elasticsearch多字段聚合实现方式
huan的学习记录
11-15 8772
我们知道在`sql`中是可以实现 `group by 字段a,字段b`，那么这种效果在`elasticsearch`中该如何实现呢？此处我们记录在`elasticsearch`中的3种方式来实现这个效果。
Elasticsearch 基本使用（四）聚合查询
冰糖的博客
06-18 8473
说到聚合查询，马上会想到 SQL 中的 group by，ES中也有类似的功能，名叫 Aggregation。默认，ES的聚合以分组内数量倒序排序。我们基于上面最后的统计结果，自定义聚合结果排序默认排序方式按数量升序之前说过，默认分组，提供了两个字段的返回；key 和 doc_count，如果要自定义这两个基本字段排序方式，需要在前面加上 下划线 _当然还可以按照其他聚合函数的结果排序。
es 排序 聚合统计_elasticsearch聚合统计出现次数
05-13
在 Elasticsearch 中，可以使用聚合（Aggregation）实现对文档进行聚合统计，其中包括出现次数的统计。下面是一个示例： 假设我们有一个名为 "sales" 的索引，包含以下文档： ``` { "product": "A", "price": 10.0, "timestamp": "2021-08-01T10:00:00Z" } { "product": "B", "price": 15.0, "timestamp": "2021-08-01T10:05:00Z" } { "product": "A", "price": 12.0, "timestamp": "2021-08-01T10:10:00Z" } { "product": "C", "price": 20.0, "timestamp": "2021-08-01T10:15:00Z" } { "product": "A", "price": 8.0, "timestamp": "2021-08-01T10:20:00Z" } { "product": "B", "price": 18.0, "timestamp": "2021-08-01T10:25:00Z" } ``` 现在，我们想要统计每个产品出现的次数，可以使用以下聚合查询： ``` { "aggs": { "products": { "terms": { "field": "product" } } } } ``` 其中，"aggs" 是聚合查询的关键字，"products" 是我们给这个聚合起的名字，"terms" 表示我们要按照某个字段进行分组，"field" 指定了我们要按照哪个字段进行分组。 运行上述查询后，得到的结果如下： ``` { "aggregations": { "products": { "buckets": [ { "key": "A", "doc_count": 3 }, { "key": "B", "doc_count": 2 }, { "key": "C", "doc_count": 1 } ] } } } ``` 其中，"key" 表示产品名称，"doc_count" 表示该产品出现的次数。 如果想要对出现次数进行排序，可以使用以下聚合查询： ``` { "aggs": { "products": { "terms": { "field": "product", "order": { "_count": "desc" } } } } } ``` 其中，"order" 表示按照什么字段进行排序，"_count" 表示按照出现次数进行排序，"desc" 表示降序排列。 运行上述查询后，得到的结果如下： ``` { "aggregations": { "products": { "buckets": [ { "key": "A", "doc_count": 3 }, { "key": "B", "doc_count": 2 }, { "key": "C", "doc_count": 1 } ] } } } ``` 其中，产品 A 出现的次数最多，排在第一位。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值