今天遇到一个需求,需要从es导出大量数据,因为es设置了最多导出10W,同时又为了不给es太大的压力,将导出文件分为一天一个,查询方式是一个小时为区间。当然页面输入的时间是一个大范围,内部分为多次查询的话不能出现两次查询出现同样的时间。
例如:2023-11-16 00:00:00~2023-11-18 00:00:00
那么查询次数应该是
第一天
2023-11-16 00:00:00 ~2023-11-16 00:59:59
2023-11-16 01:00:00 ~2023-11-16 01:59:59
2023-11-16 02:00:00 ~2023-11-16 02:59:59
…
2023-11-16 23:00:00 ~2023-11-16 23:59:59
第二天
2023-11-17 00:00:00 ~2023-11-17 00:59:59
2023-11-17 01:00:00 ~2023-11-17 01:59:59
2023-11-17 02:00:00 ~2023-11-17 02:59:59
…
2023-11-17 23:00:00 ~2023-11-17 23:59:59
第三天
2023-11-18 00:00:00 ~2023-11-18 00:00:00
话不多说贴出代码
package com.ultrapower.tomas.alarm.search.utils;
import org.apache.commons.lang.time.DateUtils;
import tomas.services.util.DateUtility;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.*;
public class DateUtil {
private static SimpleDateFormat sfMin00=new SimpleDateFormat("yyyy-MM-dd HH:00:00");
private static SimpleDateFormat sfMin59=new SimpleDateFormat("yyyy-MM-dd HH:59:59");
private static SimpleDateFormat sf00=new SimpleDateFormat("yyyy-MM-dd 00:00:00");
private static SimpleDateFormat sf23=new SimpleDateFormat("yyyy-MM-dd 23:59:59");
public static void main(String[] args) throws ParseException, CloneNotSupportedException, InterruptedException {
String startTime="2023-11-01 12:22:22";
String endTime="2023-11-01 21:33:22";
System.out.println(splitDateByHour(startTime,endTime));
// System.out.println(splitDateByDay(startTime,endTime));;
}
public static List<Map<String,String>> splitDateByHour(String startTime, String endTime){
Date currentDate= DateUtility.parse(startTime);
Date endDate=DateUtility.parse(endTime);
List<Map<String,String>> dataList=new ArrayList<>();
boolean first=true;
while(currentDate.compareTo(endDate)<=0){
Map<String,String> dataMap=new HashMap<>();
dataMap.put("startTime",first?DateUtility.format(currentDate):sfMin00.format(currentDate));
first=false;
dataMap.put("endTime", DateUtils.addHours(currentDate,1).compareTo(endDate)>0?DateUtility.format(endDate):sfMin59.format(currentDate));
dataList.add(dataMap);
currentDate=DateUtils.addHours(currentDate,1);
}
return dataList;
}
public static List<Map<String,String>> splitDateByDay(String startTime,String endTime){
Date currentDate=DateUtility.parse(startTime);
Date endDate=DateUtility.parse(endTime);
List<Map<String,String>> dataList=new ArrayList<>();
boolean first=true;
while(currentDate.compareTo(endDate)<=0){
Map<String,String> dataMap=new HashMap<>();
dataMap.put("startTime",first?DateUtility.format(currentDate):sf00.format(currentDate));
first=false;
dataMap.put("endTime", DateUtils.addDays(currentDate,1).compareTo(endDate)>0?DateUtility.format(endDate):sf23.format(currentDate));
dataList.add(dataMap);
currentDate=DateUtils.addDays(currentDate,1);
}
return dataList;
}
}
注:这里的DateUtility工具是我们项目格式化日期的工具,这里改成自己需要的日期格式化方式。