Java批量数据采用线程池分批同步处理

第一步:申明线程池
public ExecutorService taskExecutor = new ThreadPoolExecutor(6, Integer.MAX_VALUE, 0, TimeUnit.MILLISECONDS, new SynchronousQueue<Runnable>());
第二步:分批处理数据

// 每批次数量
Integer batchCount = 100;Integer start = 0;
// 结尾
List list=new ArrayList<>();
Integer end = 100;
int loopCount = (int) Math.ceil((float) list.size() / batchCount);

// 采用线程池的同步计数类
CountDownLatch countDownLatch = new CountDownLatch(loopCount);
for (int k = 0; k < loopCount; k++) {
    final Integer startTemp = start;
    final Integer endTemp = end;
    // 开启线程池
    taskExecutor.execute(new Runnable() {
        @Override
        public void run() {
            try {
               //todo 书写业务代码逻辑

            } catch (Exception ex) {
                LOGGER.error("getQwAllUserInfo error:", ex);
            } finally {
                countDownLatch.countDown();
            }
        }
    });
    //计算下次分片数据
    start = end;
    end = start + batchCount > list.size() ? list.size() : batchCount + start;
}
最后等所有线程数据处理完毕

countDownLatch.await();
————————————————
原文链接:https://blog.csdn.net/wu_wenjie/article/details/120867357
  • 0
    点赞
  • 5
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
好的,我可以为您提供java自定义线程池分批导出excel的代码,以下是示例代码: ```java import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.TimeUnit; import org.apache.poi.ss.usermodel.CellStyle; import org.apache.poi.ss.usermodel.Font; import org.apache.poi.ss.usermodel.IndexedColors; import org.apache.poi.ss.usermodel.Row; import org.apache.poi.ss.usermodel.Sheet; import org.apache.poi.ss.usermodel.Workbook; import org.apache.poi.xssf.usermodel.XSSFCellStyle; import org.apache.poi.xssf.usermodel.XSSFFont; import org.apache.poi.xssf.usermodel.XSSFWorkbook; public class ExcelExporter { private static final int THREAD_COUNT = 4; private static final int BATCH_SIZE = 2000; private static ExecutorService threadPool = Executors.newFixedThreadPool(THREAD_COUNT); public void exportDataToExcel() throws Exception { Workbook workbook = new XSSFWorkbook(); Sheet sheet = workbook.createSheet("Data"); // Set header row style Font headerFont = workbook.createFont(); headerFont.setBold(true); headerFont.setColor(IndexedColors.WHITE.getIndex()); CellStyle headerCellStyle = workbook.createCellStyle(); headerCellStyle.setFont(headerFont); headerCellStyle.setFillForegroundColor(IndexedColors.BLUE.getIndex()); headerCellStyle.setFillPattern(XSSFCellStyle.SOLID_FOREGROUND); Row headerRow = sheet.createRow(0); headerRow.createCell(0).setCellValue("Column 1"); headerRow.createCell(1).setCellValue("Column 2"); headerRow.createCell(2).setCellValue("Column 3"); for (int i = 0; i < 3; i++) { headerRow.getCell(i).setCellStyle(headerCellStyle); } // Start exporting data for (int i = 0; i < getTotalRowCount(); i += BATCH_SIZE) { int startRow = i + 1; int endRow = Math.min(i + BATCH_SIZE, getTotalRowCount()) + 1; threadPool.execute( new ExportTask( workbook, sheet, startRow, endRow)); } threadPool.shutdown(); threadPool.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS); } private static int getTotalRowCount() { // Return the total number of rows to export return 10000; } private static class ExportTask implements Runnable { private final Workbook workbook; private final Sheet sheet; private final int startRow; private final int endRow; ExportTask( Workbook workbook, Sheet sheet, int startRow, int endRow) { this.workbook = workbook; this.sheet = sheet; this.startRow = startRow; this.endRow = endRow; } @Override public void run() { try { // Simulate data retrieval Thread.sleep(100); // Export data to sheet for (int i = startRow; i < endRow; i++) { Row row = sheet.createRow(i); row.createCell(0).setCellValue("Value " + i); row.createCell(1).setCellValue("Value " + i * 2); row.createCell(2).setCellValue("Value " + i * 3); } } catch (Exception ex) { ex.printStackTrace(); } } } } ``` 该代码中,我们使用了线程池来提高导出数据到excel时的效率。我们先设置一个每批次导出的行数,然后根据总行数和批次大小分批导出,每批次启动一个线程来处理。最后,等待所有线程完成后关闭线程池,导出工作完成。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值