Poi多线程分割

 业务:利用sax将表格解析为xml快速读取表格百万级数据,利用多线程将这批数据分割为以一千为单位的小型的excel,根据表头分割出表名,只取第一张表的所有数据

public static void DataInsertData() throws InterruptedException {

    for (int i = 0; i < stringlist.size(); i++) {
        TableEntity tableEntity = tabledata.get(stringlist.get(i));
        String columnnum = tableEntity.getColumnnum();
        String[] split = columnnum.split(",");
        ArrayList<String> tablelins = new ArrayList<>();
        for (int j = 0; j < tableslist.size(); j++) {
            ArrayList<String> arrlist = new ArrayList<>();
            for (int k = 0; k < split.length; k++) {
                String s = tableslist.get(j).get(Integer.parseInt(split[k]));
                arrlist.add(s);
            }
            tablelins.add(arrlist.toString());
            list.add(arrlist);
            if(j%1000==0){
                ThreadExcelImport threadExcelImport = new ThreadExcelImport(list);
                Thread.sleep(10000);
                threadExcelImport.start();
                list.clear();
            }
        }
        tabledata.get(stringlist.get(i)).setColumnvalue(tablelins);
    }
}

多线程执行生成以一千为单位的excel格式

package com.bootdo.gi;

import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;

import java.io.*;
import java.util.List;
import java.util.Random;

/**
 * Created by tiantian on 2019/7/31.
 */
public class ThreadExcelImport extends Thread {
    private static Workbook wb;

    private List<List<String>> list;

    public ThreadExcelImport(List<List<String>> list) {
        this.list = list;
    }

    public List<List<String>> getList() {
        return list;
    }

    public void setList(List<List<String>> list) {
        this.list = list;
    }

    @Override
    public void run() {

    wb = new XSSFWorkbook();
    Sheet sheet = wb.createSheet();
    for (int i = 0; i <list.size() ; i++) {
        Row row = sheet.createRow(i);
        List<String> rowList = this.list.get(i);
        for (int j = 0; j <rowList.size() ; j++) {
            Cell cell = row.createCell(j);
            cell.setCellValue(rowList.get(j));
        }
    }

    try {
        wb.write(new FileOutputStream(new File("D:\\生成的大量数据\\数据表"+new Random().nextInt(100)+".xlsx")));
    } catch (IOException e) {
        e.printStackTrace();
    }

}
}
以下是Java多线程读取Excel的示例代码: ```java import java.io.File; import java.io.FileInputStream; import java.io.IOException; import java.util.ArrayList; import java.util.List; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import org.apache.poi.ss.usermodel.Cell; import org.apache.poi.ss.usermodel.Row; import org.apache.poi.ss.usermodel.Sheet; import org.apache.poi.ss.usermodel.Workbook; import org.apache.poi.ss.usermodel.WorkbookFactory; public class ExcelReader { private static final int THREAD_COUNT = 5; // 线程数 private static final String FILE_PATH = "test.xlsx"; // Excel文件路径 public static void main(String[] args) throws Exception { File file = new File(FILE_PATH); if (!file.exists()) { throw new IOException("文件不存在!"); } ExecutorService executorService = Executors.newFixedThreadPool(THREAD_COUNT); Workbook workbook = WorkbookFactory.create(new FileInputStream(file)); int sheetCount = workbook.getNumberOfSheets(); for (int i = 0; i < sheetCount; i++) { Sheet sheet = workbook.getSheetAt(i); int rowCount = sheet.getPhysicalNumberOfRows(); int perThreadRowCount = rowCount / THREAD_COUNT; for (int j = 0; j < THREAD_COUNT; j++) { int startIndex = j * perThreadRowCount; int endIndex = (j == THREAD_COUNT - 1) ? rowCount : (j + 1) * perThreadRowCount; executorService.execute(new ExcelReaderTask(sheet, startIndex, endIndex)); } } executorService.shutdown(); } private static class ExcelReaderTask implements Runnable { private Sheet sheet; private int startIndex; private int endIndex; public ExcelReaderTask(Sheet sheet, int startIndex, int endIndex) { this.sheet = sheet; this.startIndex = startIndex; this.endIndex = endIndex; } @Override public void run() { List<List<String>> dataList = new ArrayList<>(); for (int i = startIndex; i < endIndex; i++) { Row row = sheet.getRow(i); if (row == null) { continue; } List<String> rowData = new ArrayList<>(); int cellCount = row.getPhysicalNumberOfCells(); for (int j = 0; j < cellCount; j++) { Cell cell = row.getCell(j); if (cell == null) { rowData.add(""); } else { rowData.add(cell.toString()); } } dataList.add(rowData); } // TODO: 将dataList插入MongoDB } } } ``` 该示例代码使用了Apache POI库来读取Excel文件,使用多线程的方式提高读取速度。具体实现方式是将Excel文件按行分割成多个区间,每个区间由一个线程负责读取。读取后的数据可以插入MongoDB等数据库中。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值