java中实现分片读取本地文件

一、背景

最近遇到一个需求需要把某个本地文件上传到oss上，但是因为文件太大了(5GB+)被限制了。所以只能分片上传。

二、思路

将本地大文件分片读取，然后分片上传

三、实现

public class Demo {

    //指定文件达到4GB使用分片上传

    private static final Long MAX_LENGTH = 4294967296L;

    
    //用来验证截取
    public static String convert(InputStream input) throws IOException {
        StringBuilder stringBuilder = new StringBuilder();

        try (BufferedReader reader = new BufferedReader(new InputStreamReader(input))) {
            String line;

            while ((line = reader.readLine()) != null) {
                stringBuilder.append(line);
            }
        } catch (IOException e) {
            throw e;
        } finally {
            if (input != null) {
                input.close();
            }
        }
        return stringBuilder.toString();
    }

    public static void main(String[] args) throws IOException {
        //分片大小为每片 200MB 可按需调整
        int part_length = 209715200;
        String filePath = "C:\\Users\\Administrator\\Downloads\\test.txt";
        File file1 = new File("C:\\Users\\Administrator\\Downloads\\test.txt");
        //=================  这里初始化分片上传 ====================
        long part_num = file1.length() / part_length;
        long last_part_length = file1.length() % part_length;
        long current = 0L;
        for (int i = 0 ; i < part_num ; i++) {
            byte[] buffer = new byte[part_length];
            RandomAccessFile file = new RandomAccessFile(filePath, "r");
            InputStream inputStream = null;
            try {
                file.seek(current);
                file.read(buffer, 0, part_length);
                inputStream  = new ByteArrayInputStream(buffer);
                //=================  用inputStream将该分片上传 ====================
                current += part_length;
            } finally {
                file.close();
                inputStream.close();
            }
        }
        //可能存在不足分片大小的最后一片文件
        if (last_part_length > 0) {
            byte[] buffer = new byte[(int) last_part_length];
            RandomAccessFile file = new RandomAccessFile(filePath, "r");
            InputStream inputStream = null;
            try {
                file.seek((part_num * part_length));
                file.read(buffer, 0, (int) last_part_length);
                inputStream = new ByteArrayInputStream(buffer);
                //=================  用inputStream将该分片上传 ====================
            } finally {
                file.close();
                inputStream.close();
            }
        }

        //=================  分片上传提交  ====================
    }

}

四、结论

经本人测试可以跑通，分片上传完成的文件下载下来和原文件相同。(可以自己本地创建一个txt测试)。