阿里云分片上传

最新推荐文章于 2024-07-30 15:37:33 发布

loet6010

最新推荐文章于 2024-07-30 15:37:33 发布

阅读量1.1k

点赞数

文章标签： java

本文链接：https://blog.csdn.net/loet6010/article/details/119913643

版权

使用阿里云上传文件，如果碰到大文件上传可能单文件上传就不那么好用了，阿里云提供了分片上传的接口。即将大文件切割成一个个小文件上传，全部上传后再合并为原文件。

分片上传与单文件上传有所不同，下面通过源码一步步来了解。

首先是获取分片上传ID，就是个批次ID的意思，这一个文件的所有分片都在此ID下。

        InitiateMultipartUploadRequest request = new InitiateMultipartUploadRequest(ossBucketName, uploadFileName);
        InitiateMultipartUploadResult result = ossClient.initiateMultipartUpload(request);
        String uploadId = result.getUploadId();

然后计算分片数量，这里要注意一个文件最大支持10000个分片。

            long fileLength = file.getSize();
            int partCount = (int) (fileLength / PART_SIZE);
            if (fileLength % PART_SIZE != 0) {
                partCount++;
            }
            if (partCount > 10000) {
                throw new RuntimeException("超出分片数量限制");
            }

然后就可以开始上传了，上传可以用线程池进行多线程上传，我这里用的是自定义线程池如下：

    private static final ThreadFactory THREAD_FACTORY = new ThreadFactoryBuilder().setNameFormat("alioss-pool-%d")
            .build();
    private static final ExecutorService executorService = new ThreadPoolExecutor(10, 100, 60L, TimeUnit.SECONDS,
            new ArrayBlockingQueue<>(100), THREAD_FACTORY);

上传如下：

            for (int i = 0; i < partCount; i++) {
                long startPos = i * PART_SIZE;
                long curPartSize = (i + 1 == partCount) ? (fileLength - startPos) : PART_SIZE;
                executorService.execute(new PartUploader(file, startPos, curPartSize, i + 1, uploadId, uploadFileName));
            }

执行上传代码

                inputStream = this.localFile.getInputStream();
                long skipLong = inputStream.skip(this.startPos);
                UploadPartRequest uploadPartRequest = new UploadPartRequest();
                uploadPartRequest.setBucketName(ossBucketName);
                uploadPartRequest.setKey(uploadFileName);
                uploadPartRequest.setUploadId(this.uploadId);
                uploadPartRequest.setInputStream(inputStream);
                uploadPartRequest.setPartSize(this.partSize);
                uploadPartRequest.setPartNumber(this.partNumber);
                UploadPartResult uploadPartResult = ossClient.uploadPart(uploadPartRequest);
                synchronized (PART_E_TAGS) {
                    PART_E_TAGS.add(uploadPartResult.getPartETag());
                }

验证上传是否完成

            if (PART_E_TAGS.size() != partCount) {
                throw new IllegalStateException("文件分片上传失败");
            }

全部成功后完成上传，将分片合并为完整文件

        PART_E_TAGS.sort(Comparator.comparingInt(PartETag::getPartNumber));
        CompleteMultipartUploadRequest completeMultipartUploadRequest = new CompleteMultipartUploadRequest(
                ossBucketName, uploadFileName, uploadId, PART_E_TAGS);
        ossClient.completeMultipartUpload(completeMultipartUploadRequest);

如果某个分片上传失败了，还可以取消该次分片上传，删除所有已上传分片

        AbortMultipartUploadRequest abortMultipartUploadRequest = new AbortMultipartUploadRequest(ossBucketName,
                uploadFileName, uploadId);
        ossClient.abortMultipartUpload(abortMultipartUploadRequest);