Minio8.x Java 分片上传注意点

最新推荐文章于 2024-03-18 16:02:13 发布

有量天尊ts

最新推荐文章于 2024-03-18 16:02:13 发布

阅读量2.1k

点赞数 1

分类专栏：问题记录文章标签： java

本文链接：https://blog.csdn.net/qq_35275697/article/details/126175568

版权

问题记录专栏收录该内容

5 篇文章 0 订阅

订阅专栏

Minio8.x Java 分片上传注意点

网上有很多关于minio的分片上传示例代码,基本上思路都差不多,只是8.x版本(具体哪个版本开始没看)之后的整合方式有个地方需要注意

分片上传需要调用获取块,初始化上传获取上传连接,合并等接口,所以需要新建一个Client继承MinioCilent然后调用其父级方法

8.x之后出现了MinioAsyncClient类,是S3Base抽象类的子类,上面说的那些方法都是S3Base的方法,所以,8.x之后自定义Client不再是继承MinioCilent,而应该继承MinioAsyncClient

public class MinioAsyncClient extends S3Base {
  private MinioAsyncClient(
      HttpUrl baseUrl,
      String region,
      boolean isAwsHost,
      boolean isFipsHost,
      boolean isAccelerateHost,
      boolean isDualStackHost,
      boolean useVirtualStyle,
      Provider provider,
      OkHttpClient httpClient) {
    super(
        baseUrl,
        region,
        isAwsHost,
        isFipsHost,
        isAccelerateHost,
        isDualStackHost,
        useVirtualStyle,
        provider,
        httpClient);
  }

自定义Client

public class CustomMinioClient extends MinioAsyncClient {

    public CustomMinioClient(MinioAsyncClient client) {
        super(client);
    }
    //创建分块上传任务
    public String initMultiPartUpload(String bucket, String region, String object, Multimap<String, String> headers, Multimap<String, String> extraQueryParams) throws IOException, InvalidKeyException, NoSuchAlgorithmException, InsufficientDataException, InterruptedException, InternalException, XmlParserException, InvalidResponseException, ExecutionException {
        CompletableFuture<CreateMultipartUploadResponse> future = createMultipartUploadAsync(bucket, region, object, headers, extraQueryParams);
        return future.get().result().uploadId();
    }
    //合并指定上传任务的分块文件
    public CompletableFuture<ObjectWriteResponse> mergeMultipartUpload(String bucketName, String region, String objectName, String uploadId, Part[] parts, Multimap<String, String> extraHeaders, Multimap<String, String> extraQueryParams) throws IOException, InvalidKeyException, NoSuchAlgorithmException, InsufficientDataException, InternalException, XmlParserException {
        return this.completeMultipartUploadAsync(bucketName, region, objectName, uploadId, parts, extraHeaders, extraQueryParams);
    }
    //获取指定上传任务内的已上传的分块信息
    public CompletableFuture<ListPartsResponse> listMultipart(String bucketName, String region, String objectName, Integer maxParts, Integer partNumberMarker, String uploadId, Multimap<String, String> extraHeaders, Multimap<String, String> extraQueryParams) throws NoSuchAlgorithmException, InsufficientDataException, IOException, InvalidKeyException, XmlParserException, InternalException {
        return this.listPartsAsync(bucketName, region, objectName, maxParts, partNumberMarker, uploadId, extraHeaders, extraQueryParams);
    }
}

很多都是CompletableFuture,异步操作,需要同步的可以这样处理(调用合并接口的部分代码)

 CustomMinioClient customMinioClient = minioUtil.getCustomMinioClient();
        String bucketName = minioProperties.getBucketName();
        String region = minioProperties.getRegion();
        String objectName = minioUtil.getObjectName(id);
        try {
            CompletableFuture<ListPartsResponse> completableFuture = customMinioClient.listMultipart(
                    bucketName,
                    region,
                    objectName,
                    1000,
                    0,
                    uploadId,
                    null,
                    null);
            ListPartsResult partsResult = completableFuture.get().result();
            List<Part> parts = partsResult.partList();
            if (! CollectionUtils.isEmpty(parts)) {
                Part[] partArr = new Part[parts.size()];
                for (int i = 0; i < parts.size(); i++) {
                    partArr[i] = new Part(parts.get(i).partNumber(), parts.get(i).etag());
                }

                CompletableFuture<ObjectWriteResponse> objectWriteResponseCompletableFuture = customMinioClient.mergeMultipartUpload(
                        bucketName,
                        region,
                        objectName,
                        uploadId,
                        partArr,
                        null,
                        null);
                ObjectWriteResponse objectWriteResponse = objectWriteResponseCompletableFuture.get();
                log.info("合并分片成功:" + objectWriteResponse.toString());
                return true;
            }

        } catch (Exception e) {
            throw new SysException("合并分片出错");
        }