Minio对象存储及Springboot整合Minio分片上传

已于 2023-02-23 09:54:28 修改
阅读量2.2k 收藏 16
点赞数 7
分类专栏： Minio对象存储 文章标签： 阿里云 java Powered by 金山文档
于 2023-02-23 09:44:23 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/jlonghou/article/details/129174248
版权

  1. 简介

MinIO 是一个基于Apache License v2.0开源协议的对象存储服务。它兼容亚马逊S3云存储服务接口，非常适合于存储大容量非结构化的数据，例如图片、视频、日志文件、备份数据和容器/虚拟机镜像等，而一个对象文件可以是任意大小，从几kb到最大5T不等。

MinIO是一个非常轻量的服务,可以很简单的和其他应用的结合，类似 NodeJS, Redis 或者 MySQL。目前支持JavaScript 、Java、Python、Golang、.NET。

MinIO 是一款高性能、分布式的对象存储系统. 它是一款软件产品, 可以100%的运行在标准硬件。即X86等低成本机器也能够很好的运行MinIO。

MinIO与传统的存储和其他的对象存储不同的是：它一开始就针对性能要求更高的私有云标准进行软件架构设计。因为MinIO一开始就只为对象存储而设计。所以他采用了更易用的方式进行设计，它能实现对象存储所需要的全部功能，在性能上也更加强劲，它不会为了更多的业务功能而妥协，失去MinIO的易用性、高效性。 这样的结果所带来的好处是：它能够更简单的实现局有弹性伸缩能力的原生对象存储服务。

MinIO在传统对象存储用例（例如辅助存储，灾难恢复和归档）方面表现出色。同时，它在机器学习、大数据、私有云、混合云等方面的存储技术上也独树一帜。当然，也不排除数据分析、高性能应用负载、原生云的支持。

在中国：阿里巴巴、腾讯、百度、中国联通、华为、中国移动等等9000多家企业也都在使用MinIO产品

MinIO现在也是CNCF成员，在云原生存储部分和ceph等一起作为目前的解决方案之一。

  1. Windows/Linux单机部署Minio

  • 下载

国内下载地址

根据系统下载安装包

  • Windows

  1. 下载安装包

  1. 在minio.exe目录打开cmd

  1. 配置参数并启动

# 设置用户名
set MINIO_ACCESS_KEY=admin
# 设置密码（8位）
set MINIO_SECRET_KEY=admin123
# 指定启动端口(未指定默认9000)及存储位置
minio.exe  server  --address 0.0.0.0:9999 D:/data

  1. 登录地址IP+9999，输入用户名及密码，搭建完成

  • Linux

  1. 下载安装包，因为我用的是华为云鲲鹏Centos7服务器，所以下载的ARM64版本

  1. 创建相关目录

# 创建文件存储目录
mkdir -p /data/minio
# 创建程序存放目录，并上传minio至此目录
mkdir -p /usr/local/minio
cd /usr/local/minio
# 修改可读权限
chmod +x minio 
# 设置用户名
export  MINIO_ACCESS_KEY=admin
# 设置密码
export  MINIO_SECRET_KEY=admin123

  1. 启动并访问首页

# 启动，此处只是演示，实际使用需nohup或注册为服务启动
./minio server --address 0.0.0.0:9998 /data/minio

  1. Spring Boot集成Minio

  • 环境搭建

首先我们搭建一个spring boot基础工程，引入以下依赖

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <!-- https://mvnrepository.com/artifact/io.minio/minio -->
        <dependency>
            <groupId>io.minio</groupId>
            <artifactId>minio</artifactId>
            <version>8.3.4</version>
        </dependency>
        <dependency>
            <groupId>com.squareup.okhttp3</groupId>
            <artifactId>okhttp</artifactId>
            <version>4.9.2</version>
        </dependency>
        <dependency>
            <groupId>cn.hutool</groupId>
            <artifactId>hutool-all</artifactId>
            <version>5.7.13</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-configuration-processor</artifactId>
        </dependency>
        <!-- https://mvnrepository.com/artifact/commons-fileupload/commons-fileupload -->
        <dependency>
            <groupId>commons-fileupload</groupId>
            <artifactId>commons-fileupload</artifactId>
            <version>1.4</version>
        </dependency>
  • 配置文件设置

在application.yml中设置

# Tomcat
server:
  port: 8888
  
spring:
  # 配置文件上传大小限制
  servlet:
    multipart:
      max-file-size: 100MB
      max-request-size: 100MB
      
# Minio配置
minio:
  # 访问的url
  endpoint: http://127.0.0.1:9000
  # API的端口
  port: 9000
  # 秘钥
  accessKey: minioadmin
  secretKey: minioadmin
  bucketName: miniohjl # 桶名 我这是给出了一个默认桶名
  image-size: 10485760 # 我在这里设定了 图片文件的最大大小
  file-size: 1073741824 # 此处是设定了文件的最大大小
  allowFileType: jpg,png,jpeg,zip,rar,doc,docx,xls,xlsx,img,iso
    # 分片上传有效期: 秒
  chunkUploadExpirySecond: 86400
  • 配置类设置
@Data
@Configuration
@ConfigurationProperties(prefix = "minio")
public class ParamConfig {
    private String endpoint;
    private String port;
    private String accessKey;
    private String secretKey;
    private String bucketName;
    /**
     * 运行上传的文件类型
     */
    private String allowFileType;

    /**
     * 分片上传有效期（单位：秒）
     */
    private Integer chunkUploadExpirySecond;
}
  • 工具类整合
package com.example.minio.common.utils;

import cn.hutool.core.date.DateUtil;
import com.example.minio.common.config.ParamConfig;
import io.minio.*;
import io.minio.http.Method;
import io.minio.messages.Bucket;
import io.minio.messages.DeleteObject;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;
import org.springframework.web.multipart.MultipartFile;

import javax.annotation.PostConstruct;
import java.io.InputStream;
import java.util.Date;
import java.util.List;
import java.util.Optional;
import java.util.stream.Stream;

/**
 * @Author: HOUJL
 * @Date: 2023/2/14
 * @Description:
 */
@Slf4j
@Component
public class MinIoUtil {
    public static MinioClient minioClient;
    public final ParamConfig paramConfig;

    public MinIoUtil(ParamConfig paramConfig) {
        this.paramConfig = paramConfig;
    }

    /**
     * 初始化minio配置
     */
    @PostConstruct
    public void init() {
        try {
            log.info("Minio Initialize........................");
            minioClient = MinioClient.builder().endpoint(paramConfig.getEndpoint()).credentials(paramConfig.getAccessKey(), paramConfig.getSecretKey()).build();
            createBucket(paramConfig.getBucketName());
            log.info("Minio Initialize........................successful");
        } catch (Exception e) {
            log.error("初始化minio配置异常: 【{}】", e.fillInStackTrace());
        }
    }

    /******************************  Operate Bucket Start  ******************************/

    /**
     * 判断bucket是否存在
     */
    @SneakyThrows(Exception.class)
    public boolean bucketExists(String bucketName) {
        return minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucketName).build());
    }

    /**
     * 创建bucket
     */
    @SneakyThrows(Exception.class)
    public void createBucket(String bucketName) {
        boolean isExist = minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucketName).build());
        if (!isExist) {
            minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucketName).build());
        }
    }

    /**
     * 获取全部bucket
     */
    @SneakyThrows(Exception.class)
    public List<Bucket> getAllBuckets() {
        return minioClient.listBuckets();
    }

    /**
     * 根据bucketName获取其相关信息
     *
     * @param bucketName
     * @return
     */
    @SneakyThrows(Exception.class)
    public Optional<Bucket> getBucket(String bucketName) {
        return getAllBuckets().stream().filter(b -> b.name().equals(bucketName)).findFirst();
    }


    /**
     * 根据bucketName删除Bucket，true：删除成功； false：删除失败，文件或已不存在
     *
     * @param bucketName
     * @throws Exception
     */
    @SneakyThrows(Exception.class)
    public void removeBucket(String bucketName) {
        minioClient.removeBucket(RemoveBucketArgs.builder().bucket(bucketName).build());
    }

    /******************************  Operate Bucket End  ******************************/

    /**
     * 上传本地文件
     *
     * @param bucketName 存储桶
     * @param fileName   对象名称
     * @param filePath   本地文件路径
     * @return
     */
    @SneakyThrows(Exception.class)
    public ObjectWriteResponse uploadFile(String bucketName, String fileName, String filePath) {
        return minioClient.uploadObject(
                UploadObjectArgs.builder()
                        .bucket(bucketName)
                        .object(fileName)
                        .filename(filePath)
                        .build());
    }

    /**
     * 上传文件
     * 返回可以直接预览文件的URL
     */
    public String uploadFile(MultipartFile file) {
        try {
            //如果存储桶不存在则创建
            if (!bucketExists(paramConfig.getBucketName())) {
                createBucket(paramConfig.getBucketName());
            }
            String originalFilename = file.getOriginalFilename();
            if ("".equals(originalFilename)) {
                originalFilename = file.getName();
            }
            //保证文件不重名(并且没有特殊字符)
            String fileName = DateUtil.format(new Date(), "yyyyMMddHHmmss") + "_" + originalFilename;
            minioClient.putObject(PutObjectArgs.builder()
                    .bucket(paramConfig.getBucketName())
                    .object(fileName)
                    .stream(file.getInputStream(), file.getInputStream().available(), -1)
                    .contentType(file.getContentType())
                    .build());

            return getPreviewFileUrl(paramConfig.getBucketName(), fileName);
        } catch (Exception e) {
            log.error(e.getMessage(), e);
            return "";
        }
    }

    @SneakyThrows(Exception.class)
    public String putFile(String bucketName, String fileName, InputStream stream) {
        return upload(bucketName, fileName, stream, "application/octet-stream");
    }

    /**
     * 文件上传
     * 返回下载文件url地址 和下面upload方法仅传参不同
     * bucketName 也可以直接从ParamConfig对象中获取
     */
    @SneakyThrows(Exception.class)
    public String upload(String bucketName, String fileName, InputStream stream, String contentType) {
        minioClient.putObject(PutObjectArgs.builder()
                .bucket(bucketName)
                .object(fileName)
                .stream(stream, stream.available(), -1)
                .contentType(contentType)
                .build());
        return getPreviewFileUrl(bucketName, fileName);
    }

    /**
     * 文件上传
     * 返回下载文件url地址  和上面upload方法仅传参不同
     */
    @SneakyThrows(Exception.class)
    public String upload(String bucketName, MultipartFile file) {
        final InputStream is = file.getInputStream();
        final String fileName = file.getOriginalFilename();
        minioClient.putObject(PutObjectArgs.builder()
                .bucket(bucketName)
                .object(fileName)
                .stream(is, is.available(), -1)
                .contentType(file.getContentType())
                .build());
        is.close();
        return getPreviewFileUrl(bucketName, fileName);
    }

    /**
     * 删除文件
     *
     * @param bucketName: 桶名
     * @param fileName:   文件名
     */
    @SneakyThrows(Exception.class)
    public void deleteFile(String bucketName, String fileName) {
        minioClient.removeObject(
                RemoveObjectArgs.builder()
                        .bucket(bucketName)
                        .object(fileName)
                        .build());
    }

    /**
     * 批量删除
     *
     * @param fileNames
     */
    @SneakyThrows(Exception.class)
    public void removeFiles(List<String> fileNames) {
        removeFiles(paramConfig.getBucketName(), fileNames);
    }

    /**
     * @param bucketName
     * @param fileNames
     */
    @SneakyThrows(Exception.class)
    public void removeFiles(String bucketName, List<String> fileNames) {
        Stream<DeleteObject> stream = fileNames.stream().map(DeleteObject::new);
        minioClient.removeObjects(RemoveObjectsArgs.builder().bucket(bucketName).objects(stream::iterator).build());
    }

    /**
     * 获取minio文件的下载或者预览地址
     * 取决于调用本方法的方法中的PutObjectOptions对象有没有设置contentType
     *
     * @param bucketName: 桶名
     * @param fileName:   文件名
     */
    @SneakyThrows(Exception.class)
    public String getPreviewFileUrl(String bucketName, String fileName) {
        GetPresignedObjectUrlArgs args = GetPresignedObjectUrlArgs.builder()
                .bucket(bucketName)
                .object(fileName)
                .method(Method.GET).build();
        return minioClient.getPresignedObjectUrl(args);
    }

    /**
     * 获取文件外链
     *
     * @param bucketName 存储桶
     * @param fileName   文件名
     * @param expires    过期时间 <=7 秒 （外链有效时间（单位：秒））
     * @return url
     */
    @SneakyThrows(Exception.class)
    public String getPresignedObjectUrl(String bucketName, String fileName, Integer expires) {
        return minioClient.getPresignedObjectUrl(
                GetPresignedObjectUrlArgs.builder()
                        .method(Method.GET)
                        .bucket(bucketName)
                        .object(fileName)
                        .expiry(expires)
                        .build()
        );
    }

    /**
     * 获取文件信息, 如果抛出异常则说明文件不存在
     *
     * @param bucketName 存储桶
     * @param objectName 文件名称
     * @return
     */
    @SneakyThrows(Exception.class)
    public String getFileStatusInfo(String bucketName, String objectName) {
        return minioClient.statObject(
                StatObjectArgs.builder()
                        .bucket(bucketName)
                        .object(objectName)
                        .build()).toString();

    }

    /**
     * 断点下载
     *
     * @param bucketName 存储桶
     * @param objectName 文件名称
     * @param offset     起始字节的位置
     * @param length     要读取的长度
     * @return 二进制流
     */
    @SneakyThrows(Exception.class)
    public InputStream getObject(String bucketName, String objectName, long offset, long length) {
        return minioClient.getObject(
                GetObjectArgs.builder()
                        .bucket(bucketName)
                        .object(objectName)
                        .offset(offset)
                        .length(length)
                        .build());

    }

    /**
     * 下载
     *
     * @param bucketName 存储桶
     * @param objectName 文件名称
     * @return 二进制流
     */
    @SneakyThrows(Exception.class)
    public InputStream dowload(String bucketName, String objectName) {
        return minioClient.getObject(
                GetObjectArgs.builder()
                        .bucket(bucketName)
                        .object(objectName)
                        .build());

    }
}
  • Minio对象存储服务接口调用
package com.example.minio.controller;

import com.example.minio.common.config.ParamConfig;
import com.example.minio.common.utils.MinIoUtil;
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiOperation;
import org.apache.commons.io.IOUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;

import javax.servlet.http.HttpServletResponse;
import java.io.InputStream;

/**
 * @Author: HOUJL
 * @Date: 2023/2/14
 * @Description:
 */
@RestController
@RequestMapping("/minio")
@Api(value = "Minio对象存储服务", tags = "Minio对象存储服务")
public class MinioController {
    private final static Logger LOGGER = LoggerFactory.getLogger(MinioController.class);

    public final MinIoUtil minIoUtil;
    private final ParamConfig paramConfig;

    public MinioController(MinIoUtil minIoUtil, ParamConfig paramConfig) {
        this.minIoUtil = minIoUtil;
        this.paramConfig = paramConfig;
    }


    @ApiOperation(value = "文件上传")
    @PostMapping("/upload")
    public String upload(@RequestParam("file") MultipartFile file) {
        try {
            String fileUrl = minIoUtil.uploadFile(file);
            return fileUrl;
        } catch (Exception e) {
            LOGGER.error("上传失败");
            return "上传失败";
        }
    }

    @ApiOperation(value = "删除文件")
    @DeleteMapping("/delete")
    public void delete(@RequestParam("fileName") String fileName) {
        minIoUtil.deleteFile(paramConfig.getBucketName(), fileName);
    }

    @ApiOperation(value = "获取文件详情")
    @GetMapping("/info")
    public String getFileStatusInfo(@RequestParam("fileName") String fileName) {
        return minIoUtil.getFileStatusInfo(paramConfig.getBucketName(), fileName);
    }

    @ApiOperation(value = "获取文件外链")
    @GetMapping("/url")
    public String getPresignedObjectUrl(@RequestParam("fileName") String fileName) {
        return minIoUtil.getPresignedObjectUrl(paramConfig.getBucketName(), fileName, 7);

    }

    @ApiOperation(value = "文件下载")
    @GetMapping("/download")
    public void download(@RequestParam("fileName") String fileName, HttpServletResponse response) {
        try {
            InputStream fileInputStream = minIoUtil.dowload(paramConfig.getBucketName(), fileName);
            response.setHeader("Content-Disposition", "attachment;filename=" + fileName);
            response.setContentType("application/force-download");
            response.setCharacterEncoding("UTF-8");
            IOUtils.copy(fileInputStream, response.getOutputStream());
        } catch (Exception e) {
            LOGGER.error("下载失败");
        }
    }

}

  1. SpringBoot 分片上传、断点续传、秒传、直传Minio

参考

本文为CSDN博主「单人可khalil」的原创文章。

原文链接：https://blog.csdn.net/m0_46493080/article/details/128303690

  • 大致的流程如下：

前端获取文件MD5，发送至后台判断是否有该文件，有则直接转存；

前端调用初始化接口，后端调用 minio 初始化，返回分片上传地址和 uploadId；

前端上传分片文件；

上传完成后，前端发送请求至后台服务，后台服务调用 minio 合并文件；

前端代码地址：https://github.com/lanweihong/vue-minio-upload-sample

流程图如下：

  • 实现过程
  • 基本配置见Spring Boot集成Minio
  • 获取ParamConfig默认属性
package com.example.minio.common.config;

import io.minio.MinioClient;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * @Author: HOUJL
 * @Date: 2023/2/21
 * @Description:
 */
@Configuration
@EnableConfigurationProperties(ParamConfig.class)
@Slf4j
public class MinioConfig {

    private ParamConfig paramConfig;

    @Autowired
    public void setMinioPropertiesConfig(ParamConfig paramConfig) {
        this.paramConfig = paramConfig;
    }

    @Bean
    public CustomMinioClient customMinioClient() {
        MinioClient minioClient;
        try {
            minioClient = MinioClient.builder()
                    .endpoint(paramConfig.getEndpoint())
                    .credentials(paramConfig.getAccessKey(), paramConfig.getSecretKey())
                    .build();
        } catch (Exception e) {
            log.error("初始化 Minio 客户端失败：" + e.getMessage());
            throw e;
        }
        return new CustomMinioClient(minioClient);
    }
}
  • 自定义Minio继承 MinioClient 
package com.example.minio.common.config;

import com.google.common.collect.Multimap;
import io.minio.CreateMultipartUploadResponse;
import io.minio.ListPartsResponse;
import io.minio.MinioClient;
import io.minio.ObjectWriteResponse;
import io.minio.errors.*;
import io.minio.messages.Part;

import java.io.IOException;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;

/**
 * @Author: HOUJL
 * @Date: 2023/2/21
 * @Description: SpringBoot 分片上传、断点续传、秒传、直传Minio
 * 自定义Minio继承 MinioClient
 */
public class CustomMinioClient extends MinioClient {

    protected CustomMinioClient(MinioClient client) {
        super(client);
    }

    /**
     * 初始化分片上传、获取 uploadId
     *
     * @param bucketName       bucketName 存储桶名称
     * @param region           region
     * @param objectName       objectName 文件名称
     * @param headers          headers 请求头
     * @param extraQueryParams extraQueryParams
     * @return
     * @throws ServerException
     * @throws InsufficientDataException
     * @throws ErrorResponseException
     * @throws NoSuchAlgorithmException
     * @throws IOException
     * @throws InvalidKeyException
     * @throws XmlParserException
     * @throws InvalidResponseException
     * @throws InternalException
     */
    public String getUploadId(String bucketName, String region, String objectName,
                              Multimap<String, String> headers, Multimap<String, String> extraQueryParams)
            throws ServerException, InsufficientDataException, ErrorResponseException, NoSuchAlgorithmException, IOException, InvalidKeyException, XmlParserException, InvalidResponseException, InternalException {
        CreateMultipartUploadResponse response = this.createMultipartUpload(bucketName, region, objectName, headers, extraQueryParams);

        return response.result().uploadId();
    }

    /**
     * 合并分片
     *
     * @param bucketName 桶名称
     * @param region
     * @param objectName 文件名称
     * @param uploadId 上传的 uploadId
     * @param parts 分片集合
     * @param extraHeaders
     * @param extraQueryParams
     * @return
     * @throws ServerException
     * @throws InsufficientDataException
     * @throws ErrorResponseException
     * @throws NoSuchAlgorithmException
     * @throws IOException
     * @throws InvalidKeyException
     * @throws XmlParserException
     * @throws InvalidResponseException
     * @throws InternalException
     */
    public ObjectWriteResponse mergeMultipart(String bucketName, String region, String objectName, String uploadId,
                                              Part[] parts, Multimap<String, String> extraHeaders, Multimap<String, String> extraQueryParams)
            throws ServerException, InsufficientDataException, ErrorResponseException, NoSuchAlgorithmException, IOException, InvalidKeyException, XmlParserException, InvalidResponseException, InternalException {
        return this.completeMultipartUpload(bucketName, region, objectName, uploadId, parts, extraHeaders, extraQueryParams);
    }

    /**
     * 查询当前上传后的分片信息
     *
     * @param bucketName 桶名称
     * @param region
     * @param objectName 文件名称
     * @param maxParts 分片数量
     * @param partNumberMaker 分片起始值
     * @param uploadId 上传的 uploadId
     * @param extraHeaders
     * @param extraQueryParams
     * @return
     * @throws ServerException
     * @throws InsufficientDataException
     * @throws ErrorResponseException
     * @throws NoSuchAlgorithmException
     * @throws IOException
     * @throws InvalidKeyException
     * @throws XmlParserException
     * @throws InvalidResponseException
     * @throws InternalException
     */
    public ListPartsResponse listMultipart(String bucketName, String region, String objectName, Integer maxParts, Integer partNumberMaker,
                                           String uploadId, Multimap<String, String> extraHeaders, Multimap<String, String> extraQueryParams) throws ServerException, InsufficientDataException, ErrorResponseException, NoSuchAlgorithmException, IOException, InvalidKeyException, XmlParserException, InvalidResponseException, InternalException {
        return this.listParts(bucketName, region, objectName, maxParts, partNumberMaker, uploadId, extraHeaders, extraQueryParams);
    }
}
  • minio、上传、合并分片的核心方法
package com.example.minio.common.helper;

import cn.hutool.core.date.LocalDateTimeUtil;
import com.example.minio.common.config.CustomMinioClient;
import com.example.minio.common.config.ParamConfig;
import com.example.minio.common.exception.BusinessException;
import com.example.minio.dto.MinioUploadInfo;
import com.google.common.collect.HashMultimap;
import io.minio.GetPresignedObjectUrlArgs;
import io.minio.ListPartsResponse;
import io.minio.ObjectWriteResponse;
import io.minio.errors.*;
import io.minio.http.Method;
import io.minio.messages.Part;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import java.io.IOException;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.time.LocalDateTime;
import java.time.temporal.ChronoUnit;
import java.util.*;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;

/**
 * @Author: HOUJL
 * @Date: 2023/2/22
 * @Description:
 */
@Component
@Slf4j
public class MinioHelper {

    private final ParamConfig paramConfig;
    private final CustomMinioClient customMinioClient;

    @Autowired
    public MinioHelper(ParamConfig paramConfig, CustomMinioClient customMinioClient) {
        this.paramConfig = paramConfig;
        this.customMinioClient = customMinioClient;
    }

    /**
     * 初始化获取 uploadId
     *
     * @param objectName  文件名
     * @param partCount   分片总数
     * @param contentType contentType
     * @return
     */
    public MinioUploadInfo initMultiPartUpload(String objectName, int partCount, String contentType) {
        HashMultimap<String, String> headers = HashMultimap.create();
        headers.put("Content-Type", contentType);

        String uploadId = "";
        List<String> partUrlList = new ArrayList<>();
        try {
            // 获取 uploadId
            uploadId = customMinioClient.getUploadId(paramConfig.getBucketName(),
                    null,
                    objectName,
                    headers,
                    null);
            Map<String, String> paramsMap = new HashMap<>(2);
            paramsMap.put("uploadId", uploadId);
            for (int i = 1; i <= partCount; i++) {
                paramsMap.put("partNumber", String.valueOf(i));
                // 获取上传 url
                String uploadUrl = customMinioClient.getPresignedObjectUrl(GetPresignedObjectUrlArgs.builder()
                        // 注意此处指定请求方法为 PUT，前端需对应，否则会报 `SignatureDoesNotMatch` 错误
                        .method(Method.PUT)
                        .bucket(paramConfig.getBucketName())
                        .object(objectName)
                        // 指定上传连接有效期
                        .expiry(paramConfig.getChunkUploadExpirySecond(), TimeUnit.SECONDS)
                        .extraQueryParams(paramsMap).build());

                partUrlList.add(uploadUrl);
            }
        } catch (Exception e) {
            log.error("initMultiPartUpload Error:" + e);
            return null;
        }
        // 过期时间
        LocalDateTime expireTime = LocalDateTimeUtil.offset(LocalDateTime.now(), paramConfig.getChunkUploadExpirySecond(), ChronoUnit.SECONDS);
        MinioUploadInfo result = new MinioUploadInfo();
        result.setUploadId(uploadId);
        result.setExpiryTime(expireTime);
        result.setUploadUrls(partUrlList);
        return result;
    }

    /**
     * 分片合并
     *
     * @param objectName 文件名
     * @param uploadId   uploadId
     * @return
     */
    public String mergeMultiPartUpload(String objectName, String uploadId) {
        // todo 最大1000分片 这里好像可以改吧
        Part[] parts = new Part[1000];
        int partIndex = 0;
        ListPartsResponse partsResponse = listUploadPartsBase(objectName, uploadId);
        if (null == partsResponse) {
            log.error("查询文件分片列表为空");
            throw new BusinessException("分片列表为空");
        }
        for (Part partItem : partsResponse.result().partList()) {
            parts[partIndex] = new Part(partIndex + 1, partItem.etag());
            partIndex++;
        }
        ObjectWriteResponse objectWriteResponse;
        try {
            objectWriteResponse = customMinioClient.mergeMultipart(paramConfig.getBucketName(), null, objectName, uploadId, parts, null, null);
        } catch (Exception e) {
            log.error("分片合并失败：" + e);
            throw new BusinessException("分片合并失败：" + e.getMessage());
        }
        if (null == objectWriteResponse) {
            log.error("合并失败，合并结果为空");
            throw new BusinessException("分片合并失败");
        }
        return objectWriteResponse.region();
    }

    /**
     * 获取已上传的分片列表
     *
     * @param objectName 文件名
     * @param uploadId   uploadId
     * @return
     */
    public List<Integer> listUploadChunkList(String objectName, String uploadId) {
        ListPartsResponse partsResponse = listUploadPartsBase(objectName, uploadId);
        if (null == partsResponse) {
            return Collections.emptyList();
        }
        return partsResponse.result().partList().stream().map(Part::partNumber).collect(Collectors.toList());
    }

    private ListPartsResponse listUploadPartsBase(String objectName, String uploadId) {
        int maxParts = 1000;
        ListPartsResponse partsResponse;
        try {
            partsResponse = customMinioClient.listMultipart(paramConfig.getBucketName(), null, objectName, maxParts, 0, uploadId, null, null);
        } catch (ServerException | InsufficientDataException | ErrorResponseException | NoSuchAlgorithmException | IOException | XmlParserException | InvalidKeyException | InternalException | InvalidResponseException e) {
            log.error("查询文件分片列表错误：{}，uploadId:{}", e, uploadId);
            return null;
        }
        return partsResponse;
    }
}

基本工具类

package com.example.minio.common.exception;

import com.example.minio.common.enums.MessageEnum;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;

/**
 * @Author: HOUJL
 * @Date: 2023/2/22
 * @Description:
 */
@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public abstract class BaseErrorException extends RuntimeException {

    private static final long serialVersionUID = 6386720492655133851L;
    private int code;
    private String error;

    public BaseErrorException(MessageEnum messageEnum) {
        this.code = messageEnum.getCode();
        this.error = messageEnum.getMessage();
    }
}

package com.example.minio.common.exception;

import com.example.minio.common.enums.MessageEnum;
import lombok.Data;

/**
 * @Author: HOUJL
 * @Date: 2023/2/22
 * @Description:
 */
@Data
public class BusinessException extends BaseErrorException {

    private static final long serialVersionUID = 2369773524406947262L;

    public BusinessException(MessageEnum messageEnum) {
        super(messageEnum);
    }

    public BusinessException(String error) {
        super.setCode(-1);
        super.setError(error);
    }
}

package com.example.minio.common.utils;

import org.springframework.beans.BeanUtils;

import java.util.Collections;
import java.util.List;
import java.util.stream.Collectors;

/**
 * @Author: HOUJL
 * @Date: 2023/2/22
 * @Description:
 */
public class HongBeanUtils {

    public static <DTO> List<DTO> doListToDtoList(List<?> doList, Class<DTO> dtoClass) {
        return pojoListConvert(doList, dtoClass);
    }

    public static <VO> List<VO> dtoListToVoList(List<?> dtoList, Class<VO> voClass) {
        return pojoListConvert(dtoList, voClass);
    }

    public static <DTO> DTO doToDto(Object doEntity, Class<DTO> clazz) {
        return doToDto(doEntity, clazz, "");
    }

    public static <DTO> DTO doToDto(Object doEntity, Class<DTO> clazz, String... ignoreProperties) {
        if (null == doEntity || null == clazz) {
            return null;
        }
        return pojoConvert(doEntity, clazz, ignoreProperties);
    }

    public static <VO> VO dtoToVo(Object dtoEntity, Class<VO> voClass, String... ignoreProperties) {
        if (null == dtoEntity || null == voClass) {
            return null;
        }
        return pojoConvert(dtoEntity, voClass, ignoreProperties);
    }

    public static <T> T copy(Object source, Class<T> targetClass) {
        return pojoConvert(source, targetClass);
    }

    private static <T> List<T> pojoListConvert(List<?> sourceList, Class<T> targetClass) {
        if (sourceList.size() == 0 || null == targetClass) {
            return Collections.emptyList();
        }
        return sourceList.stream().map(item -> pojoConvert(item, targetClass)).collect(Collectors.toList());
    }

    private static <T> T pojoConvert(Object sourceEntity, Class<T> targetClass, String... ignoreProperties) {
        T targetInstance;
        try {
            targetInstance = targetClass.newInstance();
        } catch (IllegalAccessException | InstantiationException exception) {
            return null;
        }
        if (ignoreProperties.length > 0) {
            BeanUtils.copyProperties(sourceEntity, targetInstance, ignoreProperties);
        } else {
            BeanUtils.copyProperties(sourceEntity, targetInstance);
        }
        return targetInstance;
    }
}
package com.example.minio.common.enums;

/**
 * @Author: HOUJL
 * @Date: 2023/2/22
 * @Description:
 */
public class CommonEnums {

    public enum MinioFileStatusEnum {
        /**
         * 待上传
         * 已上传
         * 上传中
         */
        UN_UPLOADED,
        UPLOADED,
        UPLOADING
    }
}
package com.example.minio.common.enums;

import lombok.Getter;

/**
 * @Author: HOUJL
 * @Date: 2023/2/22
 * @Description: 消息枚举
 */
public enum MessageEnum {

    /**
     * 消息枚举
     */
    FAIL(-1, "操作失败"),
    SUCCESS(200, "操作成功"),
    RECORD_NOT_EXISTED(1001, "记录不存在"),
    PARAM_NOT_NULL(1002, "参数不能为空"),
    PARAM_INVALID(1003, "参数错误"),
    UPLOAD_FILE_NOT_NULL(1004, "上传文件不能为空");

    MessageEnum(int value, String text) {
        this.code = value;
        this.message = text;
    }

    @Getter
    private final int code;

    @Getter
    private final String message;

    public static MessageEnum valueOf(int value) {
        MessageEnum[] enums = values();
        for (MessageEnum enumItem : enums) {
            if (value == enumItem.getCode()) {
                return enumItem;
            }
        }
        return null;
    }
}
  • Minio文件切片上传接口调用
package com.example.minio.controller;

import cn.hutool.json.JSONObject;
import com.alibaba.druid.util.StringUtils;
import com.example.minio.dto.MinioOperationResult;
import com.example.minio.dto.MinioUploadInfo;
import com.example.minio.param.GetMinioUploadInfoParam;
import com.example.minio.param.MergeMinioMultipartParam;
import com.example.minio.service.IFileUploadInfoService;
import com.example.minio.vo.JsonResult;
import io.swagger.annotations.Api;
import org.simpleframework.xml.core.Validate;
import org.springframework.web.bind.annotation.*;

import javax.validation.Valid;

/**
 * @Author: HOUJL
 * @Date: 2023/2/22
 * @Description:
 */
@RestController
@RequestMapping("/file-upload")
@Api(value = "Minio文件切片上传", tags = "Minio文件切片上传")
public class SliceUploadController {

    private final IFileUploadInfoService fileService;

    public SliceUploadController(IFileUploadInfoService fileService) {
        this.fileService = fileService;
    }

    /**
     * 获取上传 url
     *
     * @param param 参数
     * @return
     */
    @PostMapping("/upload")
    public JsonResult<MinioUploadInfo> getUploadId(@Validate @RequestBody GetMinioUploadInfoParam param) {
        MinioUploadInfo minioUploadId = fileService.getUploadId(param);
        return JsonResult.ok(minioUploadId);
    }

    /**
     * 校验文件是否存在
     *
     * @param md5 文件 md5
     * @return
     */
    @GetMapping("/upload/check")
    public JsonResult<MinioOperationResult> checkFileUploadedByMd5(@RequestParam("md5") String md5) {
        MinioOperationResult result = fileService.checkFileExistsByMd5(md5);
        return JsonResult.ok(result);
    }

    /**
     * 合并文件
     *
     * @param param
     * @return
     */
    @PostMapping("/upload/merge")
    public JsonResult<JSONObject> mergeUploadFile(@Valid MergeMinioMultipartParam param) {
        String result = fileService.mergeMultipartUpload(param);
        if (StringUtils.isEmpty(result)) {
            return JsonResult.error("合并失败");
        }
        JSONObject object = new JSONObject();
        object.set("url", result);
        return JsonResult.ok(object);
    }

}
package com.example.minio.service;

import com.baomidou.mybatisplus.extension.service.IService;
import com.example.minio.dto.MinioOperationResult;
import com.example.minio.dto.MinioUploadInfo;
import com.example.minio.entity.FileUploadInfo;
import com.example.minio.param.GetMinioUploadInfoParam;
import com.example.minio.param.MergeMinioMultipartParam;

import java.util.List;

/**
 * @Author: HOUJL
 * @Date: 2023/2/22
 * @Description:
 */
public interface IFileUploadInfoService extends IService<FileUploadInfo> {

    /**
     * 获取分片上传信息
     *
     * @param param 参数
     * @return
     */
    MinioUploadInfo getUploadId(GetMinioUploadInfoParam param);

    /**
     * 检查文件是否存在
     *
     * @param md5 md5
     * @return
     */
    MinioOperationResult checkFileExistsByMd5(String md5);

    /**
     * 查询已上传的分片序号
     *
     * @param objectName 文件名
     * @param uploadId   uploadId
     * @return
     */
    List<Integer> listUploadParts(String objectName, String uploadId);

    /**
     * 分片合并
     *
     * @param param 参数
     * @return
     */
    String mergeMultipartUpload(MergeMinioMultipartParam param);
}
package com.example.minio.service.impl;

import com.alibaba.druid.util.StringUtils;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import com.example.minio.common.enums.CommonEnums;
import com.example.minio.common.helper.MinioHelper;
import com.example.minio.dao.FileUploadInfoMapper;
import com.example.minio.dto.MinioFileChunkUploadInfoDTO;
import com.example.minio.dto.MinioFileUploadInfoDTO;
import com.example.minio.dto.MinioOperationResult;
import com.example.minio.dto.MinioUploadInfo;
import com.example.minio.entity.FileUploadInfo;
import com.example.minio.param.GetMinioUploadInfoParam;
import com.example.minio.param.MergeMinioMultipartParam;
import com.example.minio.param.MinioFileChunkUploadInfoParam;
import com.example.minio.param.MinioFileUploadInfoParam;
import com.example.minio.service.IFileUploadInfoService;
import com.example.minio.service.IMinioFileChunkUploadInfoService;
import com.example.minio.service.IMinioFileUploadInfoService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;

import java.util.List;
import java.util.stream.Collectors;

/**
 * @Author: HOUJL
 * @Date: 2023/2/22
 * @Description:
 */
@Service
@Slf4j
public class FileUploadInfoServiceImpl extends ServiceImpl<FileUploadInfoMapper, FileUploadInfo> implements IFileUploadInfoService {

    private final MinioHelper minioHelper;
    private final IMinioFileUploadInfoService minioFileUploadInfoService;
    private final IMinioFileChunkUploadInfoService minioFileChunkUploadInfoService;

    public FileUploadInfoServiceImpl(MinioHelper minioHelper, IMinioFileUploadInfoService minioFileUploadInfoService, IMinioFileChunkUploadInfoService minioFileChunkUploadInfoService) {
        this.minioHelper = minioHelper;
        this.minioFileUploadInfoService = minioFileUploadInfoService;
        this.minioFileChunkUploadInfoService = minioFileChunkUploadInfoService;
    }

    @Override
    public MinioUploadInfo getUploadId(GetMinioUploadInfoParam param) {
        MinioUploadInfo uploadInfo;
        MinioFileUploadInfoDTO minioFileUploadInfo = this.minioFileUploadInfoService.getByFileMd5(param.getFileMd5());
        if (null == minioFileUploadInfo) {
            // 计算分片数量
            double partCount = Math.ceil(param.getFileSize() / param.getChunkSize());
            log.info("总分片数：" + partCount);
            uploadInfo = minioHelper.initMultiPartUpload(param.getFileName(), (int) partCount, param.getContentType());
            if (null != uploadInfo) {
                MinioFileUploadInfoParam saveParam = new MinioFileUploadInfoParam();
                saveParam.setUploadId(uploadInfo.getUploadId());
                saveParam.setFileMd5(param.getFileMd5());
                saveParam.setFileName(param.getFileName());
                saveParam.setTotalChunk((int) partCount);
                saveParam.setFileStatus(CommonEnums.MinioFileStatusEnum.UN_UPLOADED.ordinal());
                // 保存文件上传信息
                minioFileUploadInfoService.saveMinioFileUploadInfo(saveParam);

                MinioFileChunkUploadInfoParam chunkUploadInfoParam = new MinioFileChunkUploadInfoParam();
                chunkUploadInfoParam.setUploadUrls(uploadInfo.getUploadUrls());
                chunkUploadInfoParam.setUploadId(uploadInfo.getUploadId());
                chunkUploadInfoParam.setExpiryTime(uploadInfo.getExpiryTime());
                chunkUploadInfoParam.setFileMd5(param.getFileMd5());
                chunkUploadInfoParam.setFileName(param.getFileName());
                // 保存分片上传信息
                minioFileChunkUploadInfoService.saveMinioFileChunkUploadInfo(chunkUploadInfoParam);
            }
            return uploadInfo;
        }
        // 查询分片上传地址
        List<MinioFileChunkUploadInfoDTO> list = minioFileChunkUploadInfoService.listByFileMd5AndUploadId(minioFileUploadInfo.getFileMd5(), minioFileUploadInfo.getUploadId());
        List<String> uploadUrlList = list.stream().map(MinioFileChunkUploadInfoDTO::getChunkUploadUrl).collect(Collectors.toList());
        uploadInfo = new MinioUploadInfo();
        uploadInfo.setUploadUrls(uploadUrlList);
        uploadInfo.setUploadId(minioFileUploadInfo.getUploadId());
        return uploadInfo;
    }

    @Override
    public MinioOperationResult checkFileExistsByMd5(String md5) {
        MinioOperationResult result = new MinioOperationResult();
        MinioFileUploadInfoDTO minioFileUploadInfo = this.minioFileUploadInfoService.getByFileMd5(md5);
        if (null == minioFileUploadInfo) {
            result.setStatus(CommonEnums.MinioFileStatusEnum.UN_UPLOADED.ordinal());
            return result;
        }
        // 已上传
        if (minioFileUploadInfo.getFileStatus() == CommonEnums.MinioFileStatusEnum.UPLOADED.ordinal()) {
            result.setStatus(CommonEnums.MinioFileStatusEnum.UPLOADED.ordinal());
            result.setUrl(minioFileUploadInfo.getFileUrl());
            return result;
        }
        // 查询已上传分片列表并返回已上传列表
        List<Integer> chunkUploadedList = listUploadParts(minioFileUploadInfo.getFileName(), minioFileUploadInfo.getUploadId());
        result.setStatus(CommonEnums.MinioFileStatusEnum.UPLOADING.ordinal());
        result.setChunkUploadedList(chunkUploadedList);
        return result;
    }

    @Override
    public List<Integer> listUploadParts(String objectName, String uploadId) {
        return minioHelper.listUploadChunkList(objectName, uploadId);
    }

    @Override
    public String mergeMultipartUpload(MergeMinioMultipartParam param) {
        String result = minioHelper.mergeMultiPartUpload(param.getFileName(), param.getUploadId());
        if (!StringUtils.isEmpty(result)) {
            MinioFileUploadInfoParam fileUploadInfoParam = new MinioFileUploadInfoParam();
            fileUploadInfoParam.setFileUrl(result);
            fileUploadInfoParam.setFileMd5(param.getMd5());
            fileUploadInfoParam.setFileStatus(CommonEnums.MinioFileStatusEnum.UPLOADED.ordinal());

            // 更新状态
            minioFileUploadInfoService.updateFileStatusByFileMd5(fileUploadInfoParam);
        }
        return result;
    }
}

IMinioFileUploadInfoService和MinioFileUploadInfoServiceImpl

package com.example.minio.service;

import com.baomidou.mybatisplus.extension.service.IService;
import com.example.minio.dto.MinioFileUploadInfoDTO;
import com.example.minio.entity.FileUploadInfo;
import com.example.minio.param.MinioFileUploadInfoParam;

/**
 * @Author: HOUJL
 * @Date: 2023/2/22
 * @Description:
 */
public interface IMinioFileUploadInfoService extends IService<FileUploadInfo> {
    /**
     * 保存
     *
     * @param param 参数对象
     * @return
     */
    MinioFileUploadInfoDTO saveMinioFileUploadInfo(MinioFileUploadInfoParam param);

    /**
     * 修改文件状态
     *
     * @param param 参数对象
     * @return
     */
    int updateFileStatusByFileMd5(MinioFileUploadInfoParam param);

    /**
     * 根据文件 md5 查询
     *
     * @param fileMd5 文件 md5
     * @return
     */
    MinioFileUploadInfoDTO getByFileMd5(String fileMd5);
}

package com.example.minio.service.impl;

import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import com.example.minio.common.enums.MessageEnum;
import com.example.minio.common.exception.BusinessException;
import com.example.minio.common.utils.HongBeanUtils;
import com.example.minio.dao.IMinioFileUploadInfoMapper;
import com.example.minio.dto.MinioFileUploadInfoDTO;
import com.example.minio.entity.FileUploadInfo;
import com.example.minio.param.MinioFileUploadInfoParam;
import com.example.minio.service.IMinioFileUploadInfoService;
import org.springframework.beans.BeanUtils;
import org.springframework.stereotype.Service;

import java.util.Date;

/**
 * @Author: HOUJL
 * @Date: 2023/2/22
 * @Description:
 */
@Service
public class MinioFileUploadInfoServiceImpl extends ServiceImpl<IMinioFileUploadInfoMapper, FileUploadInfo> implements IMinioFileUploadInfoService {

    private final IMinioFileUploadInfoMapper minioFileUploadInfoMapper;

    public MinioFileUploadInfoServiceImpl(IMinioFileUploadInfoMapper minioFileUploadInfoMapper) {
        this.minioFileUploadInfoMapper = minioFileUploadInfoMapper;
    }

    @Override
    public MinioFileUploadInfoDTO saveMinioFileUploadInfo(MinioFileUploadInfoParam param) {
        FileUploadInfo minioFileUploadInfo = null;
        if (null == param.getId()) {
            minioFileUploadInfo = new FileUploadInfo();
        } else {
            minioFileUploadInfo = this.minioFileUploadInfoMapper.selectById(param.getId());
            if (null == minioFileUploadInfo) {
                throw new BusinessException(MessageEnum.RECORD_NOT_EXISTED);
            }
            minioFileUploadInfo.setUpdateTime(new Date());
        }
        BeanUtils.copyProperties(param, minioFileUploadInfo, "id");
        int result;
        if (null == param.getId()) {
            result = this.minioFileUploadInfoMapper.insert(minioFileUploadInfo);
        } else {
            result = this.minioFileUploadInfoMapper.updateById(minioFileUploadInfo);
        }
        if (result == 0) {
            throw new BusinessException(MessageEnum.FAIL);
        }
        return HongBeanUtils.doToDto(minioFileUploadInfo, MinioFileUploadInfoDTO.class);
    }

    @Override
    public int updateFileStatusByFileMd5(MinioFileUploadInfoParam param) {
        FileUploadInfo minioFileUploadInfo = this.minioFileUploadInfoMapper.getByFileMd5(param.getFileMd5());
        if (null == minioFileUploadInfo) {
            throw new BusinessException(MessageEnum.RECORD_NOT_EXISTED);
        }
        minioFileUploadInfo.setFileStatus(param.getFileStatus());
        minioFileUploadInfo.setFileUrl(param.getFileUrl());
        return this.minioFileUploadInfoMapper.updateById(minioFileUploadInfo);
    }

    @Override
    public MinioFileUploadInfoDTO getByFileMd5(String fileMd5) {
        FileUploadInfo minioFileUploadInfo = this.minioFileUploadInfoMapper.getByFileMd5(fileMd5);
        if (null == minioFileUploadInfo) {
            return null;
        }
        return HongBeanUtils.doToDto(minioFileUploadInfo, MinioFileUploadInfoDTO.class);
    }
}

IMinioFileChunkUploadInfoService和MinioFileChunkUploadInfoServiceImpl

package com.example.minio.service;

import com.baomidou.mybatisplus.extension.service.IService;
import com.example.minio.dto.MinioFileChunkUploadInfoDTO;
import com.example.minio.entity.MinioFileChunkUploadInfoDO;
import com.example.minio.param.MinioFileChunkUploadInfoParam;

import java.util.List;

/**
 * @Author: HOUJL
 * @Date: 2023/2/22
 * @Description:
 */
public interface IMinioFileChunkUploadInfoService extends IService<MinioFileChunkUploadInfoDO> {

    /**
     * 保存
     *
     * @param param
     * @return
     */
    boolean saveMinioFileChunkUploadInfo(MinioFileChunkUploadInfoParam param);

    List<MinioFileChunkUploadInfoDO> listByFileMd5(String fileMd5);

    List<MinioFileChunkUploadInfoDTO> listByFileMd5AndUploadId(String fileMd5, String uploadId);
}

package com.example.minio.service.impl;

import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import com.example.minio.common.utils.HongBeanUtils;
import com.example.minio.dao.IMinioFileChunkUploadInfoMapper;
import com.example.minio.dto.MinioFileChunkUploadInfoDTO;
import com.example.minio.entity.MinioFileChunkUploadInfoDO;
import com.example.minio.param.MinioFileChunkUploadInfoParam;
import com.example.minio.service.IMinioFileChunkUploadInfoService;
import org.springframework.stereotype.Service;

import java.util.ArrayList;
import java.util.List;

/**
 * @Author: HOUJL
 * @Date: 2023/2/22
 * @Description:
 */
@Service
public class MinioFileChunkUploadInfoServiceImpl extends ServiceImpl<IMinioFileChunkUploadInfoMapper, MinioFileChunkUploadInfoDO> implements IMinioFileChunkUploadInfoService {

    private final IMinioFileChunkUploadInfoMapper iMinioFileChunkUploadInfoMapper;

    public MinioFileChunkUploadInfoServiceImpl(IMinioFileChunkUploadInfoMapper iMinioFileChunkUploadInfoMapper) {
        this.iMinioFileChunkUploadInfoMapper = iMinioFileChunkUploadInfoMapper;
    }

    @Override
    public boolean saveMinioFileChunkUploadInfo(MinioFileChunkUploadInfoParam param) {
        List<MinioFileChunkUploadInfoDO> list = new ArrayList<>();
        for (int i = 0; i < param.getUploadUrls().size(); i++) {
            MinioFileChunkUploadInfoDO tempObj = new MinioFileChunkUploadInfoDO();
            tempObj.setChunkNumber(i + 1);
            tempObj.setFileMd5(param.getFileMd5());
            tempObj.setUploadId(param.getUploadId());
            tempObj.setExpiryTime(param.getExpiryTime());
            tempObj.setChunkUploadUrl(param.getUploadUrls().get(i));
            list.add(tempObj);
        }
        int result = this.iMinioFileChunkUploadInfoMapper.batchInsert(list);
        return result != 0;
    }

    @Override
    public List<MinioFileChunkUploadInfoDO> listByFileMd5(String fileMd5) {
        return iMinioFileChunkUploadInfoMapper.listByFileMd5(fileMd5);
    }

    @Override
    public List<MinioFileChunkUploadInfoDTO> listByFileMd5AndUploadId(String fileMd5, String uploadId) {
        List<MinioFileChunkUploadInfoDO> list = iMinioFileChunkUploadInfoMapper.listByFileMd5AndUploadId(fileMd5, uploadId);
        return HongBeanUtils.doListToDtoList(list, MinioFileChunkUploadInfoDTO.class);
    }
}
  • 具体实体和vo/dto/param见资源详细代码

参考

https://blog.csdn.net/lemon_TT/article/details/124675675

https://blog.csdn.net/lemon_TT/article/details/127090238

https://blog.csdn.net/qq_43437874/category_10562215.html?spm=1001.2014.3001.5482

https://blog.csdn.net/m0_46493080/article/details/128303690?spm=1001.2014.3001.5502

https://min.io/

https://min.io/download 中文开发文档：http://docs.minio.org.cn/docs/

榔娃
关注
  • 7
    点赞
  • 踩
  • 16
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
专栏目录
SpringBoot项目文件上传适配本地存储、MinIO存储、FastDfs存储
chenjp111111的博客
08-05 799
分布式文件上传本地存储、MinIO存储、FastDfs存储
java文件上传Minio——文件、文件分片上传
qq_45778079的博客
07-15 5824
关于文件上传Minio的一些基本介绍（如文件上传、分片上传、md5值秒传、断点续传等）； 本文主要介绍的是单文件上传minio、多文件分片上传minio的demo！其他上传方式请留言主页文章哦！
Java实战：Spring Boot利用MinIO实现文件切片上传
oandy0的博客
03-10 1430
本文将详细介绍如何在 Spring Boot 应用程序中使用 MinIO 实现文件切片极速上传技术。我们将深入探讨 MinIO 的概念、文件切片上传的原理，以及如何使用 Spring Boot 和 MinIO 实现文件切片上传和合并。
Minio入门系列【6】JAVA集成Minio之对象操作API使用详解
热门推荐
吴名氏的博客
09-19 6万+
Minio入门系列【6】JAVA集成Minio之对象操作API使用详解
Minio对象存储及Springboot整合Minio分片上传文章-代码
02-23
Minio对象存储及Springboot整合Minio分片上传文章----代码
SpringBoot(31) 整合MinIO实现文件上传与下载
08-17
SpringBoot(31) 整合MinIO实现文件上传与下载
springboot集成minio实现了分片上传功能源码+项目说明.zip
12-20
springboot集成minio实现了分片上传功能源码+项目说明.zip vue版本的：前端 后端 快速开始 后端 修改配置文件application.yml: minio: endpoint: accessKey: secretKey: bucketName: downloadUri: #配置下载的ip和端口 path: #如果生产环境配置nginx域名解析，这里可以配置分片上传的ip和端口或者域名 前端页面 修改frontend/js/upload.js: 'http://127.0.0.1:8080/file/multipart/create' 'http://127.0.0.1:8080/file/multipart/complete' 改为你的后端地址即可 使用 在系统文件下打开frontend/upload.html运行，选择大于5m的文件上传。
minio设置文件上传大小
大家好，我是云村的王子
11-04 4695
minio
Minio服务器如何设置默认文件大小
yang_weidong的博客
07-06 3741
Minio服务器如何设置默认文件大小
Minio入门系列【16】Minio分片上传文件putObject接口流程源码分析
记录知识、锤炼自我
10-26 3万+
前言 为了更好的理解和优化Minio文件上传，本篇文档对MInio中上传文件putObject接口源码分析以下。 基于Java 客户端 API Controller层上传文件接口： @PostMapping("/upload") @ResponseBody public Object upload(MultipartFile file, String bucketName) throws IOException { return minioTemplate.putOb
Springboot项目修改文件传输（minio）限制大小
linzi19900517的博客
07-28 7499
Springboot项目修改文件传输（minio）限制大小。
minio分片上传Demo.zip
07-04
minio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片上传Demo.zipminio分片
minio 安装步骤以及springboot 如何集成minio
08-07
包含minio 安装步骤、springboot 如何集成minio，封装的minio 操作工具类，增加方法设置桶的权限
Minio入门系列【18】Minio+vue-uploader 分片上传方案及案例详解（源码文尾附上）
记录知识、锤炼自我
03-15 1万+
文章目录前言优化方案案例1. 前言 我们之前分析过Minio 的上传接口源码，其是进行了分块，再上传分块到Minio 服务器，最后再对块进行合并。 针对大文件的上传，如果采用上传到文件服务，再上传到Minio，其效率是非常低的，首先上传到文件服务（会存放在Tomcat 临时目录）就已经比较慢了。 针对大文件的上传，我们需要一个优化方案。 优化方案 前端服务进行分片处理，将分片信息传递给文件服务，文件服务返回所有分片的上传链接及uploadId。 前端服务直接请求Minio 服务器，并发处理上传分片 所有
SpringBoot + minio实现分片上传、秒传、续传
2301_78526383的博客
06-28 1059
MinIO是一个基于Go实现的高性能、兼容S3协议的对象存储。它采用GNU AGPL v3开源协议，项目地址是https://github.com/minio/minio。引用官网：MinIO是根据GNU Affero通用公共许可证v3.0发布的高性能对象存储。它与Amazon S3云存储服务兼容。使用MinIO构建用于机器学习，分析和应用程序数据工作负载的高性能基础架构。官网地址：文档地址：
SpringBoot + minio实现文件切片极速上传技术（api调用和minio部署，不涉及底层逻辑））
IT小辉同学
06-02 6533
有朋友让我放下！我想，拿起来就已经很难，那放下大概也不会容易！！！
minio文件上传
CVPR收割机的博客
02-29 962
不然后端接收不到参数，导致debug也进不去。
spring boot - minio 分片上传
面朝大海，春暖花开
06-13 430
【代码】spring boot - minio 分片上传。
springboot整合minio分片
最新发布
03-27
当我们需要在Spring Boot项目中使用MinIO对象存储服务，并且需要支持文件分片上传时，可以按照以下步骤进行整合： 1. 添加MinIO和Spring Boot的依赖：在项目的pom.xml文件中添加MinIO和Spring Boot的依赖，例如： ```xml <dependency> <groupId>io.minio</groupId> <artifactId>minio</artifactId> <version>8.0.6</version> </dependency> ``` 2. 配置MinIO连接信息：在Spring Boot的配置文件（如application.properties或application.yml）中配置MinIO的连接信息，包括endpoint、accessKey和secretKey等，例如： ```yaml minio: endpoint: http://localhost:9000 access-key: your-access-key secret-key: your-secret-key ``` 3. 创建MinIO客户端：在Spring Boot的配置类中创建MinIO客户端的Bean，例如： ```java @Configuration public class MinioConfig { @Value("${minio.endpoint}") private String endpoint; @Value("${minio.access-key}") private String accessKey; @Value("${minio.secret-key}") private String secretKey; @Bean public MinioClient minioClient() { return MinioClient.builder() .endpoint(endpoint) .credentials(accessKey, secretKey) .build(); } } ``` 4. 实现文件分片上传：在业务逻辑中使用MinIO客户端进行文件分片上传，可以使用`putObject`方法将文件分片上传到MinIO服务器，例如： ```java @Autowired private MinioClient minioClient; public void uploadFile(String bucketName, String objectName, InputStream inputStream, long size) throws Exception { // 设置分片大小 long partSize = 5 * 1024 * 1024; // 5MB // 初始化分片上传 CreateMultipartUploadResponse response = minioClient.createMultipartUpload( CreateMultipartUploadArgs.builder() .bucket(bucketName) .object(objectName) .build()); String uploadId = response.uploadId(); // 分片上传 int partNumber = 1; long offset = 0; while (offset < size) { // 读取分片数据 byte[] data = new byte[(int) Math.min(partSize, size - offset)]; inputStream.read(data); // 上传分片 UploadPartResponse uploadPartResponse = minioClient.uploadPart( UploadPartArgs.builder() .bucket(bucketName) .object(objectName) .uploadId(uploadId) .partNumber(partNumber) .stream(new ByteArrayInputStream(data), data.length, -1) .build()); // 记录已上传的分片 Part part = Part.builder() .partNumber(partNumber) .eTag(uploadPartResponse.etag()) .build(); parts.add(part); partNumber++; offset += data.length; } // 完成分片上传 CompleteMultipartUploadResponse completeResponse = minioClient.completeMultipartUpload( CompleteMultipartUploadArgs.builder() .bucket(bucketName) .object(objectName) .uploadId(uploadId) .parts(parts) .build()); } ``` 以上就是Spring Boot整合MinIO实现文件分片上传的基本步骤。在实际应用中，还可以根据需求进行更多的配置和处理，例如设置文件分片大小、处理上传进度等。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值