简单入门Minio

Minio

通过b站视频老师学到的一个技术 总结我用到的地方 详细的可以查看链接
https://www.bilibili.com/video/BV1AY411N7te?p=1

一句话：分布式文件存储系统

简介：

MinIO是一个基于Apache License v2.0开源协议的对象存储服务。它兼容亚马逊S3云存储服务接口，非常适合于存储大容量非结构化的
数据，例如图片、视频、日志文件、备份数据和容器/虚拟机镜像等，而一个对象文件可以是任意大小， 从几kb到最大5T不等。
MinlO是一个非常轻量的服务, 可以很简单的和其他应用的结合，类似NodeJS, Redis或者MySQL。

官网：

https://min.io/

基本概念

Object： 基本对象，要存储的东西

Bucket：存储数据的逻辑空间 相互独立 可以理解为顶层目录

Drive: 存储数据的磁盘 Minio启动时 以参数的方式传入

Set: 一组Drive的集合

分布式部署根据集群规模自动划分一个或多个Set，每个Set中的Drive分布在不同位置。一个对象存储在一个Set上。(For example: 1..646} is divided into 4 sets each of size 16.)
- 一个对象存储在一个Set上
- 一个集群划分为多个Set
- 一个Set包含的Drive数量是固定的,默认由系统根据集群规模自动计算得出
- 一个SET中的Drive尽可能分布在不同的节点上

纠删码EC (Erasure Code)

MinIO使用纠删码机制来保证高可靠性,使用highwayhash来处理数据损坏( Bit Rot Protection)。关于纠删码，简单来说就是可以通过数学计算，把丢失的数据进行还原，它可以将n份原始数据,增加m份数据，并能通过n+m份中的任意n份数据，还原为原始数据。即如果有任意小于等于m份的数据失效，仍然能通过剩下的数据还原出来。

环境搭建

文档：https://docs.min.io/

mac

wget https://dl.min.io/server/minio/release/darwin-amd64/minio
chmod +x minio
./minio server /data

linux

wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio

export MINIO_ROOT_USER = 用户名
export MINIO_ROOT_PASSWORD = 八位密码

# 默认配置目录：${HOME}/.minio  可以通过 --config-dir 自定义目录

./minio server --console-address ":9001" /data(文件目录)

使用docker安装

mkdir -p /mydocker/minio/data /mydocker/minio/config

docker run -d \
  -p 9000:9000 \
  -p 9001:9001 \
  --name minio \
  -v /mydocker/minio/data:/data \
  -v /mydocker/minio/config:/root/.minio \
  -e "MINIO_ROOT_USER=admin" \
  -e "MINIO_ROOT_PASSWORD=12345678" \
  quay.io/minio/minio server /data --console-address ":9001"

访问9001 即可打开可视化面板

客户端使用

https://docs.min.io/docs/minio-client-quickstart-guide.html

Linux安装

wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
./mc --help

windows安装

https://dl.min.io/client/mc/release/windows-amd64/mc.exe

mac

brew install minio/stable/mc
mc --help

配置mc

默认配置文件在 ~/.mc/config.json

# 查询host配置
mc config host ls

# 添加minio服务
mc config host add <自定义名字> http://ip:port 用户名 密码

# 移除服务
mc config host remove <自定义名字>

命令

alias       set, remove and list aliases in configuration file
ls          list buckets and objects
mb          make a bucket
rb          remove a bucket
cp          copy objects
mirror      synchronize object(s) to a remote site
cat         display object contents
head        display first 'n' lines of an object
pipe        stream STDIN to an object
share       generate URL for temporary access to an object
find        search for objects
sql         run sql queries on objects
stat        show object metadata
mv          move objects
tree        list buckets and objects in a tree format
du          summarize disk usage recursively
retention   set retention for object(s)
legalhold   set legal hold for object(s)
diff        list differences in object name, size, and date between two buckets
rm          remove objects
encrypt    manage bucket encryption config
event       manage object notifications
watch       listen for object notification events
undo        undo PUT/DELETE operations
policy      manage anonymous access to buckets and objects
tag         manage tags for bucket(s) and object(s)
ilm         manage bucket lifecycle
version     manage bucket versioning
replicate   configure server side bucket replication
admin       manage MinIO servers
update      update mc to latest release

查阅文档！https://docs.min.io/docs/minio-client-complete-guide.html

集成Springboot

<!--自带的okhttp版本号有问题  所以需要排除并且再引入一个-->       
<dependency>
  <groupId>io.minio</groupId>
  <artifactId>minio</artifactId>
  <version>8.3.9</version>
  <exclusions>
    <exclusion>
      <groupId>com.squareup.okhttp3</groupId>
      <artifactId>okhttp</artifactId>
    </exclusion>
  </exclusions>
</dependency>
<dependency>
  <groupId>com.squareup.okhttp3</groupId>
  <artifactId>okhttp</artifactId>
  <version>4.9.0</version>
</dependency>

@Configuration
public class MinioConfig {

    @Autowired
    private MinioProperties minioProperties;

    @Bean
    public MinioClient minioClient(){

        MinioClient minioClient = MinioClient.builder()
                .endpoint(minioProperties.getEndpoint())
                .credentials(minioProperties.getAccessKey(), minioProperties.getSecretKey())
                .build();

        return minioClient;
    }

}

@Component
@Data
@ConfigurationProperties(prefix = "minio")
public class MinioProperties {

    private String endpoint;
    private String accessKey;
    private String secretKey;
}

minio:
  endpoint: http://服务器的ip:端口号（默认9000）
  accessKey: 用户名
  secretKey: 密码
  bucketName: 空间名称

package com.example.demo.controller;

import com.alibaba.fastjson.JSON;
import io.minio.GetObjectArgs;
import io.minio.ListObjectsArgs;
import io.minio.MinioClient;
import io.minio.PutObjectArgs;
import io.minio.RemoveObjectArgs;
import io.minio.Result;
import io.minio.StatObjectArgs;
import io.minio.StatObjectResponse;
import io.minio.messages.Item;
import org.apache.commons.compress.utils.IOUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;

import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.InputStream;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;

/**
 * @author zhangan
 * @date 2022/4/29 00:43
 */
@RestController
public class MinioController {

    @Autowired
    private MinioClient minioClient;

    @Value("${minio.bucketName}")
    private String bucketName;

    /**
     * 列出所有文件
     * @return 列出所有文件
     */
    @GetMapping("/list")
    public List<Object> list() throws Exception {
        Iterable<Result<Item>> results = minioClient.listObjects(ListObjectsArgs.builder().bucket(bucketName).build());
        Iterator<Result<Item>> iterator = results.iterator();
        List<Object> items = new ArrayList<>();
        String format = "{'fileName':'%s','fileSize':'%s'}";
        while (iterator.hasNext()) {
            Item item = iterator.next().get();
            items.add(JSON.parse(String.format(format, item.objectName(), item.size())));
        }

        return items;
    }

    /**
     *  下载文件
     * @param response 返回
     * @param fileName 文件名
     */
    @RequestMapping("/download/{fileName}")
    public void download(HttpServletResponse response,
                         @PathVariable("fileName") String fileName) {
        InputStream inputStream = null;
        StatObjectResponse statObjectResponse = null;
        try {
            // 获取对象信息
            statObjectResponse = minioClient.statObject(StatObjectArgs.builder()
                    .bucket(bucketName).object(fileName).build());
            response.setContentType(statObjectResponse.contentType());
            response.setHeader("Content-Disposition", "attachment;filename=" +
                    URLEncoder.encode(fileName, StandardCharsets.UTF_8));

            // 文件下载
            inputStream = minioClient.getObject(GetObjectArgs.builder()
                    .bucket(bucketName).object(fileName).build());
            IOUtils.copy(inputStream, response.getOutputStream());

        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (inputStream != null) {
                try {
                    inputStream.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }

    /**
     * 删除文件
     * @param fileName 文件名
     * @return
     */
    @DeleteMapping("/delete/{fileName}")
    public String delete(@PathVariable("fileName") String fileName) {
        try {
            minioClient.removeObject(RemoveObjectArgs.builder()
                    .bucket(bucketName).object(fileName).build());
        } catch (Exception e) {
            return "失败";
        }
        return "成功";
    }

    /**
     * 上传文件
     * @param file
     * @return
     */
    @PostMapping("/upload")
    public String upload(@RequestParam(name = "file", required = false) MultipartFile[] file) {

        if (file == null || file.length == 0) {
            return "上传文件不能为空";
        }
        for (MultipartFile multipartFile : file) {
            String originalFilename = multipartFile.getOriginalFilename();
            InputStream inputStream = null;
            try {
                inputStream = multipartFile.getInputStream();
                minioClient.putObject(PutObjectArgs.builder()
                        .bucket(bucketName).object(originalFilename)
                        .stream(inputStream, multipartFile.getSize(), -1).build());
            } catch (Exception e) {
                return "上传失败";
            } finally {
                if (inputStream != null) {
                    try {
                        inputStream.close();
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }
            }

        }
        return "上传成功";
    }
}