MinIO 部署及使用

环境信息

● Ubuntu 20.04.4 LTS
● Docker version 20.10.12
● Docker Compose version v2.6.1

安装及部署

安装docker

1、安装docker
sudo apt-get install -y docker.io

2、启动docker服务：
systemctl start docker

3、设置开机启动：
systemctl enable docker

4、查看docker状态：
systemctl status docker

安装docker-compose

Docker Compose存放在Git Hub，不太稳定。
你可以通过执行下面的命令，高速安装Docker Compose。

curl -L https://get.daocloud.io/docker/compose/releases/download/v2.6.1/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

安装部署MinIO

编辑docker-compose.yaml文件

官方推荐地址：下载地址
我们稍作修改如下：

version: '3.7'

# Settings and configurations that are common for all containers
x-minio-common: &minio-common
  image: minio/minio
  command: server --console-address ":9001" http://minio{1...4}/data
  expose:
    - "9000"
  # environment:
    # MINIO_ROOT_USER: minioadmin
    # MINIO_ROOT_PASSWORD: minioadmin
  healthcheck:
    test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
    interval: 30s
    timeout: 20s
    retries: 3

# starts 4 docker containers running minio server instances.
# using nginx reverse proxy, load balancing, you can access
# it through port 9000.
services:
  minio1:
    <<: *minio-common
    hostname: minio1
    ports:
      - "9001:9001"
    volumes:
      - .data/data1:/data

  minio2:
    <<: *minio-common
    hostname: minio2
    ports:
      - "9002:9001"
    volumes:
      - ./data/data2:/data

  minio3:
    <<: *minio-common
    hostname: minio3
    ports:
      - "9003:9001"
    volumes:
      - ./data/data3:/data

  minio4:
    <<: *minio-common
    hostname: minio4
    ports:
      - "9004:9001"
    volumes:
      - ./data/data4:/data

  nginx:
    image: nginx:1.19.2-alpine
    hostname: nginx
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "9000:9000"
    depends_on:
      - minio1
      - minio2
      - minio3
      - minio4

编辑nginx.conf配置文件

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  4096;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    keepalive_timeout  65;

    # include /etc/nginx/conf.d/*.conf;

    upstream minio {
        server minio1:9000;
        server minio2:9000;
        server minio3:9000;
        server minio4:9000;
    }

    server {
        listen       9000;
        listen  [::]:9000;
        server_name  localhost;

        # To allow special characters in headers
        ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
        client_max_body_size 0;
        # To disable buffering
        proxy_buffering off;

        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;

            proxy_connect_timeout 300;
            # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            chunked_transfer_encoding off;

            proxy_pass http://minio;
        }
    }

}

执行启动命令，看到各个节点healthy状态即成功

docker-compose up -d

MinIO 客户端SDK使用

MinIO Client（mc）

MinIO Client (mc)为ls，cat，cp，mirror，diff，find等UNIX命令提供了一种替代方案。它支持文件系统和兼容Amazon S3的云存储服务（AWS Signature v2和v4）。

Copyls       列出文件和文件夹。
mb       创建一个存储桶或一个文件夹。
cat      显示文件和对象内容。
pipe     将一个STDIN重定向到一个对象或者文件或者STDOUT。
share    生成用于共享的URL。
cp       拷贝文件和对象。
mirror   给存储桶和文件夹做镜像。
find     基于参数查找文件。
diff     对两个文件夹或者存储桶比较差异。
rm       删除文件和对象。
events   管理对象通知。
watch    监听文件和对象的事件。
policy   管理访问策略。
session  为cp命令管理保存的会话。
config   管理mc配置文件。
update   检查软件更新。
version  输出版本信息。

MinIO Client（mc）提供了“ admin”子命令来对您的MinIO部署执行管理任务。

Copyservice     服务重启并停止所有MinIO服务器
update      更新更新所有MinIO服务器
info        信息显示MinIO服务器信息
user        用户管理用户
group       小组管理小组
policy      MinIO服务器中定义的策略管理策略
config      配置管理MinIO服务器配置
heal        修复MinIO服务器上的磁盘，存储桶和对象
profile     概要文件生成概要文件数据以进行调试
top         顶部提供MinIO的顶部统计信息
trace       跟踪显示MinIO服务器的http跟踪
console     控制台显示MinIO服务器的控制台日志
prometheus  Prometheus管理Prometheus配置
kms         kms执行KMS管理操作

Java SDK

MinIO Java Client SDK提供简单的API来访问任何与Amazon S3兼容的对象存储服务。

Java 1.8或更高版本:

● OracleJDK 8.0
● OpenJDK8.0

pom文件中增加maven依赖

<dependency>
    <groupId>io.minio</groupId>
    <artifactId>minio</artifactId>
    <version>7.0.2</version>
</dependency>

编写操作minio代码

/**
 * 检查桶是否存在，不存在则创建桶
 * 使用putObject上传一个文件到存储桶
 *
 */
public static void main(String[] args) throws NoSuchAlgorithmException, IOException, InvalidKeyException, XmlPullParserException {
    try {
        // 使用MinIO服务的URL，端口，Access key和Secret key创建一个MinioClient对象
        MinioClient minioClient = new MinioClient("http://124.71.229.30:9000/", "minioadmin", "minioadmin");

        // 检查存储桶是否已经存在
        boolean isExist = minioClient.bucketExists("asiatrip");
        if(isExist) {
            System.out.println("Bucket already exists.");
        } else {
            // 创建一个名为asiatrip的存储桶，用于存储照片的zip文件。
            minioClient.makeBucket("asiatrip");
        }

        // 使用putObject上传一个文件到存储桶中。
        minioClient.putObject("asiatrip","Dism++10.1.1000.100_2d2bf466baca088c4b35248f5a7316f4e00cac0b.zip", "D:\\Dism++10.1.1000.100_2d2bf466baca088c4b35248f5a7316f4e00cac0b.zip");
        System.out.println("D:\\Dism++10.1.1000.100_2d2bf466baca088c4b35248f5a7316f4e00cac0b.zip is successfully uploaded as xxxx.zip to `asiatrip` bucket.");
    } catch(MinioException e) {
        System.out.println("Error occurred: " + e);
    }
}


/**
 * 删除桶内对象
 */
public void delete(String objectName) {
    try {
        MinioClient minioClient = new MinioClient(ENDPOINT, ACCESS_KEY, SECRET_KEY);
        minioClient.removeObject(BUCKET_NAME, objectName);

    } catch (Exception e) {
        e.printStackTrace();
    }
}

.NET SDK

引入Nuget包

Minio.AspNetCore

配置appsettings文件

"Minio": {
  "Endpoint": "127.0.0.1:9000",
  "Region": "127.0.0.1",
  "AccessKey": "minioadmin",
  "SecretKey": "minioadmin",
  "BucketName": "omsfile",
  "FileURL": "http://127.0.0.1:9000/file"
}

添加服务

services.AddMinio(options =>
{
     options.Endpoint = Configuration["Minio:Endpoint"];  // 这里是在配置文件接口访问域名
     options.Region = Configuration["Minio:Region"];  // 地址
     options.AccessKey = Configuration["Minio:AccessKey"];  // 用户名
     options.SecretKey = Configuration["Minio:SecretKey"];   // 密码

});

服务接口 IMinIOService

public interface IMinIOService
{
    /// <summary>
    /// 上传
    /// </summary>
    /// <param name="file">文件</param>
    /// <returns></returns>
    Task<Result<FileModel>> UploadAsync(FormFileCollection file);

    /// <summary>
    /// 上传图片
    /// </summary>
    /// <param name="file">文件</param>
    /// <returns></returns>

    Task<Result<FileModel>> UploadImageAsync(FormFileCollection file);

    /// <summary>
    /// 上传pdf
    /// </summary>
    /// <param name="file"></param>
    /// <returns></returns>
    Task<Result<FileModel>> UploadPdf(Stream file);
}

服务接口实现 MinIOService

/// <summary>
/// 上传文件相关
/// </summary>
public class MinIOService : IMinIOService
    {
        public AppSetting _settings { get; }
        public IHostingEnvironment _hostingEnvironment { get; set; }
        public MinioClient _client { get; set; }
        public MinioOptions _minioOptions { get; set; }
        public MinIOService(IOptions<AppSetting> setting, IHostingEnvironment hostingEnvironment, MinioClient client, IOptions<MinioOptions> minioOptions)
        {
            _settings = setting.Value;
            _hostingEnvironment = hostingEnvironment;
            _client = client;
            _minioOptions = minioOptions.Value;
        }
        //获取图片的返回类型
        public static Dictionary<string, string> contentTypDict = new Dictionary<string, string> {
               {"bmp","image/bmp" },
               {"jpg","image/jpeg"},
               {"jpeg","image/jpeg"},
               {"jpe","image/jpeg"},
               {"png","image/png"},
               {"gif","image/gif"},
               {"ico","image/x-ico"},
               {"tif","image/tiff"},
               {"tiff","image/tiff"},
               {"fax","image/fax"},
               {"wbmp","image//vnd.wap.wbmp"},
               {"rp","image/vnd.rn-realpix"} };


        /// <summary>
        /// 上传图片
        /// </summary>
        /// <param name="file">文件</param>
        /// <returns></returns>

        public async Task<Result<FileModel>> UploadImageAsync(FormFileCollection file)
        {
            Result<FileModel> res = new Result<FileModel>(false, "上传失败");

            //获得文件扩展名
            string fileNameEx = System.IO.Path.GetExtension(file[0].FileName).Replace(".", "");

            //是否是图片，现在只能是图片上传 文件类型 或扩展名不一致则返回
            if (contentTypDict.Values.FirstOrDefault(c => c == file[0].ContentType.ToLower()) == null || contentTypDict.Keys.FirstOrDefault(c => c == fileNameEx) == null)
            {
                res.Msg = "图片格式不正确";
                return res;
            }
            else
                return await UploadAsync(file);
        }



        /// <summary>
        /// 上传
        /// </summary>
        /// <param name="file">文件</param>
        /// <returns></returns>

        public async Task<Result<FileModel>> UploadAsync(FormFileCollection file)
        {
            Result<FileModel> res = new Result<FileModel>(false, "上传失败");



            try
            {

                //存储桶名
                string bucketName = _settings.BucketName;

                Zhengwei.Minio.FileModel fileModel = new FileModel();
                await CreateBucket(bucketName);
                var newFileName = CreateNewFileName(bucketName, file[0].FileName);
                await _client.PutObjectAsync(bucketName, newFileName, file[0].OpenReadStream(), file[0].Length,
                    file[0].ContentType);
                fileModel.Url = $"{_settings.FileURL}{newFileName}";

                //是否是图片，是图片进行压缩
                if (contentTypDict.Values.Contains(file[0].ContentType.ToLower()))
                {
                    string path = $"{_hostingEnvironment.ContentRootPath}/wwwroot/imgTemp/";
                    if (!Directory.Exists(Path.GetDirectoryName(path)))
                        Directory.CreateDirectory(Path.GetDirectoryName(path));

                    var bImageName = $"{newFileName}";
                    var savepath = $"{path}{newFileName}";//保存绝对路径
                    #region 保存原图到本地
                    using (FileStream fs = System.IO.File.Create(path + newFileName))
                    {
                        file[0].CopyTo(fs);
                        fs.Flush();
                    }
                    #endregion

                    //#region 保存缩略图到本地
                    //var bUrlRes = TencentCloudImageHelper.GetThumbnailImage(240, newFileName, path);
                    //#endregion

                    //上传压缩图

                    using (var sw = new FileStream(savepath, FileMode.Open))
                    {
                        await _client.PutObjectAsync(bucketName, bImageName, sw, sw.Length,
             "image/jpeg");
                        fileModel.Url = $"{_settings.FileURL}{bImageName}";
                    }

                    if (Directory.Exists(Path.GetDirectoryName(path)))
                        Directory.Delete(Path.GetDirectoryName(path), true);

                }
                res.IsSuccess = true;
                res.Msg = "上传成功";
                res.Data = fileModel;
                return res;
            }
            catch (Exception e)
            {

                return res;
            }
        }

        public async Task<Result<FileModel>> UploadPdf(Stream file)
        {
            Result<FileModel> res = new Result<FileModel>(false, "上传失败");
            try
            {

                //存储桶名
                string bucketName = _settings.BucketName;

                FileModel fileModel = new FileModel();
                await CreateBucket(bucketName);
                var newFileName = CreateNewFileName(bucketName, "授权书.pdf");
                await _client.PutObjectAsync(bucketName, newFileName, file, file.Length,
                    "application/pdf");
                fileModel.Url = $"{_settings.FileURL}{newFileName}";

                res.IsSuccess = true;
                res.Msg = "上传成功";
                res.Data = fileModel;
                return res;
            }
            catch (Exception e)
            {

                return res;
            }
        }

        private async Task CreateBucket(string bucketName)
        {
            var found = await _client.BucketExistsAsync(bucketName);
            if (!found)
            {
                await _client.MakeBucketAsync(bucketName);
                //设置只读策略
                var pObj = new
                {
                    Version = "2012-10-17",
                    Statement = new[]
                    {
                       new
                       {
                           Effect = "Allow",
                           Principal = new
                           {
                               AWS = new [] {"*"}
                           },
                           Action = new [] {"s3:GetBucketLocation", "s3:ListBucket"},
                           Resource = new []
                           {
                               $"arn:aws:s3:::{bucketName}"
                           }
                       },
                       new
                       {
                           Effect = "Allow",
                           Principal = new
                           {
                               AWS = new [] {"*"}
                           },
                           Action = new [] {"s3:GetObject"},
                           Resource = new []
                           {
                               $"arn:aws:s3:::{bucketName}/*"
                           }
                       }
                   }
                };
                var po = JsonSerializer.Serialize(pObj);
                await _client.SetPolicyAsync(bucketName, po);
            }
        }

        private string CreateNewFileName(string bucketName, string oldFileName)
        {
            var dt = Guid.NewGuid().ToString().Replace("-", "").Substring(10) + DateTimeOffset.Now.ToUnixTimeSeconds();
            var extensions = Path.GetExtension(oldFileName);
            var newFileName = $"{bucketName}-{dt}{extensions}";
            return newFileName;
        }
    }

注入服务

services.AddSingleton<IMinIOService, MinIOService>();

其它需要的类

 public class Result<T>
    {
        public Result(bool isSuccess,string msg)
        {
            this.IsSuccess = isSuccess;
            this.Msg = msg;

        }
        public bool IsSuccess { get; set; }
        public string Code { get; set; }
        public string Msg { get; set; }
        /// <summary>
        /// 返回数据列表
        /// </summary>
        public Object Data { set; get; }
    }
    

public class FileModel
{
    public string Url { get; set; }
}

上传文件的api

[ApiController]
[Route("[controller]")]
public class FileManagerController : Controller
{
    public IMinIOService _minIOService { get; set; }
    public FileManagerController(IMinIOService minIOService)
    {
        this._minIOService = minIOService;
    }
    [Route("UploadImg")]
    /// <summary>
    /// 上传图片
    /// </summary>
    /// <returns></returns>
    [HttpPost]
    public async Task<Result<FileModel>> UploadImg(FormFileCollection file)
    {
        return await _minIOService.UploadImageAsync(file);
    }
}

启动swagger，选择要上传的文件上传，测试效果

上传后的文件

权限policy控制

创建policy

分配policy