MinIO 初探

MinIO

一 概述

MinIO 是一个高性能的对象存储原生支持 Kubernetes 部署的解决方案。 MinIO 提供了一个 Amazon Web Services S3 兼容 API 并支持所有核心 S3 功能。协议 GNU AGPL v3. 使用Go语言开发, 有各种其他语言API, (C++,JAVA, JS 等)

分布式的Minio可以让你将多块硬盘（甚至在不同的机器上）组成一个对象存储服务。由于硬盘分布在不同的节点上，分布式Minio避免了单点故障。它非常适合支持大量非结构化的数据，而一个文件可以是任意大小，从几十k到最大的5TB，同时他也是一个非常轻量的服务，可以结合其他的应用进行使用。

相关官方资源(en)

• Community - https://slack.min.io
• Documentation - https://docs.min.io
• Blog - https://blog.min.io
• Website - https://min.io

二 研究目标:

1: MinIO 环境服务端和客户端 编译安装配置, 完成文件上传和下载命令行实验.

2: 直接通过 API 不落地访问服务端文件- 输出test程序

三 实验

实验1: 编译安装配置,资源上传下载

服务端源码 Server 下载地址地址 :https://github.com/minio/minio/releases/tag/RELEASE.2022-07-06T20-29-49Z,

解压源码编译命令go env, go build ,(最低go环境 1.17)

环境配置可参考此篇blog https://blog.csdn.net/chijingjing/article/details/124686479?spm=1001.2014.3001

服务端编译好的exe下载 https://dl.min.io/server/minio/release/windows-amd64/

运行minio.exe 提示

PS D:\opensource\openminio\minio-RELEASE.2022-07-06T20-29-49Z> .\minio.exe
NAME:
  minio.exe - High Performance Object Storage

DESCRIPTION:
  Build high performance data infrastructure for machine learning, analytics and application data workloads with MinIO

USAGE:
  minio.exe [FLAGS] COMMAND [ARGS...]

COMMANDS:
  server   start object storage server
  gateway  start object storage gateway

FLAGS:
  --certs-dir value, -S value  path to certs directory (default: "C:\\Users\\chijing\\.minio\\certs")
  --quiet                      disable startup and info messages
  --anonymous                  hide sensitive information from logging
  --json                       output logs in JSON format
  --help, -h                   show help
  --version, -v                print the version

VERSION:
  DEVELOPMENT.GOGET
PS D:\opensource\openminio\minio-RELEASE.2022-07-06T20-29-49Z>

c++客户端 https://github.com/minio/minio-cpp

开启服务:

单机:

minio.exe server D:\opensource\openminio\dir\dir{1…64}

PS D:\opensource\openminio\minio-RELEASE.2022-07-06T20-29-49Z> .\minio.exe server start D:\opensource\openminio\dir\
Formatting 1st pool, 1 set(s), 2 drives per set.
WARNING: Host local has more than 1 drives of set. A host failure will result in data becoming unavailable.
MinIO Object Storage Server
Copyright: 2015-0000 MinIO, Inc.
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Version: DEVELOPMENT.GOGET (go1.18 windows/amd64)

Status:         2 Online, 0 Offline.
API: http://192.168.31.29:9000  http://169.254.217.25:9000  http://169.254.122.10:9000  http://127.0.0.1:9000
RootUser: minioadmin
RootPass: minioadmin

WARNING: Console endpoint is listening on a dynamic port (35381), please use --console-address ":PORT" to choose a static port.
Console: http://192.168.31.29:35381 http://169.254.217.25:35381 http://169.254.122.10:35381 http://127.0.0.1:35381
RootUser: minioadmin
RootPass: minioadmin

Command-line: https://docs.min.io/docs/minio-client-quickstart-guide
   $ mc.exe alias set myminio http://192.168.31.29:9000 minioadmin minioadmin

Documentation: https://docs.min.io

.\minio.exe server start D:\opensource\openminio\dir\ --console-address “192.168.31.17:9001”

默认用户密码 : minioadmin/minioadmin, 必须在http://192.168.31.17:9001/identity/users 上修改ACCESSKEY 否则无法连接

实验2

下载和获取文件流

环境安装
go env
go get github.com/minio/minio-go/v7

package main

import (
	"context"
	"io"
	"log"
	"os"

	"github.com/minio/minio-go/v7"
	"github.com/minio/minio-go/v7/pkg/credentials"
)

func main() {
	s3Client, err := minio.New("192.168.31.17:9000", &minio.Options{
		Creds:  credentials.NewStaticV4("chijingjing", "chijingjing", ""),
		Secure: false,
	})
	if err != nil {
		log.Fatalln(err)
	}
	//直接下载整个文件 参数 桶名称,对象名称,保存路径
	if err := s3Client.FGetObject(context.Background(), "ttt", "BOU1_4M_S.fcs", "D:\\a.fcs", minio.GetObjectOptions{}); err != nil {
		log.Fatalln(err)
	}
	log.Println("Successfully saved BOU1_4M_S.fcs")

	//打开文件流
	reader, err := s3Client.GetObject(context.Background(), "ttt", "BOU1_4M_S.fcs", minio.GetObjectOptions{})
	if err != nil {
		log.Fatalln(err)
	}
	defer reader.Close()

	localFile, err := os.Create("D:\\b.fcs")
	if err != nil {
		log.Fatalln(err)
	}
	defer localFile.Close()

	stat, err := reader.Stat()
	if err != nil {
		log.Fatalln(err)
	}

	if _, err := io.CopyN(localFile, reader, stat.Size/2); err != nil {
		log.Fatalln(err)
	}
	log.Println("Successfully saved b.fcs")
}

四 结论

能力总结:

可支持文件上传下载, 并且支持流式文件访问

资原总结:

服务端源码 Server 下载地址地址 :https://github.com/minio/minio/releases/tag/RELEASE.2022-07-06T20-29-49Z,编译参考官网,

或者此篇blog https://blog.csdn.net/chijingjing/article/details/124686479?spm=1001.2014.3001

服务端编译好的exe下载 https://dl.min.io/server/minio/release/windows-amd64/

c++客户端 https://github.com/minio/minio-cpp