上篇文章【SpringBoot通过Minio实现大文件分片上传】中,很多问到了关于分片上传,未使用过期的历史分片如何清理的问题,其实官方默认有策略会定时清理历史过期临时文件,查了一下发现这方便资料非常少,而且官方文档也没有说明,于是带着问题查询了下Minio相关的配置才引出这篇文章。
查询到的资料有限,本文章主要说明以下几个方便:
- 关于过期分片的清理问题,担心过多冗余分片占用磁盘空间,Minio是否会自动清理?
- 是否可自定义清理策略?
- Minio部署有哪些可自定义配置?如何配置?
1. 过期分片是否会自动清理
答案显而易见,临时上传的分片,会被定时清理。否则随着时间推移,垃圾定会越积越多。那么问题又来了,有些系统用户量大,系统调用频繁可能堆积速度快,那么是否可配置清理频率呢?
2. 清理策略是否可配置
查询到官方github已有优秀的用户反馈并提交了部分清理策略修改的PR请求(感兴趣的可查看:fix: allow configuring cleanup of stale multipart uploads),并已合并到主分支,如果使用可更新到该版本(RELEASE.2021-10-06T23-36-31Z)之后的新版本。
在此可查看此版本更新内容:
RELEASE.2021-10-06T23-36-31Z
其中包含一条即为:
- Allow configuring cleanup of stale multipart uploads. See (#13354) for more details.
此时,我们发现可通过配置部分清理策略,那么接下来看如果使用这些配置,以及一些其他配置,且Docker及原生部署是否有区别?Mc是否可设置?
3. 新增配置项解释
首先看下清理策略的配置包含哪些(英文描述挺简单,相信大家一定没问题):
配置参数 | Env参数名(后续说明如何使用) | 说明 |
---|---|---|
stale_uploads_expiry | MINIO_API_STALE_UPLOADS_EXPIRY | set to expire stale multipart uploads older than this value, defaults to 24 hours |
stale_uploads_cleanup_interval | MINIO_API_STALE_UPLOADS_CLEANUP_INTERVAL | set to change intervals when stale multipart uploads are expired, defaults to every 6 hours |
delete_cleanup_interval | MINIO_API_DELETE_CLEANUP_INTERVAL | set to change intervals when deleted objects are permanently deleted from “.trash” folder, defaults to every 5 minutes |
其他配置 | ||
requests_max | MINIO_API_REQUESTS_MAX | set the maximum number of concurrent requests, e.g. “1600” |
requests_deadline | MINIO_API_REQUESTS_DEADLINE | set the deadline for API requests waiting to be processed e.g. “1m” |
cluster_deadline | MINIO_API_CLUSTER_DEADLINE | set the deadline for cluster readiness check e.g. “10s” |
cors_allow_origin | MINIO_API_CORS_ALLOW_ORIGIN | set comma separated list of origins allowed for CORS requests e.g. “https://example1.com,https://example2.com” |
remote_transport_deadline | MINIO_API_REMOTE_TRANSPORT_DEADLINE | set the deadline for API requests on remote transports while proxying between federated instances e.g. “2h” |
list_quorum | MINIO_API_LIST_QUORUM | set the acceptable quorum expected for list operations e.g. “optimal”, “reduced”, “disk”, “strict”, defaults to “strict” |
replication_workers | MINIO_API_SECURE_CIPHERS | set the number of replication workers, defaults to 100 |
replication_failed_workers | MINIO_API_REPLICATION_WORKERS | set the number of replication workers for recently failed replicas, defaults to 4 |
transition_workers | MINIO_API_TRANSITION_WORKERS | set the number of transition workers, defaults to 100 |
由此新增的三个参数可看出,我们可配置过期分片清理的阈值及频率,增加了灵活性。
4. 配置使用
官方也有说明部分参数的使用,可查阅:minio-server-configuration-guide。接下来看下在Docker及原生部署中如何使用:
docker run -p 9000:9000 \
-e "MINIO_ACCESS_KEY=tuine" \
-e "MINIO_SECRET_KEY=ZtG8tN5zo9EzcKMi" \
-e "MINIO_API_STALE_UPLOADS_EXPIRY=24" \
minio/minio server /data
minio:
image: minio/minio:latest
container_name: minio
restart: always
command: server /data/
volumes:
- ./data/:/data
ports:
- "9000:9000"
environment:
- TZ=Asia/Shanghai
- MINIO_ACCESS_KEY=tuine
- MINIO_SECRET_KEY=ZtG8tN5zo9EzcKMi
- MINIO_API_STALE_UPLOADS_EXPIRY=24
export MINIO_SITE_REGION="us-west-0"
export MINIO_API_STALE_UPLOADS_EXPIRY=24
minio server /data
通过Mc设置,使用配置参数
mc admin config set myminio/ api requests_max=1600 stale_uploads_expiry=24
mc admin service restart myminio/
配置完成后,启动或者重启即可生效。
如果文章对你有用,欢迎点赞收藏,谢谢!如有不正确或不理解的欢迎评论指出。