【s3cmd】

一个用来操作s3接口的客户端。

1. 安装(文章以ubuntu为操作环境)
1.1 tar包安装

提前安装依赖工具：

apt-get install python-dateutil
apt-get install setuptools

下载：

https://github.com/s3tools/s3cmd/releases
wget https://github.com/s3tools/s3cmd/releases/download/v2.3.0/s3cmd-2.3.0.tar.gz
tar -xvf s3cmd-2.3.0.tar.gz
cd  s3cmd-2.3.0
python setup.py install

1.2 直接安装(所需依赖会一步安装)

apt-get install s3cmd

2. 配置与常见错误

配置Access Key ID 和 Secret Access Key

s3cmd --configure

博主实操所遇问题如下：

Invalid header value '{"code":-1,"message":"cluster.internal","detail":"{\\"code\\":500,\\"type\\":\\"UNKNOWN\\",\age\\":\\"Http\xe5\x93\x8d\xe5\xba\x94\xe9\x94\x99\xe8\xaf\xaf\\",\\"cause\\":{\\"err\\":\\"404 page not found\\\\n\\"},\\"details\\":null}"

上述问题可以通过 vim ~/.s3cfg解决，配置文件内容如下:

[default]
access_key = access key
secret_key = secret key
access_token = 
add_encoding_exts = 
add_headers = 
bucket_location = cn
ca_certs_file = 
cache_file = 
check_ssl_certificate = True
check_ssl_hostname = True
cloudfront_host = cloudfront.amazonaws.com
default_mime_type = binary/octet-stream
delay_updates = False
delete_after = False
delete_after_fetch = False
delete_removed = False
dry_run = False
enable_multipart = True
encrypt = False
expiry_date = 
expiry_days = 
expiry_prefix = 
follow_symlinks = False
force = False
get_continue = False
gpg_command = /usr/bin/gpg
gpg_decrypt = %(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
gpg_encrypt = %(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
gpg_passphrase = 
guess_mime_type = True
host_base = s3.cn-north-1.jcloudcs.com
host_bucket = %(bucket)s.s3.cn-north-1.jcloudcs.com
human_readable_sizes = False
invalidate_default_index_on_cf = False
invalidate_default_index_root_on_cf = True
invalidate_on_cf = False
kms_key = 
limit = -1
limitrate = 0
list_md5 = False
log_target_prefix = /home/eric/jcloud/s3.log
long_listing = False
max_delete = -1
mime_type = 
multipart_chunk_size_mb = 15
multipart_max_chunks = 10000
preserve_attrs = True
progress_meter = True
proxy_host = 
proxy_port = 0
put_continue = False
recursive = False
recv_chunk = 65536
reduced_redundancy = False
requester_pays = False
restore_days = 1
restore_priority = Standard
send_chunk = 65536
server_side_encryption = False
signature_v2 = False
simpledb_host = sdb.amazonaws.com
skip_existing = False
socket_timeout = 300
stats = False
stop_on_error = False
storage_class = 
urlencoding_mode = normal
use_http_expect = False
use_https = True
use_mime_magic = True
verbosity = WARNING
website_endpoint = http://%(bucket)s.s3.%(location)s.amazonaws.com/
website_error = 
website_index = index.html

下图问题要检查下配置文件在key的空间授权

下图问题是由于命令不对，正确命令是s3cd mb s3://test3/

3. 常用命令
列举所有的Buckets

s3cmd ls

创建 bucket，且 bucket 名称是唯一的，不能重复。

s3cmd mb s3://my-bucket-name

删除空 bucket

s3cmd rb s3://my-bucket-name

列举 Bucket 中的内容

s3cmd ls s3://my-bucket-name

上传 file.txt 到某个 bucket(忽略文件大小，自动分片，单片默认15M)
上传大文件时，使用 --multipart-chunk-size-mb=size 指定的分片大小必须是4的倍数，否则上传会报 400（InvalidPartOrder）

s3cmd put file.txt s3://my-bucket-name/file.txt

上传并将权限设置为所有人可读(--acl-private，也可以是私有)

s3cmd put --acl-public file.txt s3://my-bucket-name/file.txt

批量上传文件

s3cmd put ./* s3://my-bucket-name/

下载文件（支持批量下载，直接指定多个文件，如果下载终断，比如ctrl+c，带上参数--continue,可以实现断点下载）

s3cmd get s3://my-bucket-name/file.txt file.txt
s3cmd get s3://tccpoc/t.py s3://tccpoc/up.py

批量下载

s3cmd get s3://my-bucket-name/* ./

删除文件(支持批量删除，直接指定多个 bucket 对象)

s3cmd del s3://my-bucket-name/file.txt
s3cmd del s3://my-bucket-name/file.txt s3://my-bucket-name/file2.txt

获取对应的bucket所占用的空间大小

s3cmd du -H s3://my-bucket-name