aws cli 上传文件到s3命令,节流与AWS CLI命令S3

I'm running a backup script using AWS CLI to perform an S3 sync command every night on my MediaTemple server. This has run without fail for months, but I updated my Plesk installation and now every night, when the backup script runs, MediaTemple disables my server due to excessive usage. The limits I seem to be crossing are as follows:

RESOURCE INFO:

Packets per second limit: 35000

Packets per second detected: 42229.11667000000306870788

Bytes per second limit: 50000000

Bytes per second detected: 61801446.10000000149011611938

They also include a networking snapshot at the time they take the server offline which includes many open connections to Amazon IP addresses (9 at time of the snapshot).

Is there anything I can do to throttle the connections to AWS? Preferably I'm looking for an option within the AWS API (though I haven't seen anything useful in the documentation), but barring that, is there something I can do on my end to manage the connections at the network level?

解决方案

I ended up using Trickle and capping download & upload speeds at 20,000 kb/s. This let me use my existing script without much modification (all I had to do was add the trickle call to the beginning of the command).

Also, it looks like bandwidth throttling has been added as an issue to AWS CLI, so hopefully this will all be a non-issue for folks if that gets implemented.

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值