(1) S3 connect reset by peer

当从S3获取并解压tar包到ECFS时,遇到报错并中断,问题频繁发生。分析发现与网络相关,特别是TCP连接被reset。原因是VFS缓存满导致socket接收缓冲区满,TCP连接被重置。解决方案是使用xbstream避免数据断断续续,因为它直接写入磁盘,不会导致缓冲区满。
摘要由CSDN通过智能技术生成

问题描述 && 初步分析定位

问题描述

s3cmd -q get s3://zh_test/data_30.tar - | tar -xvf - -C ./

在使用上面的命令,从S3中get 一个tar包,并同时解压到本地ECFS分布式存储的卷中时,会发生报错,导致get中断。

问题复现频率非常高

s3cmd 日志


#s3cmd -q get s3://zh_test/data_30.tar - | tar -xvf - -C ./

data_30G



!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

    An unexpected error has occurred.

  Please try reproducing the error using

  the latest s3cmd code from the git master

  branch found at:

    https://github.com/s3tools/s3cmd

  and have a look at the known issues list:

    https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions

  If the error persists, please report the

  following lines (removing any private

  info as necessary) to:

   s3tools-bugs@lists.sourceforge.net





!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!



Invoked as: /usr/bin/s3cmd -q get s3://zh_test/data_30.tar -

Problem: error: [Errno 104] Connection reset by peer

S3cmd: 2.0.2

python: 2.7.5 (default, May 3 2017, 07:55:04)

[GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]

environment LANG=en_US.UTF-8



Traceback (most recent call last):

  File "/usr/bin/s3cmd", line 3092, in <module>

    rc = main()

  File "/usr/bin/s3cmd", line 3001, in main

    rc = cmd_func(args)

  File "/usr/bin/s3cmd", line 590, in cmd_object_get

    response = s3.object_get(uri, dst_stream, destination, start_position = start_position, extra_label = seq_label)

  
  • 1
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值