【错误记录】What does “Connection Reset by Peer“ mean?

1. flink报错

Caused by: com.dtstack.flinkx.throwable.FlinkxRuntimeException: can't get file size from hdfs, file = hdfs://xxx/home//hive/warehouse/xxx.db/hive_odshp_15842_daily/20221225/.data/1d2d0bb788b617c9378dfdb4416e952d_349_0
	at com.dtstack.flinkx.connector.hdfs.sink.BaseHdfsOutputFormat.getCurrentFileSize(BaseHdfsOutputFormat.java:169) ~[flinkx-connector-hdfs-1.12-SNAPSHOT.jar:?]
	at com.dtstack.flinkx.sink.format.BaseFileOutputFormat.checkCurrentFileSize(BaseFileOutputFormat.java:137) ~[flinkx-core-1.12-SNAPSHOT.jar:?]
	at com.dtstack.flinkx.sink.format.BaseFileOutputFormat.writeSingleRecordInternal(BaseFileOutputFormat.java:128) ~[flinkx-core-1.12-SNAPSHOT.jar:?]
	at com.dtstack.flinkx.sink.format.BaseRichOutputFormat.writeSingleRecord(BaseRichOutputFormat.java:427) ~[flinkx-core-1.12-SNAPSHOT.jar:?]
	at com.dtstack.flinkx.sink.format.BaseRichOutputFormat.writeRecord(BaseRichOutputFormat.java:244) ~[flinkx-core-1.12-SNAPSHOT.jar:?]
	at com.dtstack.flinkx.sink.format.BaseRichOutputFormat.writeRecord(BaseRichOutputFormat.java:84) ~[flinkx-core-1.12-SNAPSHOT.jar:?]
	at com.dtstack.flinkx.sink.DtOutputFormatSinkFunction.invoke(DtOutputFormatSinkFunction.java:117) ~[flinkx-core-1.12-SNAPSHOT.jar:?]
	at org.apache.flink.table.runtime.operators.sink.SinkOperator.processElement(SinkOperator.java:72) ~[flink-table-blink_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:71) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:46) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:26) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:50) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:28) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at StreamExecCalc$56.processElement(Unknown Source) ~[?:?]
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:71) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:46) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:26) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:50) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:28) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.api.operators.StreamSourceContexts$ManualWatermarkContext.processAndCollect(StreamSourceContexts.java:317) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.api.operators.StreamSourceContexts$WatermarkContext.collect(StreamSourceContexts.java:411) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at com.dtstack.flinkx.source.DtInputFormatSourceFunction.run(DtInputFormatSourceFunction.java:135) ~[flinkx-core-1.12-SNAPSHOT.jar:?]
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:267) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
Caused by: java.io.IOException: DestHost:destPort 10.160.228.94:9000 , LocalHost:localPort hdfs/10.160.119.185:0. Failed on local exception: java.io.IOException: Connection reset by peer
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_60]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_60]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_60]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_60]
	at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:833) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:808) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1549) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.ipc.Client.call(Client.java:1491) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.ipc.Client.call(Client.java:1388) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) ~[hadoop-common-3.2.1U20.jar:?]
	at com.sun.proxy.$Proxy34.getBlockLocations(Unknown Source) ~[?:?]
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getBlockLocations(ClientNamenodeProtocolTranslatorPB.java:324) ~[hadoop-hdfs-client-3.2.1U20.jar:?]
	at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source) ~[?:?]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_60]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_60]
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) ~[hadoop-common-3.2.1U20.jar:?]
	at com.sun.proxy.$Proxy35.getBlockLocations(Unknown Source) ~[?:?]
	at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:864) ~[hadoop-hdfs-client-3.2.1U20.jar:?]
	at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:853) ~[hadoop-hdfs-client-3.2.1U20.jar:?]
	at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:842) ~[hadoop-hdfs-client-3.2.1U20.jar:?]
	at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1010) ~[hadoop-hdfs-client-3.2.1U20.jar:?]
	at org.apache.hadoop.hdfs.DistributedFileSystem$5.doCall(DistributedFileSystem.java:359) ~[hadoop-hdfs-client-3.2.1U20.jar:?]
	at org.apache.hadoop.hdfs.DistributedFileSystem$5.doCall(DistributedFileSystem.java:355) ~[hadoop-hdfs-client-3.2.1U20.jar:?]
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:367) ~[hadoop-hdfs-client-3.2.1U20.jar:?]
	at org.apache.hadoop.fs.FilterFileSystem.open(FilterFileSystem.java:164) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.fs.viewfs.ChRootedFileSystem.open(ChRootedFileSystem.java:285) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.fs.viewfs.ViewFileSystem.open(ViewFileSystem.java:500) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:912) ~[hadoop-common-3.2.1U20.jar:?]
	at com.dtstack.flinkx.connector.hdfs.sink.BaseHdfsOutputFormat.getCurrentFileSize(BaseHdfsOutputFormat.java:166) ~[flinkx-connector-hdfs-1.12-SNAPSHOT.jar:?]
	at com.dtstack.flinkx.sink.format.BaseFileOutputFormat.checkCurrentFileSize(BaseFileOutputFormat.java:137) ~[flinkx-core-1.12-SNAPSHOT.jar:?]
	at com.dtstack.flinkx.sink.format.BaseFileOutputFormat.writeSingleRecordInternal(BaseFileOutputFormat.java:128) ~[flinkx-core-1.12-SNAPSHOT.jar:?]
	at com.dtstack.flinkx.sink.format.BaseRichOutputFormat.writeSingleRecord(BaseRichOutputFormat.java:427) ~[flinkx-core-1.12-SNAPSHOT.jar:?]
	at com.dtstack.flinkx.sink.format.BaseRichOutputFormat.writeRecord(BaseRichOutputFormat.java:244) ~[flinkx-core-1.12-SNAPSHOT.jar:?]
	at com.dtstack.flinkx.sink.format.BaseRichOutputFormat.writeRecord(BaseRichOutputFormat.java:84) ~[flinkx-core-1.12-SNAPSHOT.jar:?]
	at com.dtstack.flinkx.sink.DtOutputFormatSinkFunction.invoke(DtOutputFormatSinkFunction.java:117) ~[flinkx-core-1.12-SNAPSHOT.jar:?]
	at org.apache.flink.table.runtime.operators.sink.SinkOperator.processElement(SinkOperator.java:72) ~[flink-table-blink_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:71) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:46) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:26) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:50) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:28) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at StreamExecCalc$56.processElement(Unknown Source) ~[?:?]
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:71) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:46) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:26) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:50) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:28) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.api.operators.StreamSourceContexts$ManualWatermarkContext.processAndCollect(StreamSourceContexts.java:317) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.api.operators.StreamSourceContexts$WatermarkContext.collect(StreamSourceContexts.java:411) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at com.dtstack.flinkx.source.DtInputFormatSourceFunction.run(DtInputFormatSourceFunction.java:135) ~[flinkx-core-1.12-SNAPSHOT.jar:?]
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
	at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:267) ~[flink-dist_2.12-1.12.7.jar:1.12.7]
Caused by: java.io.IOException: Connection reset by peer
	at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[?:1.8.0_60]
	at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[?:1.8.0_60]
	at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[?:1.8.0_60]
	at sun.nio.ch.IOUtil.read(IOUtil.java:197) ~[?:1.8.0_60]
	at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) ~[?:1.8.0_60]
	at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:57) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) ~[hadoop-common-3.2.1U20.jar:?]
	at java.io.FilterInputStream.read(FilterInputStream.java:133) ~[?:1.8.0_60]
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[?:1.8.0_60]
	at java.io.BufferedInputStream.read(BufferedInputStream.java:265) ~[?:1.8.0_60]
	at java.io.FilterInputStream.read(FilterInputStream.java:83) ~[?:1.8.0_60]
	at java.io.FilterInputStream.read(FilterInputStream.java:83) ~[?:1.8.0_60]
	at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:567) ~[hadoop-common-3.2.1U20.jar:?]
	at java.io.DataInputStream.readInt(DataInputStream.java:387) ~[?:1.8.0_60]
	at org.apache.hadoop.ipc.Client$IpcStreams.readResponse(Client.java:1850) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1183) ~[hadoop-common-3.2.1U20.jar:?]
	at org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) ~[hadoop-common-3.2.1U20.jar:?]

 

报错含义

解释1:
在这里插入图片描述

参考
https://knowledge.broadcom.com/external/article/222862/what-does-connection-reset-by-peer-mean.html

 

解释2:

https://stackoverflow.com/questions/1434451/what-does-connection-reset-by-peer-mean

What is the meaning of the “connection reset by peer” error on a TCP connection? Is it a fatal error or just a notification or related to the network failure?

在这里插入图片描述
 
 

flink mail

https://www.mail-archive.com/issues@flink.apache.org/msg485176.html

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

roman_日积跬步-终至千里

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值