How to solve java.net.SocketTimeoutException:60000millis problem in HDFS

最新推荐文章于 2022-07-05 14:56:15 发布

月满西楼

最新推荐文章于 2022-07-05 14:56:15 发布

阅读量2.6k

点赞数

本文链接：https://blog.csdn.net/jzhang_cs/article/details/6982017

版权

在使用HDFS时，用户经常遇到Data Node与DFSClient通信失败的问题，导致'java.net.SocketTimeoutException: 60000millis timeout'错误。这通常发生在小型集群文件过多，繁忙的Data Node无法在限定时间内（如60秒）调度块。为了解决这个问题，可以缩短'ipc.ping.interval'配置项的默认值60000毫秒，将其添加到'core-site.xml'文件中，以便DFSClient更快地切换到其他Data Node。此解决方案适用于希望提高实时服务响应速度的情况。

摘要由CSDN通过智能技术生成

Many HDFS users encounter the following error when DFSClient ready file from a certain Data Node.

"java.net.SocketTimeoutException: 60000millis timeout while waiting for channel to be ready for read. ch"

The reason about this error is that the Data Node fails to communicate with DFSClient when DFSClient requests some block from that Data Node, which often occurs when the small cluster conntains too many files and the Data Node with such heavy burden cannot schedules a certain block within the limited time (for instance 60 sec). Then DFSClient waits untill this interval fires, than turn to request the other Data Node containing the same block.

If HDFS is used for in-time service, people wil find the 60 sec timeo