博客详细记录了在Hadoop集群操作时遇到的DataNode文件读写异常，错误信息为'Bad connect ack with firstBadLink...'. 原因分析指出，当Datanode上的Block超过默认xcievers线程数上限（4096）时，会导致某些Block无法进行读写。解决方案是在hdfs-site.xml配置文件中增加相应设置。
今天在集群上看到有两个任务跑失败了：
Err log:
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapreduce.job.reduces=<number>
java.io.IOException: Bad connect ack with firstBadLink as 192.168.44.57:50010
最低0.47元/天 解锁文章
扫一扫
771
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
