The reported blocks 801 needs additional 1 blocks to reach the threshold 0.9990 of total 但数当数据量比较大时,要进行分动态分区就会出现各种各样的问题,
问题1:
The reported blocks 801 needs additional 1 blocks to reach the threshold 0.9990 of total blocks 803.
故障处理,比较粗暴的方式,删除故障分区:
步骤 1 执行命令退出安全模式:hadoop dfsadmin -safemode leave
步骤 2 执行健康检查,删除损坏掉的block。()
sudo -u hdfs hdfs fsck / -delete
问题2:
Call From master/192.168.10.113 to master:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
原因:没有启动 jobhistory
执行命令
mr-jobhistory-daemon.sh start historyserver
来启动jobhistory
问题3:
RemoteException(java.io.IOException): File /user/hive/warehouse/ods.db/ods_fin_opb_feedetail_plus/.hive-staging_hive_2021-06-07_10-14-59_792_6525550450636989213-1/_task_tmp.-ext-10002/day=2021-02-04/_tmp.000006_1 could only be written to 0 of the 1 minReplication nodes. There are 3 datanode(s) running and 3 node(s) are excluded in this operation