问题:hadoop集群的namenode一直处于安全模式。
现象:开启hive时报错:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.SafeModeException):
Cannot create directory /tmp/hive/xxx/01ce6751-cfbf-4120-9d63-0a2432187af2. Name node is in safe mode.
解决:
1、手动关闭安全模式,但是下一次还是会处于安全模式。
(1)bin/hdfs dfsadmin -safemode get (功能描述:查看安全模式状态)
(2)bin/hdfs dfsadmin -safemode enter (功能描述:进入安全模式状态)
(3)bin/hdfs dfsadmin -safemode leave (功能描述:离开安全模式状态)
(4)bin/hdfs dfsadmin -safemode wait (功能描述:等待安全模式状态)
2、具体解决如下:
(1)查看当前的hdfs的blocks的状态
hdfs fsck -blocks
发现有损坏的块:
(2)手动关掉安全模式
hadoop dfsadmin -safemode leave
刷新namenode界面,可看到:
(3)hadoop fs -rm -r -f 删掉损坏的blocks
hadoop fs -rm -r -f /tmp/logs/xxx/logs-tfile/application_1639968661551_0015/hadoop102_40775
(4)再次查看blocks的状态
hdfs fsck -blocks
出现下面的界面:Status: HEALTHY
Status: HEALTHY
Number of data-nodes: 1
Number of racks: 1
Total dirs: 160
Total symlinks: 0
Replicated Blocks:
Total size: 401578760 B
Total files: 621
Total blocks (validated): 538 (avg. block size 746428 B)
Minimally replicated blocks: 538 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 538 (100.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 1.0
Missing blocks: 0
Corrupt blocks: 0
Missing replicas: 1076 (66.666664 %)
Erasure Coded Block Groups:
Total size: 0 B
Total files: 0
Total block groups (validated): 0
Minimally erasure-coded block groups: 0
Over-erasure-coded block groups: 0
Under-erasure-coded block groups: 0
Unsatisfactory placement block groups: 0
Average block group size: 0.0
Missing block groups: 0
Corrupt block groups: 0
Missing internal blocks: 0
FSCK ended at Wed Dec 08 16:51:54 CST 2021 in 20 milliseconds
The filesystem under path '/' is HEALTHY
查看hadoopweb端口:
安全模式关闭
块也正常了