下午2点11:40.502分 INFO AbstractService
Service org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService failed in state INITED; cause: org.fusesource.leveldbjni.internal.NativeDB$DBException:IO error:/var/lib/hadoop-yarn/yarn-nm-recovery/yarn-nm-state/LOCK: 权限不够
org.fusesource.leveldbjni.internal.NativeDB$DBException:IO error:/var/lib/hadoop-yarn/yarn-nm-recovery/yarn-nm-state/LOCK: 权限不够
at org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:200)
at org.fusesource.leveldbjni.internal.NativeDB.open(NativeDB.java:218)
at org.fusesource.leveldbjni.JniDBFactory.open(JniDBFactory.java:168)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService.openDatabase(NMLeveldbStateStoreService.java:944)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService.initStorage(NMLeveldbStateStoreService.java:931)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMStateStoreService.serviceInit(NMStateStoreService.java:204)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartRecoveryStore(NodeManager.java:181)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:245)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:562)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:609)
解决方案:
login the related node server, check the permission and ownership of those directories /var/lib/hadoop-*, I got the following info
[root@elephant lib]# ls -l | grep -i hadoop
d---------.2 root root 6 Nov 2505:27 hadoop-hdfs
d---------.2 root root 6 Nov 2505:27 hadoop-httpfs
d---------.2 root root 6 Nov 2505:27 hadoop-kms
d---------.2 root root 6 Nov 2505:27 hadoop-mapreduce
d---------.3 root root 29 Nov 2506:44 hadoop-yarn
they are not right. so we need to do the following steps to fix it