flink on yarn ——报错ResourceLocalizationService: Could not carry out resource dir checks

消八哥

已于 2022-03-29 17:46:25 修改

阅读量2.3k

点赞数

分类专栏： flink flink on yarn 文章标签： yarn

于 2022-03-29 17:43:38 首次发布

本文链接：https://blog.csdn.net/weixin_40308031/article/details/123827529

版权

flink 同时被 2 个专栏收录

10 篇文章 1 订阅

订阅专栏

flink on yarn

8 篇文章 1 订阅

订阅专栏

报错：

java.io.FileNotFoundException: File /home/data/software/hadoop-3.2.2/data/tmp/nm-local-dir/filecache does not exist

2022-03-29 15:58:07,592 INFO org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection: Directory /home/data/software/hadoop-3.2.2/data/tmp/nm-local-dir passed disk check, adding to list of valid directories.
2022-03-29 15:58:07,598 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Could not carry out resource dir checks for /home/data/software/hadoop-3.2.2/data/tmp/nm-local-dir, which was marked as good
java.io.FileNotFoundException: File /home/data/software/hadoop-3.2.2/data/tmp/nm-local-dir/filecache does not exist
 at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:668)
 at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:989)
 at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:658)
 at org.apache.hadoop.fs.DelegateToFileSystem.getFileStatus(DelegateToFileSystem.java:126)
 at org.apache.hadoop.fs.FilterFs.getFileStatus(FilterFs.java:122)
 at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1245)
 at org.apache.hadoop.fs.FileContext$15.next(FileContext.java:1241)
 at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
 at org.apache.hadoop.fs.FileContext.getFileStatus(FileContext.java:1247)
 at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.checkLocalDir(ResourceLocalizationService.java:1649)
 at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.checkAndInitializeLocalDirs(ResourceLocalizationService.java:1623)
 at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$1.onDirsChanged(ResourceLocalizationService.java:286)
 at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.checkDirs(DirectoryCollection.java:501)
 at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.checkDirs(LocalDirsHandlerService.java:518)
 at org.apache.hadoop.yarn.server.nodemanager.L

web页面没有可使用的内存：

原因：

因为没有配置zookeeper，RM 不能把状态存入zk，需要配置zk：

core-site.xml：

   <property>
       <name>ha.zookeeper.quorum</name>
       <value>craw-node212:2181</value>
   </property>

yarn-site.xml：

    <property>
        <name>yarn.resourcemanager.zk-address</name>
        <value>craw-node212:2181</value>
    </property>
    <property>
        <name>yarn.resourcemanager.store.class</name>
     <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
    </property>

web页面恢复正常：