Windows下eclipse+maven连接hadoop集群开发MapReduce程序错误处理

最新推荐文章于 2019-05-21 12:43:08 发布

weixin_34342992

最新推荐文章于 2019-05-21 12:43:08 发布

阅读量149

点赞数

文章标签：大数据开发工具 java

原文链接：https://my.oschina.net/OkSerIous/blog/816582

版权

2019独角兽企业重金招聘Python工程师标准>>>

错误如下：

输入图片说明

1、下载hadoop-*.tar.gz文件，并解压 2、修改hadoop-1.2.0\src\core\org\apache\hadoop\fs\FileUtil.java，搜索 Failed to set permissions of path，找到689行，把throw new IOException改为LOG.warn

private static void checkReturnValue(boolean rv, File p, 
                                       FsPermission permission
                                       ) throws IOException {
    if (!rv) {
 
      throw new IOException("Failed to set permissions of path: " + p + 
                            " to " + 
                            String.format("%04o", permission.toShort()));
 
    }
  }

修改为：

private static void checkReturnValue(boolean rv, File p, 
                                       FsPermission permission
                                       ) throws IOException {
    if (!rv) {
      /**
      throw new IOException("Failed to set permissions of path: " + p + 
                            " to " + 
                            String.format("%04o", permission.toShort()));
	 */
	  LOG.warn("Failed to set permissions of path: " + p + 
                            " to " + 
                            String.format("%04o", permission.toShort()));
    }
  }

3、.修改hadoop-1.2.0\build.xml，搜索autoreconf，移除匹配的6个executable=”autoreconf”的exec配置例如：

<!--
    <exec executable="autoreconf" dir="${native.src.dir}" searchpath="yes" 
          failonerror="yes">
       <arg value="-if"/>
    </exec>
	-->

4、在命令行切换到hadoop-1.2.0目录，执行ant(要在cygwin下执行，在windows命令行下会报错)。 5、用新生成的hadoop-1.2.0\build\hadoop-core-1.2.1-SNAPSHOT.jar替换nutch的hadoop-core-1.2.0.jar.

转载于:https://my.oschina.net/OkSerIous/blog/816582