错误信息:
Exception in thread "main" java.io.IOException:Failed to set permissions of path:\tmp\hadoop-yth\mapred\staging\yth-2036315919\.staging to 0700
官方BUG参考:
https://issues.apache.org/jira/browse/HADOOP-7682
解决方法:
一、修改core包(在文件夹内已经改好)
1、下载并解压http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-1.1.2/hadoop-1.1.2.tar.gz
2、修改hadoop-1.1.2\src\core\org\apache\hadoop\fs\FileUtil.java,搜索 Failed to set permissions of path,找到689行,把throw new IOException改为LOG.warn
3、修改hadoop-1.1.2\build.xml,搜索autoreconf,移除匹配的6个executable="autoreconf"的exec配置
4、下载解压ant,将ant目录下的bin目录加入环境变量path
5、在命令行切换到hadoop-1.1.2目录,执行ant
6、用新生成的hadoop-1.1.2\build\hadoop-core-1.1.3-SNAPSHOT.jar替换nutch的hadoop-core-1.0.3.jar
二、config加上用户信息
conf.set("df.default.name", "hdfs://hadoop1master:9000/");//设置hdfs的默认路径
conf.set("hadoop.job.ugi", "hadoop,hadoop");//用户名,组
conf.set("mapred.job.tracker", "hadoop1master:9001");//设置jobtracker在哪