为了能在调试MapReduce任务阶段有更好的工作效率,我们可以把URI的Schema设置为file:///,这样MapReduce任务就可以范围windows本地文件夹。当我在尝试这么做的时候出现了如下的空指针异常
Exception in thread "main" java.lang.NullPointerException
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1010)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:445)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:739)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:722)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:633)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:421)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:281)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:125)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:348)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
原因是hadoop.dll文件和winutils.exe文件缺失了。解决步骤为:
1,下载这两个文件,下载地址:http://download.csdn.net/download/fly_leopard/9503059
2,解压之后,把hadoop.dll拷贝到C:\Windows\System32下面
3,创建环境变量HADOOP_HOME,然后把winutils.exe文件拷贝到${HADOOP_HOME}/bin目录下