Win7 运行Hadoop WordCount 报错

在Win7下配置好hadoop集成eclipse开发环境后，运行WordCount程序的时候，可能会出现的问题：

1，java.io.IOException: Cannot run program "chmod": CreateProcess error=2

java.io.IOException: Cannot run program "chmod": CreateProcess error=2, ????????? 
at java.lang.ProcessBuilder.start(Unknown Source) 
at org.apache.hadoop.util.Shell.runCommand(Shell.java:149) 
at org.apache.hadoop.util.Shell.run(Shell.java:134) 
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286) 
at org.apache.hadoop.util.Shell.execCommand(Shell.java:354) 
at org.apache.hadoop.util.Shell.execCommand(Shell.java:337) 
at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:481) 
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:473) 
at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:280) 
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:372) 
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:484) 
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:465) 
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:372) 
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:208) 
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:142) 

解决方式：

在windows7的 环境变量 path中配置增加;C:\hadoop4win\bin;C:\hadoop4win\usr\sbin

注意path中的前后值要用分号（;）隔开。

2，可能会报内存溢出异常

java.lang.OutOfMemoryError: Java heap space
 at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:781)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
 at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
11/08/16 10:31:39 INFO mapred.JobClient:  map 0% reduce 0%
11/08/16 10:31:39 INFO mapred.JobClient: Job complete: job_local_0001
11/08/16 10:31:39 INFO mapred.JobClient: Counters: 0
Exception in thread "main" java.io.IOException: Job failed!
 at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
 at mapred.WordCount.main(WordCount.java:110)

原因是客户端程序运行时java虚拟机内存分配不够，修改run configuration（run as >run configurations），选择Arguments选项卡，在VM arguments栏中加入-Xmx1024m。如图：

或者选择中项目右键-->Properties-->在左边点击java Build Path-->选择 Libararies 选项卡-->找到JRE，点击Edit-->Installed JRE-->选择对应的JDK，Edit,如图设置

3，重启 eclipse

修改上面1，2 任意一项都要重启eclipse，否则看不到效果的！

 

4. Task Id : attempt_201003251936_0001_r_000000_0, Status : FAILED NullPointerException ConcurrentHashMap.get(ConcurrentHashMap.java:768)

Task Id : attempt_201003251936_0001_r_000000_0, Status : FAILED 
NullPointerException 
ConcurrentHashMap.get(ConcurrentHashMap.java:768)

10/03/25 19:40:21 INFO mapred.JobClient: Task Id : attempt_201003251936_0001_r_000000_0, Status : FAILED
Error: java.lang.NullPointerException
        at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
        at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
        at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)

10/03/25 19:40:21 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&taskid=attempt_201003251936_0001_r_000000_0&filter=stdout
10/03/25 19:40:21 WARN mapred.JobClient: Error reading task outputhttp://localhost:50060/tasklog?plaintext=true&taskid=attempt_201003251936_0001_r_000000_0&filter=stderr
10/03/25 19:40:27 INFO mapred.JobClient: Task Id : attempt_201003251936_0001_r_000000_1, Status : FAILED
Error: java.lang.NullPointerException
        at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
        at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.getMapCompletionEvents(ReduceTask.java:2683)
        at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$GetMapEventsThread.run(ReduceTask.java:2605)

10/03/25 19:40:27 WARN mapred.JobClient: Error reading task outputnode2.1036dhcp

原因是 windows 的计算机名称是不能包含下划线  不能以数字开头，否则会报上面的错误。

如我的计算机名称是Think_L440 ，含有下划线，故出现上面的错误。

修改计算机名称为Think440，并对应修改C:\hadoop4win\etc\networks、hosts等相关的配置文件。问题解决了！！！

修改后一定要重启电脑哦！！！