在eclipse将mapreduce程序运行在hadoop集群中

1.将集群配置文件：core-site.xml，hdfs-site.xml，mapred-site.xml，yarn-site,xml拷贝至项目的src目录下

2.将本项目打成JAR包：xxx.jar

3.在mapreduce程序main方法中添加代码：job.setJar("/xxx.jar")//jar包所在文件路径

4.其他套路性代码编写好即可运行。

5.运行时所遇错误

一不小心忘加hadoop-hdfs-2.7.3.jar，报错：

Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs
 at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2660)
 at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
 at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
 at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
 at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
 at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
 at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:172)
 at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:357)
 at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
 at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:500)
 at com.hadoop.hive.mr.ClickStreamVisit.main(ClickStreamVisit.java:127)

若使用的是job.setJarByClass(xxx.class)，运行后报错：

Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class com.hadoop.hive.mr.ClickStreamVisit$ClickStreamVisitMapper not found
 at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
 at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186)
 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:745)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: Class com.hadoop.hive.mr.ClickStreamVisit$ClickStreamVisitMapper not found
 at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
 at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
 ... 8 more