使用eclipse写mapreduce程序，然后打包程序到集群运行

一、调试eclipse运行环境

首先创建一个eclipse的Maven Project

 二、然后再pom.xml在里面写入一下配置

 <dependencies>

<dependency>

<groupId>junit</groupId>

<artifactId>junit</artifactId>

<version>RELEASE</version>

</dependency>

<dependency>

<groupId>org.apache.logging.log4j</groupId>

<artifactId>log4j-core</artifactId>

<version>2.8.2</version>

</dependency>

<dependency>

<groupId>org.apache.hadoop</groupId>

<artifactId>hadoop-common</artifactId>

<version>2.7.2</version>

</dependency>

<dependency>

<groupId>org.apache.hadoop</groupId>

<artifactId>hadoop-client</artifactId>

<version>2.7.2</version>

</dependency>

<dependency>

<groupId>org.apache.hadoop</groupId>

<artifactId>hadoop-hdfs</artifactId>

<version>2.7.2</version>

</dependency>

</dependencies>

 <build>

<plugins>

<plugin>

<artifactId>maven-compiler-plugin</artifactId>

<version>2.3.2</version>

<configuration>

<source>1.8</source>

<target>1.8</target>

</configuration>

</plugin>

<plugin>

<artifactId>maven-assembly-plugin </artifactId>

<configuration>

<descriptorRefs>

<descriptorRef>jar-with-dependencies</descriptorRef>

</descriptorRefs>

<archive>

<manifest>

<mainClass>JobMain</mainClass>

</manifest>

</archive>

</configuration>

<executions>

<execution>

<id>make-assembly</id>

<phase>package</phase>

<goals>

<goal>single</goal>

</goals>

</execution>

</executions>

</plugin>

</plugins>

</build>

三、然后找到程序主类接口

 

<mainClass>JobMain</mainClass>

更改程序主类:

 

 

四、更新主类的maven

 选这个选项:

 稍等更新之后的出现如下图:

 点击Maven install这个选项打包整个项目即可:

 在target项目出现如下包即可打包成jar包:

讲解一下:

 名字长的那个.jar:含有程序的依赖包。

 其实我们选那个名字短的就好了

然后将其上传到master主机

 然后执行Hadoop编译命令:

 hadoop jar mr-0.0.1-SNAPSHOT.jar WordcountDriver /wordcount/* /wordcount_out

执行的结果如下:

 然后查看你的输出目录: