将JAVA源代码生成jar包
1.maven的pom文件中加入以下依赖
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin </artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>MapReduce.test.MyDriver</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
其中要修改的是MapReduce.test.MyDriver,修改为主类所在路径
2.打开终端,分别输入mvn clean和mcn install然后按ctrl+Enter来执行命令,生成jar包,结果在target文件夹下
3.将jar包上传到hadoop主机上
4.启动集群
在Namenode主机上执行start-dfs.sh
在resourcemanager主机上start-yarn.sh
5.以wordcount案例为例子,执行:
hadoop jar testData/MapReduce.01-1.0-SNAPSHOT-jar-with-dependencies.jar MapReduce.test.MyDriver /user/hadoop/input/wordcount.txt /user/hadoop/output
6.没有错误的话,查看结果:
hdfs dfs -cat /user/hadoop/output/part-r-00000