MapReduce三种方式实现WordCount的JAVA编程
1.MapReduce实现(服务器内部运行jar方式)
-
Hadoop
版本:hadoop 2.6.5 -
JDK
版本:jdk1.7.0_79 -
Zk
版本:zookeeper-3.4.6 -
启动环境:
三台zookeeper:zkServer.sh start
全面启动Hadoop:start-dfs.sh
启动YarnHA主节点:start-yarn.sh
启动YarnHA从节点:yarn-daemon.sh start resourcemanager -
Eclipse创建项目并新增jar包
5.1 创建项目
5.2 维护jar包:1、Build Path; 2、Add Library; 3、User Library;
5.3 新建User Libary后维护jar
Hadoop Jar目录清单:
avro-1.7.4.jar
commons-cli-1.2.jar
commons-codec-1.4.jar
commons-collections-3.2.1.jar
commons-configuration-1.6.jar
commons-httpclient-3.1.jar
commons-io-2.4.jar
commons-lang-2.6.jar
commons-logging-1.1.3.jar
dom4j-1.6.1.jar
guava-11.0.2.jar
hadoop-annotations-2.5.2.jar
hadoop-auth-2.5.2.jar
hadoop-common-2.5.2-tests.jar
hadoop-common-2.5.2.jar
hadoop-distcp-2.5.1.jar
hadoop-hdfs-2.5.2-tests.jar
hadoop-hdfs-2.5.2.jar
hadoop-hdfs-nfs-2.5.2.jar
hadoop-mapreduce-client-app-2.5.2.jar
hadoop-mapreduce-client-common-2.5.2.jar
hadoop-mapreduce-client-core-2.5.2.jar
hadoop-mapreduce-client-hs-2.5.2.jar
hadoop-mapreduce-client-hs-plugins-2.5.2.jar
hadoop-mapreduce-client-jobclient-2.5.2-tests.jar
hadoop-mapreduce-client-jobclient-2.5.2.jar
hadoop-mapreduce-client-shuffle-2.5.2.jar
hadoop-mapreduce-examples-2.5.2.jar
hadoop-nfs-2.5.2.jar
hadoop-yarn-api-2.5.2.jar
hadoop-yarn-applications-distributedshell-2.5.2.jar
hadoop-yarn-applications-unmanaged-am-launcher-2.5.2.jar
hadoop-yarn-client-2.5.2.jar
hadoop-yarn-common-2.5.2.jar
hadoop-yarn-server-applicationhisto