编译hadoop-2.2.0源码，RHEL6.3x64-CSDN博客

本文链接：https://blog.csdn.net/guo_yang/article/details/82186412

编译环境：

OS:?? RHEL6.3.x86_64

Kernel:2.6.32-358.el6.x86_64

JDK: 1.6.0_35? x86_64

编译前准备：

1.安装依赖的rpm包

?? yum -y install? lzo-devel? zlib-devel? gcc? autoconf? automake? libtool openssl-devel ncurses-devel cmake

2.安装maven

?? 下载地址:? http://mirror.bit.edu.cn/apache/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.tar.gz

??? 解压到/usr/local/maven目录下，然后编辑/etc/profile，在其中编辑MAVEN_HOME的环境变量，保存后执行：

??? source /etc/profile? 或? .? /etc/profile

??? 使之生效。

??? 验证生效与否：

? # mvn -v

Apache Maven 3.1.1 (0728685237757ffbf44136acec0402957f723d9a; 2013-09-17 23:22:22+0800)

??? Maven home: /usr/local/maven

??? Java version: 1.6.0_35, vendor: Sun Microsystems Inc.

??? Java home: /usr/java/jdk1.6.0_35/jre

??? Default locale: zh_CN, platform encoding: UTF-8

??? OS name: "linux", version: "2.6.32-358.el6.x86_64", arch: "amd64", family: “unix"

3.安装protobuf?? http://protobuf.googlecode.com/files/protobuf-2.5.0.tar.gz

??? tar xf protobuf-2.5.0.tar.gz

??? cd protobuf-2.5.0

??? ./configure

??? make

??? make install

开始编译Hadoop

1.下载hadoop-2.2.0 源码。

??? 地址:http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.2.0/hadoop-2.2.0-src.tar.gz

2.tar xf hadoop-2.2.0-src.tar.gz

?? cd? hadoop-2.2.0-src

3.在编译前，首先修复一个2.2.0版本的一个bug，否则会在编译过程中遇到如下报错：

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-auth: Compilation failure: Compilation failure:

??? [ERROR] /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[84,13] cannot access org.mortbay.component.AbstractLifeCycle

??? [ERROR] class file for org.mortbay.component.AbstractLifeCycle not found

bug patch地址：https://issues.apache.org/jira/browse/hadoop-10110

?? 方法：修改hadoop源码目录下的hadoop-common-project/hadoop-auth/pom.xml

?? 添加或修改如下部分内容：

????? <dependency>

????????? <groupId>org.mortbay.jetty</groupId>

+??????? <artifactId>jetty-util</artifactId>

+??????? <scope>test</scope>

+??? </dependency>

+??? <dependency>

+?????? <groupId>org.mortbay.jetty</groupId>

???????? <artifactId>jetty</artifactId>

???????? <scope>test</scope>

???? </dependency>

4.在hadoop源码目录下执行：

?? #/opt/apache-maven-3.0.5/bin/mvn package -Pdist,native -DskipTests -Dtar -e -X

[INFO] Apache Hadoop Main ................................ SUCCESS [4.419s]

[INFO] Apache Hadoop Project POM ......................... SUCCESS [3.358s]

[INFO] Apache Hadoop Annotations ......................... SUCCESS [4.488s]

[INFO] Apache Hadoop Assemblies ..........................? SUCCESS [0.640s]

[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [3.900s]

[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [6.072s]

[INFO] Apache Hadoop Auth ................................ SUCCESS [4.491s]

[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [4.014s]

[INFO] Apache Hadoop Common .............................. SUCCESS [1:48.272s]

[INFO] Apache Hadoop NFS ................................. SUCCESS [10.646s]

[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.066s]

[INFO] Apache Hadoop HDFS ................................ SUCCESS [1:54.818s]

[INFO] Apache Hadoop HttpFS .............................. SUCCESS [20.414s]

[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [14.205s]

[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [5.866s]

[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.079s]

[INFO] hadoop-yarn ....................................... SUCCESS [0.361s]

[INFO] hadoop-yarn-api ................................... SUCCESS [51.017s]

[INFO] hadoop-yarn-common ................................ SUCCESS [33.649s]

[INFO] hadoop-yarn-server ................................ SUCCESS [0.230s]

[INFO] hadoop-yarn-server-common ......................... SUCCESS [13.812s]

[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [19.117s]

[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [4.240s]

[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [19.048s]

[INFO] hadoop-yarn-server-tests .......................... SUCCESS [1.012s]

[INFO] hadoop-yarn-client ................................ SUCCESS [6.523s]

[INFO] hadoop-yarn-applications .......................... SUCCESS [0.174s]

[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [3.049s]

[INFO] hadoop-mapreduce-client ........................... SUCCESS [0.169s]

[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [29.535s]

[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [3.287s]

[INFO] hadoop-yarn-site .................................. SUCCESS [0.211s]

[INFO] hadoop-yarn-project ............................... SUCCESS [6.975s]

[INFO] hadoop-mapreduce-client-common .................... SUCCESS [24.512s]

[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [3.984s]

[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [13.070s]

[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [6.213s]

[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [5.803s]

[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [2.423s]

[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [8.608s]

[INFO] hadoop-mapreduce .................................. SUCCESS [5.522s]

[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [6.932s]

[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [10.974s]

[INFO] Apache Hadoop Archives ............................ SUCCESS [3.047s]

[INFO] Apache Hadoop Rumen ............................... SUCCESS [8.680s]

[INFO] Apache Hadoop Gridmix ............................. SUCCESS [6.245s]

[INFO] Apache Hadoop Data Join ........................... SUCCESS [3.823s]

[INFO] Apache Hadoop Extras .............................. SUCCESS [4.784s]

[INFO] Apache Hadoop Pipes ............................... SUCCESS [7.803s]

[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [2.759s]

[INFO] Apache Hadoop Tools ............................... SUCCESS [0.046s]

[INFO] Apache Hadoop Distribution ........................ SUCCESS [20.150s]

[INFO] Apache Hadoop Client .............................. SUCCESS [8.368s]

[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [0.278s]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 10:56.567s

[INFO] Finished at: Wed Dec 25 23:13:59 CST 2013

[INFO] Final Memory: 59M/121M

[INFO] ————————————————————————————————————

?? 直到最后出现BUILD SUCCESS，说明编译成功。

?? 源码目录下? hadoop-dist/target/hadoop-2.2.0.tar.gz 就是我们最终编译成功的hadoop包。