CentOS6.4编译Hadoop2.2.0

1.安装jdk

不要小看这一步，起先装了个jdk1.8.0，出现如下错误。

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:2.8.1:jar (module-javadocs) on project hadoop-maven-plugins: MavenReportException: Error while creating archive:
[ERROR] Exit code: 1 - /usr/local/hadoop-2.2.0-src/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/util/Exec.java:47: error: unknown tag: String

想是jdk1.8.0修改了doc的某些内容

在/etc/profile中设置环境变量，添加

#set java environment
export JAVA_HOME=/usr/local/jdk1.7.0
export JRE_HOME=/usr/local/jdk1.7.0/jre
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar/:$JAVA_HOME/jre/lib:$JAVA_HOME/lib
export PATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/jre/bin

2.安装maven3.2.1

不需要特别安装，下载下来看只要把bin加入到环境变量PATH中，同样是修改/etc/profile文件

# set mvn environment
export PATH=$PATH:/usr/local/apache-maven-3.2.1/bin
export MAVEN_HOME=/usr/local/apache-maven-3.2.1

执行mav -version查看安装是否成功

3.安装ant1.9.3

不需要特别安装，下载下来看只要把bin加入到环境变量PATH中，同样是修改/etc/profile文件

# set ant environment
export PATH=$PATH:/usr/local/apache-ant-1.9.3/bin

执行ant -version查看安装是否成功

4.安装cmake2.8.12.2

5.安装findbugs2.0.2

这个要ant安装，只要在findbugs2.0.2的源代码路径下执行

ant

就可以了

6.安装forrest 0.9

要把bin加入到环境变量PATH中，同样是修改/etc/profile文件

# set forrest environment
export PATH=$PATH:/usr/local/apache-forrest-0.9/bin
export FORREST_HOME=$FORREST_HOEM:/usr/local/apache-forrest-0.9

7.安装zlib-devel（可能默认已经安装好了）

 yum install zlib-devel 

8.安装protobuf 2.5.0

./configure

make

make check

make install

9.修改2.2.0的一个bug

https://issues.apache.org/jira/browse/HADOOP-10110

实际上就是在hadoop-common-project/hadoop-auth/pom.xml中添加了几行代码

在<dependencies></dependencies>之间添加

   </dependency>

     <dependency>
       <groupId>org.mortbay.jetty</groupId>
+      <artifactId>jetty-util</artifactId>
+      <scope>test</scope>
+    </dependency>
+    <dependency>
+      <groupId>org.mortbay.jetty</groupId>
       <artifactId>jetty</artifactId>
       <scope>test</scope>
     </dependency>

10.还有的修改了maven的配置conf/settings.xml，我照此做了，但是感觉这个并不影响

一是在<mirror> </mirrors>之间添加

 <mirror>

        <id>nexus-osc</id>
         <mirrorOf>*</mirrorOf>
     <name>Nexusosc</name>
     <url>http://maven.oschina.net/content/groups/public/</url>
   </mirror>

二是在<profiles></profiles>之间添加

<profile>
       <id>jdk-1.7</id>
       <activation>
         <jdk>1.7</jdk>
       </activation>
       <repositories>
         <repository>
           <id>nexus</id>
           <name>local private nexus</name>
           <url>http://maven.oschina.net/content/groups/public/</url>
           <releases>
             <enabled>true</enabled>
           </releases>
           <snapshots>
             <enabled>false</enabled>
           </snapshots>
         </repository>
       </repositories>
       <pluginRepositories>
         <pluginRepository>
           <id>nexus</id>
          <name>local private nexus</name>
           <url>http://maven.oschina.net/content/groups/public/</url>
           <releases>
             <enabled>true</enabled>
           </releases>
           <snapshots>
             <enabled>false</enabled>
           </snapshots>
         </pluginRepository>
       </pluginRepositories>
     </profile>

以上这些过程我都照做了，但我不确定是否都是必须的，有些如果不安装，编译时会出现错误。

11.进入hadoop-2.2.0-src目录开始编译，输入命令

 mvn clean package -Pdist,native -DskipTests -Dtar 

我是使用root来编译的，之前在某个博客上说不能使用root编译，但是实践中发现会有permission denied的错误，我不确定错误发生的原因，但我是root的编译的。

以下是所碰到的错误：

 1. 2.2.0的源码有个bug，需要patch一下

从

https://issues.apache.org/jira/browse/HADOOP-10110

下载patch文件，放到hadoop-src的根目录下，在此目录下执行

patch -p0 > HADOOP-10110.patch

不patch会碰到如下错误

实际上这个patch就是在/hadoop-src/hadoop-common-project/hadoop-auth/pom.xml中添加了如下代码：

<dependency>
      <groupId>org.mortbay.jetty</groupId>
      <artifactId>jetty-util</artifactId>
      <scope>test</scope>
    </dependency>

但是注意这段代码要在标签<dependencies></dependencies>之间，否则会报错。

2.如果不安装protoc，报如下错误

    [ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1] 
    [ERROR]  
    [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. 
    [ERROR] Re-run Maven using the -X switch to enable full debug logging. 
    [ERROR]  
    [ERROR] For more information about the errors and possible solutions, please read the following articles: 
    [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException 
    [ERROR]  
    [ERROR] After correcting the problems, you can resume the build with the command 
    [ERROR]   mvn <goals> -rf :hadoop-common 

3.此错误

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (create-testdirs) on project hadoop-project: Error executing ant tasks: /usr/local/hadoop-2.2.0-src/hadoop-project/target/antrun/build-main.xml (Permission denied) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.

分析：

最后有个Permission denied，难道是要 root编译才行？

4. terminal会打印出编译的进度，成功是SUCCEED，失败就FAILURE，之后的全SKIPPED，可以由此定位出自己哪一步失败了。

5.此错误

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:2.8.1:jar (module-javadocs) on project hadoop-maven-plugins: MavenReportException: Error while creating archive:
[ERROR] Exit code: 1 - /usr/local/hadoop-2.2.0-src/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/util/Exec.java:47: error: unknown tag: String

分析：

看样子是没有识别出String，难道是Java的包没加入？ 

实际上是Java版本的问题，不能用1.8。