Hadoop入门01--编译hadoop2.3.0源代码

Hadoop入门01--编译hadoop2.3.0源代码

软件资料网盘地址http://yunpan.cn/cdIWiQrRL9w26  访问密码 ca41

1、下载Hadoop源代码Release2.3.0

在/app目录下新建compile子目录，在该文件夹通过SVN获取Hadoop2.2.0源代码，命令如下：

cd /app
mkdir compile
cd compile
svn checkout   http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.3.0/

从服务器上下载大约需要30分钟左右，

从服务器下载比较慢,如果想快点下载好,本人网盘里有release-2.3.0.tar.gz已经对下载的进行压缩，没有任何的修改，，欢迎大家下载

然后解压到/app/compile文件下

2、编译Hadoop源代码

<span style="font-size:12px;">[root@jxpaiwp01 release-2.3.0]# cd /app/compile/release-2.3.0/</span>

<span style="font-size:12px;">[root@jxpaiwp01 release-2.3.0]# mvn package -Pdist,native -DskipTests -Dtar</span>

编译过程还是需要从网上下载东西，所以还是大约需要30分钟

3、验证是否编译成功

到hadoop-dist/target/hadoop-2.3.0/lib/native 目录查看libhadoop.so.1.0.0属性

cd /app/compile/release-2.3.0/hadoop-dist/target/hadoop-2.3.0/lib/native

file * 

该文件为ELF 64-bit LSB则表示文件成功编译为64位（如果系统没有file命令，则使用sudo yum install file进行安装）

[root@jxp native]# cd  <span style="font-size:12px;">/app/compile/</span><span style="font-size:12px;">release-2.3.0/</span><span style="font-size:12px;">hadoop-dist/target/hadoop-2.3.0/lib/native</span>

<span style="font-size:12px;">[root@jxp native]# file *
libhadoop.a:        current ar archive
libhadooppipes.a:   current ar archive
libhadoop.so:       symbolic link to `libhadoop.so.1.0.0'
libhadoop.so.1.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
libhadooputils.a:   current ar archive
libhdfs.a:          current ar archive
libhdfs.so:         symbolic link to `libhdfs.so.0.0.0'
libhdfs.so.0.0.0:   ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped</span>

4错误信息

编译错误，

[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.apache.hadoop:hadoop-main:2.3.0 (/app/compile/release-2.3.0/pom.xml) has 1 error
[ERROR]     Unresolveable build extension: Plugin org.apache.felix:maven-bundle-plugin:2.4.0 or one of its dependencies could not be resolved: The following artifacts could not be resolved: biz.aQute.bnd:bndlib:jar:2.1.0, org.osgi:org.osgi.core:jar:4.2.0, org.apache.maven:maven-model:jar:2.0.7, org.codehaus.plexus:plexus-container-default:jar:1.0-alpha-9-stable-1, junit:junit:jar:3.8.1, org.apache.maven:maven-plugin-descriptor:jar:2.0.7, org.codehaus.plexus:plexus-archiver:jar:2.1, org.codehaus.plexus:plexus-interpolation:jar:1.15, org.codehaus.plexus:plexus-utils:jar:3.0.10: Could not transfer artifact biz.aQute.bnd:bndlib:jar:2.1.0 from/to central (https://repo.maven.apache.org/maven2): GET request of: biz/aQute/bnd/bndlib/2.1.0/bndlib-2.1.0.jar from central failed: Connection reset -> [Help 2]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/PluginManagerException

解决办法，直接修改hadoop-common-project/hadoop-auth/pom.xml，其实就是少了一个包，添加一个dependency：

[root@jxpaiwp01 release-2.3.0]# vim hadoop-common-project/hadoop-auth/pom.xml 

    <dependency>
          <groupId>org.mortbay.jetty</groupId>
             <artifactId>jetty-util</artifactId>
             <scope>test</scope>
    </dependency>