0-0 编译hadoop2.2.0

编译hadoop-2.2.0（ubuntux86_64）

###############################################

1.一些准备

sudo apt-get install cmake make -y

sudo apt-get install g++

sudo apt-get install gcc
sudo apt-get install build-essential
sudo apt-get install libssl-dev
sudo apt-get install zlib1g-dev
sudo apt-get install libglib2.0-dev

################################################

2.安装常用

(1)cp-r * /opt/,vi/etc/profile环境变量   （～/.bashrc）

/opt/jdk1.7.0_45 

/opt/apache-maven-3.0.5

/opt/findbugs-2.0.2

####编译要求是2.5.0

cd protobuf-2.5.0

 ./configure

 make

sudo make install

protoc –version

###增加国内maven 开源中国镜像

sudo vi /opt/apache-maven-3.0.5/conf/settings.xml

  <mirror>

<id>nexus-osc</id>

<mirrorOf>*</mirrorOf>

<name>Nexus osc</name>

<url>http://maven.oschina.net/content/groups/public/</url>

</mirror>

#####################################################

3.解压源码包

 

（1）  打补丁

访问：https://issues.apache.org/jira/browse/HADOOP-10110

###########

hadoop@master1:~/下载/hadoop-2.2.0-src$

wget https://issues.apache.org/jira/secure/attachment/12614482/HADOOP-10110.patch

#######

 ll -h hadoop-common-project/hadoop-auth/pom.xml

出现：

 

-rw-r--r-- 1 hadoop hadoop 6.9K 10月  7 13:52hadoop-common-project/hadoop-auth/pom.xml

#####

patch -p0 < ./HADOOP-10110.patch

出现：

patching file hadoop-common-project/hadoop-auth/pom.xml

Hunk #1 succeeded at 65 (offset 11 lines).

#####

ll -h hadoop-common-project/hadoop-auth/pom.xml

出现：（ 发现时间变了）

-rw-r--r-- 1 hadoop hadoop 7.3K  3月  3 22:47hadoop-common-project/hadoop-auth/pom.xml

 

4.编译开始

mvn clean package -DskipTests -Pdist,native,docs -Dtar

编译成功之后，在hadoop-dist/target/目录下生成有hadoop-2.2.0-tar.gz文件，这个就是我们要的东东。

###############################################################################

 

5.常见问题

【问题1】

最新的2.2.0 的SourceCode 压缩包解压出来的code有个bug 需要patch后才能编译。否则编译hadoop-auth 会提示下面错误：

[ERROR] Failed to execute goalorg.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile(default-testCompile) on project hadoop-auth: Compilation failure: Compilationfailure:

[ERROR]/home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[84,13]cannot access org.mortbay.component.AbstractLifeCycle

[ERROR] class file fororg.mortbay.component.AbstractLifeCycle not found

下载Patch :https://issues.apache.org/jira/browse/HADOOP-10110

 

【问题2】

[ERROR] Failed to execute goalorg.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on projecthadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version'did not return a version -> [Help 1]

[ERROR]

[ERROR] To see the full stacktrace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the-X switch to enable full debug logging.

 

 

 

安装protobuf

 export LD_LIBRARY_PATH=/usr/local/lib/

【问题2-1】

protoc --version

protoc: error while loadingshared libraries: libprotoc.so.8: cannot open shared object file: No such fileor directory

查看是否安装了

# ls /usr/local/lib

libhiredis.a       libltdl.a    libltdl.so.3.1.0   libprotobuf-lite.la        libprotobuf.so        libprotoc.la        liby.alibhiredis.so       libltdl.la    libprotobuf.a       libprotobuf-lite.so        libprotobuf.so.8      libprotoc.so        pkgconfiglibhiredis.so.0     libltdl.so    libprotobuf.la      libprotobuf-lite.so.8      libprotobuf.so.8.0.0  libprotoc.so.8libhiredis.so.0.10  libltdl.so.3 libprotobuf-lite.a libprotobuf-lite.so.8.0.0 libprotoc.a          libprotoc.so.8.0.0

加入到/etc/ld.so.conf，动态装入器将在其中查找共享库。

# vi /etc/ld.so.conf

include /usr/local/lib

但是在动态装入器能“看到”这一信息之前，必须将它转换到 ld.so.cache 文件中。可以通过运行 ldconfig 命令做到这一点：

# ldconfig

　　（当 ldconfig 操作结束时，您会有一个最新的 /etc/ld.so.cache 文件，它反映您对 /etc/ld.so.conf 所做的更改。从这一刻起，动态装入器在寻找共享库时会查看您在 /etc/ld.so.conf 中指定的所有新目录。 ）

# protoc --version

libprotoc 2.5.0

【问题3】

[ERROR] Failed to execute goalorg.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on projecthadoop-common: An Ant BuildException has occured: Execute failed:java.io.IOException: Cannot run program "cmake" (in directory"/home/l/software/hadoop-2.2.0-src/hadoop-common-project/hadoop-common/target/native"):error=2, No such file or directory -> [Help 1]

安装cmake

wget  http://www.cmake.org/files/v2.8/cmake-2.8.12.tar.gz

 tar zxvfcmake-2.8.12.tar.gz

cd cmake-2.8.12

./configure

make

sudo make install

【问题4】

configure: error:C++ preprocessor "/lib/cpp" fails sanity check
安装build-essential
[ERROR] Failed to execute goalorg.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on projecthadoop-hdfs: An Ant BuildException has occured: exec returned: 1 -> [Help 1]
安装libglib2.0-dev
[ERROR] Failed to execute goalorg.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on projecthadoop-pipes: An Ant BuildException has occured: exec returned: 1 -> [Help1]
安装libssl-dev