1. 安装JDK
a. 解压jdk
[bigdata001@Yuxi software]$ tar -xvzf jdk-7u60-linux-x64.tar.gz
b. 设置环境变量
PATH=$PATH:$HOME/bin
export JAVA_HOME=/home/bigdata001/BigDataPlatform/jdk1.7.0_60
export JAVA_BIN=$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export JRE_HOME=$JAVA_HOME/jre
export PATH=$PATH:$JAVA_HOME/bin
export PATH
c. 使之生效
[bigdata001@Yuxi ~]$ source .bash_profile
d. 验证
[bigdata001@Yuxi ~]$ java -version
java version "1.7.0_60"
Java(TM) SE Runtime Environment (build 1.7.0_60-b19)
Java HotSpot(TM) 64-Bit Server VM (build 24.60-b09, mixed mode)
2. 安装Maven
a. 解压
[bigdata001@Yuxi software]$ tar -xvzf apache-maven-3.2.3-bin.tar.gz
b. 配置环境变量
修改系统环境变量有两种方式,修改/etc/profile,或者在/etc/profile.d/下添加定制的shell文件,鉴于profile文件的重要性,尽量不要在profile文件里添加内容,官方建议采用第二种,以保证profile文件的绝对安全。
下面采用第二种方式:
[root@Yuxi etc]# cd /etc/profile.d/
[root@Yuxi profile.d]# touch maven.sh
向maven.sh中添加如下内容
export MAVEN_HOME='/home/bigdata001/BigDataPlatform/apache-maven-3.2.3'
export PATH=$MAVEN_HOME/bin:$PATH
c. 使之生效
[root@Yuxi profile.d]# source /etc/profile
d. 验证安装
Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4; 2014-08-12T04:58:10+08:00)
Maven home: /home/bigdata001/BigDataPlatform/apache-maven-3.2.3
Java version: 1.7.0_60, vendor: Oracle Corporation
Java home: /home/bigdata001/BigDataPlatform/jdk1.7.0_60/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-431.el6.x86_64", arch: "amd64", family: "unix"
3. 安装Protobuf
a. 安装依赖软件
[root@Yuxi ~]# yum install gcc*
b. 解压缩(注意压缩文件后缀, maven安装包是—gzip文件,解压时需加–z)
[bigdata001@Yuxi software]$ tar -xvf protobuf-2.5.0.tar.bz2
c. 安装
[bigdata001@Yuxi BigDataPlatform]$ cd protobuf-2.5.0/
[bigdata001@Yuxi protobuf-2.5.0]$ ./configure
[bigdata001@Yuxi protobuf-2.5.0]$ make
[bigdata001@Yuxi protobuf-2.5.0]$ make check
[root@Yuxi protobuf-2.5.0]# make install
d. 验证安装
[root@Yuxi protobuf-2.5.0]# protoc --version
libprotoc 2.5.0
4. 编译Hadoop
a. 解压缩
[bigdata001@Yuxi software]$ tar -xvzf hadoop-2.4.1-src.tar.gz
b. 修改Maven配置文件
切换为root权限并修改文件
[root@Yuxi BigDataPlatform]# gedit apache-maven-3.2.3/conf/settings.xml
增加profile内容
<profile>
<id>jdk-1.7</id>
<activation>
<jdk>1.7</jdk>
</activation>
<repositories>
<repository>
<id>nexus</id>
<name>local private nexus</name>
<url>http://maven.oschina.net/content/groups/public/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>nexus</id>
<name>local private nexus</name>
<url>http://maven.oschina.net/content/groups/public/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
c. 安装依赖包
[root@Yuxi hadoop-2.4.1-src]# yum install ncurses-devel
[root@Yuxi hadoop-2.4.1-src]# yum install openssl-devel
[root@Yuxi hadoop-2.4.1-src]# yum install cmake
d. 由于编译依赖的组件已经安装完成,所以直接执行编译命令即可
[root@Yuxi hadoop-2.4.1-src]# mvn package -Pdist,native -DskipTests -Dtar
经过的漫长等待,hadoop编译结束,效果如图
e. 验证编译
编译后的文件存在于/home/bigdata001/BigDataPlatform/hadoop-2.4.1-src/hadoop-dist/target
f. 验证版本号
[bigdata001@Yuxi hadoop-2.4.1]$ bin/hadoop version
Hadoop 2.4.1
Subversion Unknown -r Unknown
Compiled by root on 2014-08-18T10:51Z
Compiled with protoc 2.5.0
From source with checksum bb7ac0a3c73dc131f4844b873c74b630
This command was run using /home/bigdata001/BigDataPlatform/hadoop-2.4.1-src/hadoop-dist/target/hadoop-2.4.1/share/hadoop/common/hadoop-common-2.4.1.jar
g. 验证hadoop lib 的位数
[bigdata001@Yuxi hadoop-2.4.1]$ file lib//native/*
lib//native/libhadoop.a: current ar archive
lib//native/libhadooppipes.a: current ar archive
lib//native/libhadoop.so: symbolic link to `libhadoop.so.1.0.0'
lib//native/libhadoop.so.1.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
lib//native/libhadooputils.a: current ar archive
lib//native/libhdfs.a: current ar archive
lib//native/libhdfs.so: symbolic link to `libhdfs.so.0.0.0'
lib//native/libhdfs.so.0.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped