几点说明
最好能有 MacPort 或者其他的 Brew 类似的工具,这样能节省很多的时间。
几个必须的工具
* Unix System
这个是必须的
* JDK 1.6+
JDK版本不能太高,反正 Hadoop 2.6 在 JDK 1.8 上编译失败好几次,建议用 1.7
* Maven 3.0 or later
这个也是必须的
* Findbugs 1.3.9 (if running findbugs)
Findbugs 可以不用
* ProtocolBuffer 2.5.0
这个必须要
* CMake 2.6 or newer (if compiling native code), must be 3.0 or newer on Mac
去 CMake 的官网下就可以
* Zlib devel (if compiling native code)
* openssl devel ( if compiling native hadoop-pipes )
这些我电脑里面以前安过了,这次就没装
* Internet connection for first build (to fetch all Maven and Hadoop dependencies)
一定要记得联网,因为在安装的过程中要下载茫茫多的东西。
编译命令
mvn package -Pdist,native,docs -DskipTests -Dtar
常见的几个错误
错误提示
Exception in thread "main" java.lang.AssertionError: Missing tools.jar at: /Library/Java/JavaVirtualMachines/jdk1.7.0_71.jdk/Contents/Home/Classes/classes.jar. Expression: file.exists()
解决办法
sudo mkdir $JAVA_HOME/Classes
sudo ln -sf $JAVA_HOME/lib/tools.jar $JAVA_HOME/Classes/classes.jar
错误提示
Time out
解决办法
这个问题是在安装第5个子项目的时候,提示了一大堆,但是中心思想就是提示超时,可能是我网络太慢的原因吧,重新开始了一次就过了那个点。
最后出现这些提示则说明编译成功
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................. SUCCESS [ 1.642 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [ 1.438 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [ 2.686 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [ 0.300 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [ 2.000 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [ 3.672 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ 2.789 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [ 5.065 s]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [ 3.743 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [01:56 min]
[INFO] Apache Hadoop NFS .................................. SUCCESS [ 6.492 s]
[INFO] Apache Hadoop KMS .................................. SUCCESS [ 14.486 s]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [ 0.144 s]
[INFO] Apache Hadoop HDFS ................................. SUCCESS [03:32 min]
[INFO] Apache Hadoop HttpFS ............................... SUCCESS [03:11 min]
[INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [02:33 min]
[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [ 4.413 s]
[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [ 0.034 s]
[INFO] hadoop-yarn ........................................ SUCCESS [ 0.050 s]
[INFO] hadoop-yarn-api .................................... SUCCESS [01:28 min]
[INFO] hadoop-yarn-common ................................. SUCCESS [01:10 min]
[INFO] hadoop-yarn-server ................................. SUCCESS [ 0.133 s]
[INFO] hadoop-yarn-server-common .......................... SUCCESS [ 25.965 s]
[INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [01:16 min]
[INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [ 3.452 s]
[INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [ 8.736 s]
[INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [ 23.658 s]
[INFO] hadoop-yarn-server-tests ........................... SUCCESS [ 6.801 s]
[INFO] hadoop-yarn-client ................................. SUCCESS [ 8.720 s]
[INFO] hadoop-yarn-applications ........................... SUCCESS [ 0.030 s]
[INFO] hadoop-yarn-applications-distributedshell .......... SUCCESS [ 2.547 s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SUCCESS [ 2.043 s]
[INFO] hadoop-yarn-site ................................... SUCCESS [ 0.065 s]
[INFO] hadoop-yarn-registry ............................... SUCCESS [ 6.182 s]
[INFO] hadoop-yarn-project ................................ SUCCESS [ 5.765 s]
[INFO] hadoop-mapreduce-client ............................ SUCCESS [ 0.076 s]
[INFO] hadoop-mapreduce-client-core ....................... SUCCESS [ 25.826 s]
[INFO] hadoop-mapreduce-client-common ..................... SUCCESS [ 16.665 s]
[INFO] hadoop-mapreduce-client-shuffle .................... SUCCESS [ 3.549 s]
[INFO] hadoop-mapreduce-client-app ........................ SUCCESS [ 11.537 s]
[INFO] hadoop-mapreduce-client-hs ......................... SUCCESS [ 8.411 s]
[INFO] hadoop-mapreduce-client-jobclient .................. SUCCESS [ 22.017 s]
[INFO] hadoop-mapreduce-client-hs-plugins ................. SUCCESS [ 1.753 s]
[INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [ 7.949 s]
[INFO] hadoop-mapreduce ................................... SUCCESS [ 4.888 s]
[INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [ 14.673 s]
[INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [ 13.820 s]
[INFO] Apache Hadoop Archives ............................. SUCCESS [ 2.740 s]
[INFO] Apache Hadoop Rumen ................................ SUCCESS [ 8.339 s]
[INFO] Apache Hadoop Gridmix .............................. SUCCESS [ 5.882 s]
[INFO] Apache Hadoop Data Join ............................ SUCCESS [ 3.981 s]
[INFO] Apache Hadoop Ant Tasks ............................ SUCCESS [ 2.934 s]
[INFO] Apache Hadoop Extras ............................... SUCCESS [ 3.290 s]
[INFO] Apache Hadoop Pipes ................................ SUCCESS [ 36.593 s]
[INFO] Apache Hadoop OpenStack support .................... SUCCESS [ 6.642 s]
[INFO] Apache Hadoop Amazon Web Services support .......... SUCCESS [02:09 min]
[INFO] Apache Hadoop Client ............................... SUCCESS [ 9.129 s]
[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [ 0.281 s]
[INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [ 4.498 s]
[INFO] Apache Hadoop Tools Dist ........................... SUCCESS [ 17.674 s]
[INFO] Apache Hadoop Tools ................................ SUCCESS [ 0.072 s]
[INFO] Apache Hadoop Distribution ......................... SUCCESS [ 57.905 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 24:35 min
[INFO] Finished at: 2015-02-17T22:13:02+08:00
[INFO] Final Memory: 195M/1051M
[INFO] ------------------------------------------------------------------------
根据说明可以知道,编译后的文件在
{hadoop source dir.}/hadoop-dist/target/hadoop-2.6.0.tar.gz
最后在
{hadoop source dir.}/hadoop-dist/target/hadoop-2.6.0
使用命令
bin/hadoop version
查看版本信息,嗯,没错!就是这个!
Hadoop 2.6.0
Subversion Unknown -r Unknown
Compiled by wangzhiyu on 2015-02-17T13:48Z
Compiled with protoc 2.5.0
From source with checksum 18e43357c8f927c0695f1e9522859d6a
This command was run using /Users/wangzhiyu/Downloads/hadoop-2.6.0-src/hadoop-dist/target/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar