hadoop linux32位下载,51CTO博客-专业IT技术博客创作平台-技术成就梦想

先安装macports参考：http://blog.csdn.net/bamuta/article/details/13506893下载程序代码机器得连网，如果没联网找可以联网的机器下载，但是编译时还是要下载一些东西，所以，实在不行。最好找相同平台(可以是虚拟机)能上网的机器做下面工作，弄好了再拷回来。svn cohttp://svn.apache.org/repos/asf/hadoop/common/tags/release-2.2.0都下载到这了：[hadoop@hadoop01 hadoop]$ lsBUILDING.txt  hadoop-common-project  hadoop-maven-plugins  hadoop-toolsdev-support  hadoop-dist hadoop-minicluster  hadoop-yarn-projecthadoop-assemblies  hadoop-hdfs-project  hadoop-project  pom.xmlhadoop-client  hadoop-mapreduce-project  hadoop-project-dist安装开发环境1.必要的包sudo port install libtoolsudo port install cmakesudo port install ncurses sudo port install openssl sudo port install maven3  安装java7偶是小白，今天被人科普了JDK是可以进行编译的，而jre只能用来运行，反正在java上要注意区分2者的区别，以后详细学习java的时候再进一步了解吧。官网下载mac版本的安装文件官网地址： http://www.oracle.com/technetwork/java/javase/downloads/index.html选择jdk7的Mac OS X x64版本，jdk6没有Mac OS X版本的下载对应的dmg文件安装安装好在命令行输入：java -version 现实下面的内容java version "1.7.0_17" Java(TM) SE Runtime Environment (build 1.7.0_17-b02) Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)java就安装完成了，也无需配置环境变量删除的方法也很简单，在命令行中到 /Library/Java/JavaVirtualMachines 这个目录下，删除对应的jdk就可以了修改.bash_profile，添加export PATH=${PATH}:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Homeexport JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home  gcc也很重要gcc -vConfigured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/c++/4.2.1Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)Target: x86_64-apple-darwin13.0.0Thread model: posix mvn3 -vApache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 21:51:28+0800)Maven home: /opt/local/share/java/maven3Java version: 1.7.0_45, vendor: Oracle CorporationJava home: /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jreDefault locale: zh_CN, platform encoding: UTF-8OS name: "mac os x", version: "10.9", arch: "x86_64", family: "mac" qqqqmvn3 package -Pdist,native -DskipTests -Dtar报错：[INFO] ------------------------------------------------------------------------[INFO] BUILD FAILURE[INFO] ------------------------------------------------------------------------[INFO] Total time: 15.951s[INFO] Finished at: Fri Dec 13 18:06:44 CST 2013[INFO] Final Memory: 39M/525M[INFO] ------------------------------------------------------------------------[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-auth: Compilation failure: Compilation failure:[ERROR] /Users/JuneMAC/hadoop/release-2.2.0/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[88,11] 错误: 无法访问AbstractLifeCycle[ERROR] 找不到org.mortbay.component.AbstractLifeCycle的类文件[ERROR] /Users/JuneMAC/hadoop/release-2.2.0/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[96,29] 错误: 无法访问LifeCycle[ERROR] 找不到org.mortbay.component.LifeCycle的类文件[ERROR] /Users/JuneMAC/hadoop/release-2.2.0/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[98,10] 错误: 找不到符号[ERROR] 符号:  方法 start()[ERROR] 位置: 类型为Server的变量 server[ERROR] /Users/JuneMAC/hadoop/release-2.2.0/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[104,12] 错误: 找不到符号[ERROR] -> [Help 1][ERROR][ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR][ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1]http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException[ERROR][ERROR] After correcting the problems, you can resume the build with the command[ERROR]  mvn -rf :hadoop-auth相关讨论改错误的链接：https://issues.apache.org/jira/browse/HADOOP-10110解决方案：vi hadoop-common-project/hadoop-auth/pom.xml在org.mortbay.jettyjettytest 这段之前加一段： org.mortbay.jetty jetty-util testmvn3 package -Pdist,native -DskipTests -Dtarhadoop-auth终于成功了，然后报错：[INFO] ------------------------------------------------------------------------[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1][ERROR][ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR][ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1]http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException[ERROR][ERROR] After correcting the problems, you can resume the build with the command[ERROR]  mvn -rf :hadoop-common看到：http://blog.csdn.net/bamuta/article/details/13506893要先装protobuf，但是macports找不到这个安装，貌似只能手动安装在https://code.google.com/p/protobuf/downloads/list下载protobuf-2.5.0.tar.gz在/opt/local/var/macports/software/下建立protobuf文件夹在这个文件夹下解压缩protobuf-2.5.0.tar.gzsudo tar -zxvfprotobuf-2.5.0.tar.gzsudo sh configuresudomakesudo make checksudo make installprotoc --version  //libprotoc 2.5.0export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/libbin的路径：/usr/local/bin再次 mvn3 package -Pdist,native -DskipTests -Dtar新错误：[INFO] ------------------------------------------------------------------------[INFO] BUILD FAILURE[INFO] ------------------------------------------------------------------------[INFO] Total time: 1:33.033s[INFO] Finished at: Fri Dec 13 22:49:22 CST 2013[INFO] Final Memory: 45M/350M[INFO] ------------------------------------------------------------------------[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 2 -> [Help 1][ERROR][ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR][ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1]http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException[ERROR][ERROR] After correcting the problems, you can resume the build with the command[ERROR]  mvn -rf :hadoop-common根据http://blog.csdn.net/laozitianxia/article/details/8786922安装forresthttp://forrest.apache.org/mirrors.cgi下载：apache-forrest-0.9-sources.tar.gzapache-forrest-0.9-dependencies.tar.gz解压apache-forrest-0.9-sources.tar.gz然后将apache-forrest-0.9-dependencies.tar.gz解压后，tools/下的文件整合进/opt/local/var/macports/software/apache-forrest/apache-forrest-0.9/tools/文件夹下修改forrest环境变量export FORREST_HOME=/opt/local/var/macports/software/apache-forrest/apache-forrest-0.9export PATH=$PATH:$FORREST_HOME/binforrest -versionApache Forrest.  Run 'forrest -projecthelp' to list optionsApache Ant version 1.7.1 compiled on September 26 2008  再次 mvn3 package -Pdist,native -DskipTests -Dtar新错误：[INFO] ------------------------------------------------------------------------[INFO] BUILD FAILURE[INFO] ------------------------------------------------------------------------[INFO] Total time: 1:56.366s[INFO] Finished at: Sat Dec 14 00:59:51 CST 2013[INFO] Final Memory: 58M/442M[INFO] ------------------------------------------------------------------------[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 2 -> [Help 1][ERROR][ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR][ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1]http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException[ERROR][ERROR] After correcting the problems, you can resume the build with the command[ERROR]  mvn -rf :hadoop-common根据http://blog.csdn.net/laozitianxia/article/details/8786922安装findbugs在http://findbugs.sourceforge.net/downloads.html下载findbugs-2.0.3.tar.gz解压缩貌似直接能用，配置下环境变量export FINDBUGS_HOME=/opt/local/var/macports/software/findbugs/findbugs-2.0.3export PATH=$PATH:$FINDBUGS_HOME/bin findbugs -version2.0.3安装findbugs在http://findbugs.sourceforge.net/downloads.html下载findbugs-2.0.3.tar.gz解压缩貌似直接能用，配置下环境变量export FINDBUGS_HOME=/opt/local/var/macports/software/findbugs/findbugs-2.0.3export PATH=$PATH:$FINDBUGS_HOME/bin 重启终端findbugs -version2.0.3安装ANT在http://ant.apache.org/bindownload.cgi下载apache-ant-1.9.2-bin.tar.gz配置下环境变量# Setting Path for Antexport ANT_HOME=/opt/local/var/macports/software/apache-ant/apache-ant-1.9.2export PATH=$PATH:$ANT_HOME/bin  重启终端ant -versionApache Ant(TM) version 1.9.2 compiled on July 8 2013 再次sudomvn3 package -Pdist,native -DskipTests -Dtar新错误：/bin/sh: /Users/JuneMAC/hadoop/release-2.2.0/hadoop-common-project/hadoop-common/${env.JAVA_HOME}/bin/javah: bad substitution [INFO] ------------------------------------------------------------------------[INFO] BUILD FAILURE[INFO] ------------------------------------------------------------------------[INFO] Total time: 29.800s[INFO] Finished at: Sat Dec 14 20:14:22 CST 2013[INFO] Final Memory: 36M/96M[INFO] ------------------------------------------------------------------------[ERROR] Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-7:javah (default) on project hadoop-common: Error running javah command: Error executing command line. Exit code:1 -> [Help 1][ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1]http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException[ERROR] [ERROR] After correcting the problems, you can resume the build with the command[ERROR]  mvn -rf :hadoop-common修改hadoop-common-project/hadoop-common/pom.xml 文件中，将env.JAVA_HOME改为java.home再次sudomvn3 package -Pdist,native -DskipTests -Dtar新错误：[exec] /Users/JuneMAC/hadoop/release-2.2.0/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c:77:26: error: invalid operands to binary expression_r('void' and 'int') [exec]  if(setnetgrent(cgroup) == 1) { [exec]  ~~~~~~~~~~~~~~~~~~~ ^  ~ [exec] 1 error generated. [exec] make[2]: *** [CMakeFiles/hadoop.dir/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c.o] Error 1 [exec] make[1]: *** [CMakeFiles/hadoop.dir/all] Error 2 [exec] make: *** [all] Error 2[INFO] ------------------------------------------------------------------------[INFO] Reactor Summary:[INFO][INFO] Apache Hadoop Main ................................ SUCCESS [1.455s][INFO] Apache Hadoop Project POM ......................... SUCCESS [1.183s][INFO] Apache Hadoop Annotations ......................... SUCCESS [2.488s][INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.400s][INFO] Apache Hadoop Project Dist POM .................... SUCCESS [2.396s][INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [3.434s][INFO] Apache Hadoop Auth ................................ SUCCESS [2.611s][INFO] Apache Hadoop Auth Examples ....................... SUCCESS [2.349s][INFO] Apache Hadoop Common .............................. FAILURE [10.596s] [INFO] ------------------------------------------------------------------------[INFO] BUILD FAILURE[INFO] ------------------------------------------------------------------------[INFO] Total time: 29.226s[INFO] Finished at: Sat Dec 14 21:07:40 CST 2013[INFO] Final Memory: 36M/94M[INFO] ------------------------------------------------------------------------[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on project hadoop-common: An Ant BuildException has occured: exec returned: 2 -> [Help 1][ERROR][ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR][ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1]http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException[ERROR][ERROR] After correcting the problems, you can resume the build with the command[ERROR]  mvn -rf :hadoop-common在hadoop群里求教之后采用如下方案：修改mvn3的配置文件：/opt/local/share/java/maven3/settings.xml在…里添加国内源： nexus-osc * Nexusoschttp://maven.oschina.net/content/groups/public/在...标签中增加以下内容： jdk-1.7 1.7 nexus local private nexushttp://maven.oschina.net/content/groups/public/ true false nexus local private nexushttp://maven.oschina.net/content/groups/public/ true false注意修改jdk version number将刚才的maven 配置文件拷贝到当前用户的home目录下: settings.xml  copy 到  your_hadoop_usr_home/.m2/cp settings.xml ~/.m2最重要的一点，build your code是使用这个command line(Only for Mac OS):mvn3 clean install -P-cbuild编译之前， 你在hadoop-2.2.0-src目录(/Users/JuneMAC/hadoop/release-2.2.0)下执行mvn3 clean install –DskipTests 上面的成功后，执行下面这个，生成安装包mvn3 clean install  package -Pdist -P-cbuild  -DskipTests  -Dtar 执行完成后，可以在/Users/JuneMAC/hadoop/release-2.2.0/hadoop-dist/target/下找到hadoop-2.2.0.tar.gz将上面这个编译好的源码包解压到：/Users/JuneMAC/hadoop/然后进行相关配置解压之后的源码包和官网下载下来的源码包相对比，没有lib目录相关解释：“Here we use the additional options to stop compiling the native code. Here we use the additional options to stop compiling the native code.this is the key reason why we need use -P-cbuild option”上面这个是原因，好像不是很重要，这点先pass了，以后遇到问题再来补充说明吧

附:(官方说明)Index: hadoop-common-project/hadoop-auth/pom.xml

===================================================================

--- hadoop-common-project/hadoop-auth/pom.xml(revision 1543124)

+++ hadoop-common-project/hadoop-auth/pom.xml(working copy)

@@ -54,6 +54,11 @@

org.mortbay.jetty

+ jetty-util

+ test

+

+

+ org.mortbay.jetty

jetty

test

详细请查看:https://issues.apache.org/jira/browse/HADOOP-10110