hadoop cmake maven protobuf
问题描述
在64位linux装的hadoop,在很多地方会遇到libhadoop.so.1.0.0 which might have disabled stack guard. 是因为hadoop是32位的,需要手工编译hadoop。
hadoop为2.2.0,操作系统为oracle linux 6.3 64位。
说明:经过实测,hadoop2.6.0在Oracle Linux也能用同样的方法编译。
实例和解决过程。
遇到的问题
[hadoop@hadoop01 input]$ hadoop dfs -put ./in
DEPRECATED: Use of this script to executehdfs command is deprecated.
Instead use the hdfs command for it.
Java HotSpot(TM) 64-BitServer VM warning: You have loaded library/app/hadoop/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0 which might havedisabled stack guard. The VM will try to fix the stack guard now.
It's highly recommendedthat you fix the library with 'execstack -c <libfile>', or link it with'-z noexecstack'.
13/10/24 04:08:55 WARNutil.NativeCodeLoader: Unable to load native-hadoop library for yourplatform... using builtin-java classes where applicable
put: `in': No such file or directory
查看本地文件
[hadoop@hadoop01 input]$ file /app/hadoop/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0
/app/hadoop/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0:ELF 32-bit LSB shared object, Intel 80386,version 1 (SYSV), dynamically linked, not stripped
貌似是32位和64位的原因
http://mail-archives.apache.org/mod_mbox/hadoop-user/201208.mbox/%3C19AD42E3F64F0F468A305399D0DF39D92EA4521578@winops07.win.compete.com%3E
http://www.mail-archive.com/common-issues@hadoop.apache.org/msg52576.html
操作系统64位,软件是32位。悲剧了。。。装好的集群没法用。
解决方法:重新编译hadoop
解决方法,就是重新编译hadoop软件:
下载程序代码
机器得连网,如果没联网找可以联网的机器下载,但是编译时还是要下载一些东西,所以,实在不行。最好找相同平台(可以是虚拟机)能上网的机器做下面工作,弄好了再拷回来。
# svn checkout'http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.2.0'
都下载到这了:
[hadoop@hadoop01 hadoop]$ ls
BUILDING.txt hadoop-common-project hadoop-maven-plugins hadoop-tools
dev-support hadoop-dist hadoop-minicluster hadoop-yarn-project
hadoop-assemblies hadoop-hdfs-project hadoop-project pom.xml
hadoop-client hadoop-mapreduce-project hadoop-project-dist
安装开发环境
1.必要的包
[root@hadoop01 /]# yum install svn
[root@hadoop01 ~]# yum install autoconf automake lib