hadoop2.2.0遇到64位操作系统平台报错，重新编译hadoop

hadoop cmake  maven protobuf

问题描述

在64位linux装的hadoop，在很多地方会遇到libhadoop.so.1.0.0 which might have disabled stack guard. 是因为hadoop是32位的，需要手工编译hadoop。

hadoop为2.2.0，操作系统为oracle linux 6.3 64位。

说明：经过实测，hadoop2.6.0在Oracle Linux也能用同样的方法编译。

实例和解决过程。

遇到的问题

[hadoop@hadoop01 input]$ hadoop dfs -put ./in

DEPRECATED: Use of this script to executehdfs command is deprecated.

Instead use the hdfs command for it.

 

Java HotSpot(TM) 64-BitServer VM warning: You have loaded library/app/hadoop/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0 which might havedisabled stack guard. The VM will try to fix the stack guard now.

It's highly recommendedthat you fix the library with 'execstack -c <libfile>', or link it with'-z noexecstack'.

13/10/24 04:08:55 WARNutil.NativeCodeLoader: Unable to load native-hadoop library for yourplatform... using builtin-java classes where applicable

put: `in': No such file or directory

 

查看本地文件

[hadoop@hadoop01 input]$ file /app/hadoop/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0

/app/hadoop/hadoop-2.2.0/lib/native/libhadoop.so.1.0.0:ELF 32-bit LSB shared object, Intel 80386,version 1 (SYSV), dynamically linked, not stripped

 

貌似是32位和64位的原因

http://mail-archives.apache.org/mod_mbox/hadoop-user/201208.mbox/%3C19AD42E3F64F0F468A305399D0DF39D92EA4521578@winops07.win.compete.com%3E

http://www.mail-archive.com/common-issues@hadoop.apache.org/msg52576.html

操作系统64位，软件是32位。悲剧了。。。装好的集群没法用。

 

 

解决方法：重新编译hadoop

解决方法，就是重新编译hadoop软件：

下载程序代码

机器得连网，如果没联网找可以联网的机器下载，但是编译时还是要下载一些东西，所以，实在不行。最好找相同平台（可以是虚拟机）能上网的机器做下面工作，弄好了再拷回来。

 

# svn checkout'http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.2.0'

 

都下载到这了：

[hadoop@hadoop01 hadoop]$ ls

BUILDING.txt       hadoop-common-project     hadoop-maven-plugins  hadoop-tools

dev-support        hadoop-dist               hadoop-minicluster    hadoop-yarn-project

hadoop-assemblies  hadoop-hdfs-project       hadoop-project        pom.xml

hadoop-client      hadoop-mapreduce-project  hadoop-project-dist

安装开发环境

1.必要的包

[root@hadoop01 /]# yum install svn

[root@hadoop01 ~]# yum install autoconf automake lib