因项目需要hadoop1.2.1,现将在build过程中遇到的问题和build过程记录如下
1、创建hadoop用户
2、下载hadoop1.2.1.tar.gz
3、编译hadoop1.2.1需要用到以下几个依赖库:
m4,autoconf, automake,libtool
可用yum install m4, autoconf, automake,libtool进行安装,如果没有这个依赖库,会导致构建失败
4、安装ant
笔者一开始安装是较新的ant 1.9.3,但在构建过程中遇到了问题,因此不建议安装1.9.3版本,换了1.9.0版本后问题解决。遇到的问题如下:
/home/hadoop12/hadoop-1.2.1/build.xml:565: org.apache.jasper.JasperException: java.lang.IllegalStateException: No Java compiler available
at org.apache.jasper.JspC.processFile(JspC.java:986)
at org.apache.jasper.JspC.execute(JspC.java:1094)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.TaskAdapter.execute(TaskAdapter.java:154)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:435)
at org.apache.tools.ant.Target.performTasks(Target.java:456)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
at org.apache.tools.ant.Project.executeTarget(Project.java:1364)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
at org.apache.tools.ant.Main.runBuild(Main.java:851)
at org.apache.tools.ant.Main.startAnt(Main.java:235)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
Caused by: java.lang.IllegalStateException: No Java compiler available
at org.apache.jasper.JspCompilationContext.createCompiler(JspCompilationContext.java:224)
at org.apache.jasper.JspC.processFile(JspC.java:946)
一开始笔者按网上的说法,以为是缺少jasper-jdt.jar包,结果按照其方法找到一个tomcat安装包从中取出该jar包放到lib目录下,问题依旧。试过很多其它的解决方法,最后考虑到可能是ant版本问题,换了版本后,问题解决
5、环境变量设置
在hadoop用户下编辑.bash_profile:
export HADOOP_HOME=/home/hadoop/hadoop-1.2.1
export ANT_HOME=/opt/apache-ant-1.9.0
export JAVA_HOME=/opt/jdk1.7.0_60
#export JAVA_HOME=/usr/java/jdk1.6.0_45
PATH=$PATH:$HOME/bin:$ANT_HOME/bin:$JAVA_HOME/bin
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$HADOOP_HOME/lib
export PATH
在hadoop用户的根目录下执行:source .bash_profile使得环境变量生效,或退出重新登录使得环境变量生效
6、将hadoop-1.2.1/ivy/ivy-2.1.0.jar,复制到apache-ant-1.9.0/lib目录下
7、编辑hadoop目录下的build.xml
通过查找ivy-download,找相关内容注释掉,或删除
<!--
<target name="ivy-download" description="To download ivy" unless="offline">
<get src="${ivy_repo_url}" dest="${ivy.jar}" usetimestamp="true"/>
</target>
-->
8、到hadoop1.2.1目录下:
执行:ant compile
ant
9、编译eclipse插件,可参考:http://my.oschina.net/vigiles/blog/132238#OSC_h2_7