转自:http://chqz1987.blog.163.com/blog/static/51438311201301511853170/
及:http://blog.csdn.net/u010743716/article/details/9247207
(注:编译成功之后有可能eclipse也不能正确显示,我使用Version: 4.2.2正常)
最近在学习hadoop,首先第一步学着编译了下hadoop-eclipse插件,中间遇到了不少问题,但最终还是解决了。现将编译过程记录下来,便于后面使用。
这里是基于hadoop-1.0.4和eclipse Indigo版本。
这里以HADOOP_HOME来表示您电脑上hadoop的安装目录。
1、编辑${HADOOP_HOME}/src/contrib/下的build-contrib.xml文件
添加{version}和{eclipse.home}属性:
2、编辑${HADOOP_HOME}/src/contrib/ eclipse-plugin/下的build.xml文件<project name="hadoopbuildcontrib" xmlns:ivy="antlib:org.apache.ivy.ant">
<!-- 这里定义了 version & eclipse.home -->
<property name="version" value="1.0.4"/>
<property name="eclipse.home" value="/home/chqz/systool/eclipse/eclipse"/>
<property name="name" value="${ant.project.name}"/>
<property name="root" value="${basedir}"/>
<property name="hadoop.root" location="${root}/../../../"/>
(1) 添加hadoop-jars path,并同时加入到classpath中:
(2) 设置includeantruntime=on,防止compile时报warning:<!-- 这里添加了 hadoop-jars --> <path id="hadoop-jars"> <fileset dir="${hadoop.root}/"> <include name="hadoop-*.jar"/> </fileset> </path> <!-- Override classpath to include Eclipse SDK jars --> <path id="classpath"> <pathelement location="${build.classes}"/> <pathelement location="${hadoop.root}/build/classes"/> <path refid="eclipse-sdk-jars"/>
<!-- 将 hadoop-jars 添加到这里 --> <path refid="hadoop-jars"/> </path>
(3) 添加将要打包到plugin中的第三方包列表:<target name="compile" depends="init, ivy-retrieve-common" unless="skip.contrib"> <echo message="contrib: ${name}"/> <javac encoding="${build.encoding}" srcdir="${src.dir}" includes="**/*.java" destdir="${build.classes}" debug="${javac.debug}" deprecation="${javac.deprecation}"
<!-- 设置includeantruntime=on,防止compile报warning -->
includeantruntime="on"> <classpath refid="classpath"/> </javac> </target>
<!-- Override jar target to specify manifest --> <target name="jar" depends="compile" unless="skip.contrib"> <mkdir dir="${build.dir}/lib"/> <!-- 这里主要修改的是file中的值,注意路径一定要正确 --> <copy file="${hadoop.root}/hadoop-core-${version}.jar"
tofile="${build.dir}/lib/hadoop-core.jar" verbose="true"/> <copy file="${hadoop.root}/lib/commons-cli-1.2.jar"
todir="${build.dir}/lib" verbose="true"/> <copy file="${hadoop.root}/lib/commons-lang-2.4.jar"
todir="${build.dir}/lib" verbose="true"/> <copy file="${hadoop.root}/lib/commons-configuration-1.6.jar"
todir="${build.dir}/lib" verbose="true"/> <copy file="${hadoop.root}/lib/jackson-mapper-asl-1.8.8.jar"
todir="${build.dir}/lib" verbose="true"/> <copy file="${hadoop.root}/lib/jackson-core-asl-1.8.8.jar"
todir="${build.dir}/lib" verbose="true"/> <copy file="${hadoop.root}/lib/commons-httpclient-3.0.1.jar"
todir="${build.dir}/lib" verbose="true"/> <jar jarfile="${build.dir}/hadoop-${name}-${version}.jar" manifest="${root}/META-INF/MANIFEST.MF"> <fileset dir="${build.dir}" includes="classes/ lib/"/> <fileset dir="${root}" includes="resources/ plugin.xml"/> </jar> </target>
3:修改MANIFEST.MF文件。
注:这步在之前参考的网页里面没有。
文件路径是:/home/hadoop/hadoop_installs/hadoop-1.0.4/src/contrib/eclipse-plugin/META-INF/MANIFEST.MF
主要是修改Bundle-ClassPath
Bundle-ClassPath: classes/,lib/hadoop-core.jar,lib/jackson-core-asl-1.8.8.jar ,lib/jackson-mapper-asl-1.8.8.jar, lib/commons-configuration-1.6.jar,lib/commons-lang-2.4.jar, lib/commons-httpclient-3.0.1.jar
4、执行ant命令以生成hadoop-eclipse-plugin-${version}.jar包:
进入到 ${HADOOP_HOME}/src/contrib/ eclipse-plugin/目录下,
然后执行ant命令,输出结果如下:
[chqz@chqz eclipse-plugin]$ ant Buildfile: /home/chqz/plugin/hadoop/hadoop-1.0.4/src/contrib/eclipse-plugin/build.xml check-contrib: init: [echo] contrib: eclipse-plugin init-contrib: ivy-download: [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar [get] To: /home/chqz/plugin/hadoop/hadoop-1.0.4/ivy/ivy-2.1.0.jar [get] Not modified - so not downloaded ivy-probe-antlib: ivy-init-antlib: ivy-init: [ivy:configure] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ :: [ivy:configure] :: loading settings :: file = /home/chqz/plugin/hadoop/hadoop-1.0.4/ivy/ivysettings.xml ivy-resolve-common: [ivy:resolve] :: resolving dependencies :: org.apache.hadoop#eclipse-plugin;working@chqz [ivy:resolve] confs: [common] [ivy:resolve] found commons-logging#commons-logging;1.0.4 in maven2 [ivy:resolve] found log4j#log4j;1.2.15 in maven2 [ivy:resolve] :: resolution report :: resolve 152ms :: artifacts dl 4ms --------------------------------------------------------------------- | | modules || artifacts | | conf | number| search|dwnlded|evicted|| number|dwnlded| --------------------------------------------------------------------- | common | 2 | 0 | 0 | 0 || 2 | 0 | --------------------------------------------------------------------- ivy-retrieve-common: [ivy:retrieve] :: retrieving :: org.apache.hadoop#eclipse-plugin [sync] [ivy:retrieve] confs: [common] [ivy:retrieve] 0 artifacts copied, 2 already retrieved (0kB/8ms) [ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead [ivy:cachepath] :: loading settings :: file = /home/chqz/plugin/hadoop/hadoop-1.0.4/ivy/ivysettings.xml compile: [echo] contrib: eclipse-plugin jar: BUILD SUCCESSFUL Total time: 2 seconds最后成功生成的 hadoop-eclipse-plugin-${version}.jar在${HADOOP_HOME}/build/contrib/eclipse-plugin下。
drwxrwxr-x. 3 chqz chqz 4096 1月 15 22:50 classes drwxrwxr-x. 2 chqz chqz 4096 1月 15 21:49 examples -rw-rw-r--. 1 chqz chqz 5625159 1月 15 23:03 hadoop-eclipse-plugin-1.0.4.jar drwxrwxr-x. 2 chqz chqz 4096 1月 15 23:03 lib drwxrwxr-x. 3 chqz chqz 4096 1月 15 21:49 system drwxrwxr-x. 3 chqz chqz 4096 1月 15 21:49 test
5、将该jar包拷贝到eclpise安装目录下的plugins目录下
cp hadoop-eclipse-plugin.jar /opt/eclipse/plugins
6、重启eclipse,配置hadoop安装目录
7、window-->open perspective -->other -->Map/Reduce
8、打开hadoop视图
9、在大黄象的位置新建一个hadoop location
正常情况下,可以在项目区域可以看到