ubuntu版本12.04 64位
1、首先要下载安装hadoop,我下载的是hadoop-1.2.1.tar.gz,这个版本比较稳定,下载地址可以到官网提供的镜像http://www.apache.org/dyn/closer.cgi/hadoop/common/
2、然后安装hadoop-eclipse
我用最简单的方法安装eclipse和JDK
sudo apt-get install cdt-eclipse
3、创建hadoop管理员账号
sudo adduser hadoop
然后要求你设置hadoop帐户密码,这个命令是添加一个名为hadoop的标准帐户,我们需要的是管理员帐号
可以直接在图形界面下修改hadoop权限,将鼠标点击右上角的一个人头处,浮现列表,点击“用户账户”,解锁,然后更改为管理员权限
4、安装ssh服务
sudo apt-get install ssh openssh-server
5、ssh无密码登陆
首先用刚创建的管理员hadoop重登系统
ssh-keygen -t rsa -P ""
cd ~/.ssh
cp id_rsa.pub authorized_keys
然后就可以无密码登陆了
ssh localhost
exit
6、解压hadoop源码包
cp hadoop-1.2.1.tar.gz /home/hadoop
tar -xzvf *.tag.gz
7、配置hadoop的hadoop/conf下的hadoop-env.sh,core-site.xml,mapred-site.xml,hdfs-site.xml
配置hadoop-1.2.1/conf/hadoop-env.sh
vim /home/hadoop/hadoop-1.2.1/conf/hadoop-env.sh
搜索到JAVA_HOME
把前面的#去掉,加上本系统jdk路径,保存退出
查看本系统的JDK路径,可以通过如下命令
sudo update-alternatives --config java(我的路径是/usr/lib/jvm/java-6-openjdk-amd64)
# The java implementation to use. Required.
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk-amd64
配置hadoop-1.2.1/conf/core-site.xml
gedit /home/hadoop/hadoop-1.2.1/conf/core-site.xml
在hadoop新建hadoop_tmp目录(我的是/home/hadoop/hadoop-1.2.1/hadoop_tmp)原因是http://blog.csdn.net/bychjzh/article/details/7830508
将如下<configuration> </configuration>之间的添加进入,保存退出
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>fs.default.name</name> <value>hdfs://localhost:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/home/hadoop/hadoop-1.2.1/hadoop_tmp</value> <description>A base for other temporary directories.</description> </property> </configuration>配置hadoop-1.2.1/conf/mapre-site.xml,命令行:
vim /home/hadoop/hadoop-1.2.1/conf/mapre-site.xml.xml将如下<configuration> </configuration>之间的添加进入,保存退出
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>mapred.job.tracker</name> <value>localhost:9001</value> </property> </configuration>配置hadoop-1.2.1/conf/hdfs-site.xml,命令行:
gedit /home/hadoop/hadoop-1.2.1/conf/hdfs-site.xml将如下<configuration> </configuration>之间的添加进入,保存退出
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>dfs.replication</name> <value>1</value> </property> </configuration>
首先安装ant
sudo apt-get install ant
然后配置hadoop
修改配置文件
在hadoop解压目录下,找到src\contrib\eclipse-plugin\build.xml(我的是/home/hadoop/hadoop-1.2.1/src/contrib/eclipse-plugin/build.xml),修改以下几行:
找到src\contrib\build-contrib.xml(我的是/home/hadoop/hadoop-1.2.1/build/contrib/build-contrib.xml),添加以下几行:<path id="hadoop-core-jar"> <fileset dir="${hadoop.root}/"> <include name="hadoop*.jar"/> </fileset> </path> <!-- Override classpath to include Eclipse SDK jars --> <path id="classpath"> <pathelement location="${build.classes}"/> <pathelement location="${hadoop.root}/build/classes"/> <path refid="eclipse-sdk-jars"/> <path refid="hadoop-core-jar"/> </path> ...... <target name="jar" depends="compile" unless="skip.contrib"> <mkdir dir="${build.dir}/lib"/> <copy file="${hadoop.root}/hadoop-core-${version}.jar" tofile="${build.dir}/lib/hadoop-core.jar" verbose="true"/> <copy file="${hadoop.root}/lib/commons-cli-1.2.jar" todir="${build.dir}/lib" verbose="true"/> <copy file="${hadoop.root}/lib/commons-lang-2.4.jar" todir="${build.dir}/lib" verbose="true"/> <copy file="${hadoop.root}/lib/commons-configuration-1.6.jar" todir="${build.dir}/lib" verbose="true"/> <copy file="${hadoop.root}/lib/jackson-mapper-asl-1.8.8.jar" todir="${build.dir}/lib" verbose="true"/> <copy file="${hadoop.root}/lib/jackson-core-asl-1.8.8.jar" todir="${build.dir}/lib" verbose="true"/> <copy file="${hadoop.root}/lib/commons-httpclient-3.0.1.jar" todir="${build.dir}/lib" verbose="true"/> <jar jarfile="${build.dir}/hadoop-${name}-${version}.jar" manifest="${root}/META-INF/MANIFEST.MF"> <fileset dir="${build.dir}" includes="classes/ lib/"/> <fileset dir="${root}" includes="resources/ plugin.xml"/> </jar> </target>
<property name="version" value="1.2.1"/> <property name="ivy.version" value="2.1.0"/> <property name="eclipse.home" location="/home/hadoop/eclipse"/> //这里就是eclipse的目录,不是工作目录将src/contrib/eclipse-plugin/META-INF目录下的MANIFEST.MF文件中的(我的是/home/hadoop/hadoop-1.2.1/src/contrib/eclipse-plugin/META-INF/MANIFEST.MF)
Bundle-ClassPath: classes/,
lib/hadoop-core.jar
修改为
然后,进入目录\src\contrib\eclipse-plugin(我的是/home/hadoop/hadoop-1.2.1/src/contrib/eclipse-plugin),输入ant编译,如果一切正常就编译通过了Bundle-ClassPath: classes/, lib/hadoop-core.jar, lib/commons-cli-1.2.jar, lib/commons-configuration-1.6.jar, lib/jackson-core-asl-1.8.8.jar, lib/commons-httpclient-3.0.1.jar, lib/jackson-mapper-asl-1.8.8.jar, lib/commons-lang-2.4.jar
最后可以在{hadoophome}\build\contrib\eclipse-plugin(我的是/home/hadoop/hadoop-1.2.1/build/contrib/eclipse-plugin)路径下找到编译好的插件。
然后将编译好的插件hadoop-eclipse-plugin-1.2.1.jar 拷贝到/home/hadoop/eclipse/plugins,启动eclipse
./eclipse -clean
配置eclipse
首先是 window->perferences
然后是New Hadoop location,在General配置端口号
最后在Advanced parameters中配置core-site.xml里的hadoop.tmp.dir