一、配置hadoop
总共需要修改4个文件,"hadoop-env.sh","core-site.xml"、"hdfs-site.xml"和"mapred-site.xml"。
1.hadoop-env.sh
export JAVA_HOME=/usr/java/jdk1.7.0_45
2.core-site.xml
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/luffy/Development/hadoop-${user.name}</value>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
3.hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
4.mapred-site.xml
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
</property>
</configuration>
修改之后,设置环境变量
#set Environment
export JAVA_HOME=/usr/java/jdk1.7.0_45
export HADOOP_INSTALL=/home/luffy/Development/hadoop-1.2.1
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$CLASSPATH
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$HADOOP_INSTALL/bin:$PATH
二、配置ssh免密码登陆
因为CentOS6.4自带安装ssh,所以不需要再安装,但是要配置ssh免密码登陆,需要进行设置
1.ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
2.sudo gedit /etc/selinux/config,并修改"SELINUX=disabled",然后source /etc/selinux/config
3.chmod 600 ~/.ssh/authorized_keys
chmod 700 ~/.ssh
4.至此,设置已完成,注销再登陆即可。
三、hadoop测试
1.hadoop namenode -format
2.start-all.sh
3.此时,能访问到http://localhost:50070/和http://localhost:50030/
4.cd /home/luffy/Development/hadoop-1.2.1/
hadoop fs -put conf input
5.hadoop jar hadoop-examples-*.jar grep input output 'dfs[a-z.]+'
6.hadoop fs -cat output/*
7.stop-all.sh
以上步骤中,若都没有出现各种错误,则hadoop安装成功。