hadoop安装

根据官方文档作此记录

操作系统：ubuntu 14.04，32位；
java版本：1.7。
[img]http://dl2.iteye.com/upload/attachment/0098/8153/a8bfb138-48fc-3c18-8f84-cc17356fa44e.png[/img]

[color=red][b]安装ssh： [/b][/color]

sudo apt-get install ssh
sudo apt-get install rsync

[color=red][b]下载hadoop：[/b][/color]
下载网址 [url]http://www.apache.org/dyn/closer.cgi/hadoop/common/[/url]，自己下载最新版本2.4.1；
将包移动到自己定义的目录下（我的在/home/pmonkey/hadoop），然后解压：

tar zxvf hadoop-2.4.1.tar.gz

生成hadoop-2.4.1文件，为方便操作将解压后的文件夹重命名为hadoop

mv hadoop-2.4.1 hadoop

那么hadoop目录为/home/pmonkey/hadoop/hadoop

[color=red][b]配置hadoop环境变量：[/b][/color]
修改etc/hadoop/hadoop-env.sh文件

sudo gedit etc/hadoop/hadoop-env.sh

添加配置java_home与hadoop_home：

export JAVA_HOME=/home/pmonkey/java/java7
export HADOOP_PREFIX=/home/pmonkey/hadoop/hadoop

[img]http://dl2.iteye.com/upload/attachment/0098/8159/8ab21654-c9c6-3975-9deb-ec10808a6c83.png[/img]

[color=red][b]启动：[/b][/color]
[color=red]单机模式： [/color]

mkdir input
cp etc/hadoop/*.xml input
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.1.jar grep input output 'dfs[a-z.]+'
cat output/*

[color=red]伪分布式模式：[/color]

sudo gedit etc/hadoop/core-site.xml

添加配置

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

如图：[img]http://dl2.iteye.com/upload/attachment/0098/8161/57b389ad-0fc3-34d8-ac22-45c1b9ab8077.png[/img]

sudo gedit etc/hadoop/hdfs-site.xml

添加配置

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

如图：
[img]http://dl2.iteye.com/upload/attachment/0098/8163/2f3e47e4-83fe-37f2-9d38-ee3f9f34be6d.png[/img]
[color=red]ssh免密码登录设置：[/color]
[img]http://dl2.iteye.com/upload/attachment/0098/8165/f58d2caf-e96c-3ed1-9d97-aa16dec794c1.png[/img]如果失败执行：

ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa，cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

[color=red]运行本地MapReduce：[/color]

bin/hdfs namenode -format
sbin/start-dfs.sh

[url]http://localhost:50070/[/url] 访问namenode网络接口
[color=red]配置input/output(注意路径)：[/color]

bin/hdfs dfs -mkdirs /home/pmonkey/hadoop/hadoop
bin/hdfs dfs -put etc/hadoop /home/pmonkey/hadoop/hadoop/input
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.1.jar grep /home/pmonkey/hadoop/hadoop/input /home/pmonkey/hadoop/hadoop/output 'dfs[a-z.]+'
bin/hdfs dfs -get /home/pmonkey/hadoop/hadoop/output /home/pmonkey/hadopmonkey/hadoop/hadoop/output/*

[color=red]run a MapReduce job on YARN in a pseudo-distributed mode：[/color]
配置etc/hadoop/mapred-site.xml文件

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
</configuration>

配置etc/hadoop/yarn-site.xml文件

<configuration>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
</configuration>

sbin/start-yarn.sh

[url]http://localhost:8088/[/url]访问网络接口