Hadoop 伪分布式安装

1. 安装额外的软件：java、sshd
   JAVA_HOME：/usr/java/jdk1.8.0_73
   多网卡启动：ifup 网卡名

2. 在进行web访问之前需要完成防火墙的关闭
   systemctl stop firewalld.service

3. 解压hadoop文件，在etc/hadoop/hadoop-env.sh中定义JAVA_HOME值

4. 修改配置文件：
   core-site.xml

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://hadoop:9000</value>
    </property>
</configuration>

hdfs-site.xml

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

mapred-site.xml

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
</configuration>

yarn-site.xml

<configuration>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
</configuration>

---

格式化的时候，需要对所有的文件进行删除，包括logs下的文件，否则出现错误，error信息如下：

...
16/02/27 21:33:30 INFO mapreduce.Job: Job job_1456578825973_0005 failed with state FAILED due to: Application application_1456578825973_0005 failed 2 times due to AM Container for appattempt_1456578825973_0005_000002 exited with  exitCode: -1000
For more detailed output, check application tracking page:http://hadoop:8088/cluster/app/application_1456578825973_0005Then, click on links to logs of each attempt.
Diagnostics: Call From hadoop/127.0.0.1 to localhost:9000 failed on connection exception: java.net.ConnectException: 拒绝连接; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
java.net.ConnectException: Call From hadoop/127.0.0.1 to localhost:9000 failed on connection exception: java.net.ConnectException: 拒绝连接; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
...

在实验阶段，内存的设定不能过小，否则程序运行时间过长，且可能出现一些错误，初始值应该为1G以上，在2G的内存上，可以完成伪分布式的安装过程