[单机配置]Redhat虚拟机伪分布hadoop配置

主要参考一下两个博客，进行整理和修正

http://blog.csdn.net/yjb7268888/article/details/51766855

http://blog.csdn.net/yjb7268888/article/details/52874174

一、安装jdk

安装包：jdk-8u73-Linux-x64.rpm

 ①解压文件：rpm -ivh jdk-8u73-linux-x64.rpm

 ②打开环境变量 vi /etc/profile

     配置以下内容：export JAVA_HOME=/usr/java/jdk1.8.0_73
 export PATH=$JAVA_HOME/bin:$PATH
 export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

 ③配置生效： source /etc/profile

 ④查看java版本：java -version

    输出内容为 Java version "1.8.0_73"
   Java(TM) SE Runtime Environment (build 1.8.0_73-b02)
   Java HotSpot(TM) 64-Bit Server VM (build 25.73-b02, mixed mode)
   java版本信息正常，表明jdk正确安装。

二、ssh免密码登陆，本人的博客中已经介绍过，不赘述

三、安装hadoop

 版本：hadoop-2.7.2.tar.gz

 ①解压：tar -zxvf hadoop-2.7.2.tar.gz 

 ②修改hadoop-env.sh增加路径：export JAVA_HOME=/usr/java/jdk1.8.0_73

 ③修改xml文件：路径在../etc/hadoop中

->core-site.xml 

<configuration>
        <property>
                <name>fs.defaultFS</name>
                <value>hdfs://192.168.85.132:9000</value>
        </property>
        <property>
                <name>hadoop.tmp.dir</name>
                <value>file:/home/yao/hadoop/hadoop-2.7.3/tmp</value>
        </property>
</configuration>

其中192.168.85.132为我本机的ip，/home/yao/hadoop/hadoop-2.7.3/tmp为我自己建的tmp文件夹，用来存放该部分产生的临时文件

->hdfs-site.xml

<configuration>
        <property>
                <name>dfs.replication</name>
                <value>1</value>
        </property>
        <property>
                <name>dfs.name.dir</name>
                <value>file:/home/yao/hadoop/hadoop-2.7.3/yxjdfs/name</value>
        </property>
        <property>
                <name>dfs.data.dir</name>
                <value>file:/home/yao/hadoop/hadoop-2.7.3/yxjdfs/data</value>
        </property>
 </configuration>

->mapred-site.xml

这部分需要先执行mv mapred-site.xml.template mapred-site.xml

<configuration>
        <property>
                <name>mapreduce.framework.name</name>
                <value>yarn</value>
        </property>
</configuration>

->yarn-site.xml

<configuration>
        <property>
                <name>yarn.nodemanager.aux-services</name>
                <value>mapreduce_shuffle</value>
        </property>
</configuration>

 ④启动Hadoop

      sbin/start-dfs.sh

 [yao@node0 hadoop-2.7.3]$ sbin/start-dfs.sh 
Starting namenodes on [node0]
node0: starting namenode, logging to /home/yao/hadoop/hadoop-2.7.3/logs/hadoop-yao-namenode-node0.out
localhost: starting datanode, logging to /home/yao/hadoop/hadoop-2.7.3/logs/hadoop-yao-datanode-node0.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /home/yao/hadoop/hadoop-2.7.3/logs/hadoop-yao-secondarynamenode-node0.out

    sbin/start-yarn.sh

[yao@node0 hadoop-2.7.3]$ sbin/start-yarn.sh 
starting yarn daemons
starting resourcemanager, logging to /home/yao/hadoop/hadoop-2.7.3/logs/yarn-yao-resourcemanager-node0.out
localhost: starting nodemanager, logging to /home/yao/hadoop/hadoop-2.7.3/logs/yarn-yao-nodemanager-node0.out

   执行jps查看启动的服务

[yao@node0 hadoop-2.7.3]$ jps
26377 Jps
25965 ResourceManager
25468 NameNode
25619 DataNode
25806 SecondaryNameNode
26092 NodeManager

注意：执行序列化 bin/hdfs namenode -format时 师姐建议把新建的tmp文件夹和新建的yxjdfs中的name和data文件夹进行删除，重新执行后重新生成，不然后期可能存在以下小问题。

  对应的sbin/stop-dfs.sh   sbin/stop-yarn.sh关闭服务

遇到的问题：

1. sbin/start-dfs.sh后出现converse什么什么的错误，

127.0.0.1确认是否为主机号

2. jps后少出现一个namenode,删除tmp yxjdfs/name yxjdfs/data文件夹，运行序列化代码，重新start服务，jps就可以看到六个服务