hadoop-2.6.0-cdh5.7.0 YARN 伪分布式安装

YARN on Single Node

You can run a MapReduce job on YARN in a pseudo-distributed mode by setting a few parameters and running ResourceManager daemon
and NodeManager daemon in addition.

注意：首先必须在hdfs成功安装的情况下，才能进行yarn的安装

1. Configure parameters as follows:
   etc/hadoop/mapred-site.xml:

复制模板：
cp mapred-site.xml.template mapred-site.xml

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
</configuration>

etc/hadoop/yarn-site.xml:

<configuration>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
</configuration>

2.Start ResourceManager daemon and NodeManager daemon:
$ sbin/start-yarn.sh

web界面验证：

Browse the web interface for the ResourceManager; by default it is available at:
ResourceManager - http://47.75.250.241:8088

通过jps查看进程：ResourceManager and NodeManager

[hadoop@hadoop01 hadoop]$ jps
18547 DataNode
29326 Jps
24201 ResourceManager
18424 NameNode
18714 SecondaryNameNode
3. Run some of the examples provided

Make the HDFS directories required to execute MapReduce jobs:

 $ bin/hdfs dfs -mkdir /user
 $ bin/hdfs dfs -mkdir /user/<username>

Copy the input files into the distributed filesystem:

  $ bin/hdfs dfs -put etc/hadoop input  (官方）
  $`hdfs dfs -put hdfs-site.xml  input`  （我们只copy一个文件）

默认在/user/hadoop 下面形成 input 文件 查看文件内容 hdfs dfs -cat /user/hadoop/input

Run some of the examples provided:
$ bin/hadoop jarshare/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar grep input output 'dfs[a-z.]+'

结果：
3.1.
18/09/27 15:47:16 INFO mapreduce.Job: map 0% reduce 0%
18/09/27 15:47:21 INFO mapreduce.Job: map 100% reduce 0%
18/09/27 15:47:27 INFO mapreduce.Job: map 100% reduce 100%

3.2. http://47.75.250.241:8088
可以查看是否成功

4. Copy the output files from the distributed filesystem to the local filesystem and examine them:

$ bin/hdfs dfs -get output output
$ cat output/*

or

View the output files on the distributed filesystem:

$ bin/hdfs dfs -cat output/*

5. When you’re done, stop the daemons with:

$ sbin/stop-dfs.sh