【hadoop-hdfs】2.2、hadoop伪分布式（Pseudo-Distributed Operation）：yarn

1、配置mapred-site.xml

<!-- MapReduce执行委托给yarn --> 
<property> 
    <name>mapreduce.framework.name</name> 
    <value>yarn</value> 
</property>

2、配置yarn-site.xml

<!-- 配置nodemanager取数据的方式 --> 
<property> 
    <name>yarn.nodemanager.aux-services</name> 
    <value>mapreduce_shuffle</value> 
</property> 
<!-- 配置resourcemanager地址 --> 
<property> 
    <name>yarn.resourcemanager.hostname</name> 
    <value>hadoop101</value> 
</property> 
<!-- 解决hadoop找不到或无法加载主类 
    org.apache.hadoop.mapreduce.v2.app.MRAppMaster 
    bin下执行./hadoop classpath 将结果复制到value中 --> 
<property> 
    <name>yarn.application.classpath</name> 
    <value>/wy_root/hadoop-3.2.0/etc/hadoop:/wy_root/hadoop-3.2.0/share/hadoop/common/lib/*:/wy_root/hadoop-3.2.0/share/hadoop/common/*:/wy_root/hadoop-3.2.0/share/hadoop/hdfs:/wy_root/hadoop-3.2.0/share/hadoop/hdfs/lib/*:/wy_root/hadoop-3.2.0/share/hadoop/hdfs/*:/wy_root/hadoop-3.2.0/share/hadoop/mapreduce/lib/*:/wy_root/hadoop-3.2.0/share/hadoop/mapreduce/*:/wy_root/hadoop-3.2.0/share/hadoop/yarn:/wy_root/hadoop-3.2.0/share/hadoop/yarn/lib/*:/wy_root/hadoop-3.2.0/share/hadoop/yarn/*</value>
</property>

3、启动、执行

a、启动hdfs

/wy_root/hadoop-3.2.0/sbin/start-dfs.sh

b、启动yarn

/wy_root/hadoop-3.2.0/sbin/start-yarn.sh

c、执行MapReduce

参考

http://note.youdao.com/noteshare?id=575b8cab78cd90aefa9196b900b36cc6&sub=B245B031728C4C40A139DAFFCFAA9327

d、查看管理页面执行过程

http://hadoop103:8088

 

tip：解决hadoop 找不到或无法加载主类org.apache.hadoop.mapreduce.v2.app.MRAppMaster

https://www.jianshu.com/p/01b51bd1c77a

a、执行

/wy_root/hadoop-3.2.0/bin/hadoop classpath

b、将运行结果配置到yarn-site.xml

<!-- 解决hadoop找不到或无法加载主类 org.apache.hadoop.mapreduce.v2.app.MRAppMaster --> 
<property> 
    <name>yarn.application.classpath</name> 
    <value>/wy_root/hadoop-3.2.0/etc/hadoop:/wy_root/hadoop-3.2.0/share/hadoop/common/lib/*:/wy_root/hadoop-3.2.0/share/hadoop/common/*:/wy_root/hadoop-3.2.0/share/hadoop/hdfs:/wy_root/hadoop-3.2.0/share/hadoop/hdfs/lib/*:/wy_root/hadoop-3.2.0/share/hadoop/hdfs/*:/wy_root/hadoop-3.2.0/share/hadoop/mapreduce/lib/*:/wy_root/hadoop-3.2.0/share/hadoop/mapreduce/*:/wy_root/hadoop-3.2.0/share/hadoop/yarn:/wy_root/hadoop-3.2.0/share/hadoop/yarn/lib/*:/wy_root/hadoop-3.2.0/share/hadoop/yarn/*</value> 
</property>

c、重启yarn