配置yarn
Yarn是hadoop的资源调度器,可以配置mapreduce job基于yarn运行。
1)配置etc/hadoop/mapred-site.xml(告诉mapreduce框架,最终执行时跑的位置是yarn上)
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
2)配置etc/hadoop/yarn-site.xml(yarn的nodemanager中的services用的是mapreduce——shuffle)
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
3)启动yarn相关进程
$ sbin/start-yarn.sh
4)验证
jps
http://hadoop000:8088/
5)停止yarn相关进程
sbin/stop-yarn.sh
提交mr作业到yarn上运行:
/home/hadoop/app/hadoop-2.6.0-cdh5.7.0/share/hadoop/mapreduce里面的
hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar
hadoop jar
例如:
pi是程序名称,2 3是参数