Kerberos（1.9）Apache Hadoop（2.0.3 Yarn） 配置

环境：

环境： ubuntu 12.04, jdk 1.7

用户名都是：nachuang

域名分别是：mcw-cc-nachuang, mcw-cc-node

注：确定 Hadoop 2.0.3 集群可以正常使用，然后stop-all。

Kerberos配置：

参考： http://blog.csdn.net/edwardvsnc/article/details/8690007

Hadoop 配置：

hadoop-env.sh，hadoopcore-site.xml和hdfs-site.xml配置参考“Hadoop(1.0.4)和Kerberos配置”：

http://blog.csdn.net/edwardvsnc/article/details/8690007

mapred-site.xml:

<configuration>  
<property>  
<name>mapreduce.framework.name</name>  
<value>yarn</value>  
</property>  
<property>  
<name>mapreduce.job.tracker</name>  
<value>hdfs:/mcw-cc-nachuang:9001</value>  
<final>true</final>  
</property>  
<property>
<name>mapred.job.map.memory.mb</name>
<value>1700</value>
</property>
<property>
<name>mapred.job.reduce.memory.mb</name>
<value>1700</value>
</property>
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx1400m</value>
</property>
<property>  
<name>mapreduce.task.io.sort.mb</name>  
<value>400</value>  
</property>  
<property>  
<name>mapreduce.task.io.sort.factor</name>  
<value>10</value>  
</property>  
<property>  
<name>mapred.system.dir</name>  
<value>file:/home/nachuang/Workspace/Hadoop/hadoop-2.0.3-alpha/mapred/system</value>  
<final>true</final>  
</property>  
<property>  
<name>mapred.local.dir</name>  
<value>file:/home/nachuang/Workspace/Hadoop/hadoop-2.0.3-alpha/mapred/local</value>  
<final>true</final>  
</property>  
<property>
<name>mapred.reduce.slowstart.completed.maps</name>
<value>1</value>
</property>
<property>
  <name>mapreduce.jobtracker.kerberos.principal</name>
  <value>nachuang/_HOST@hdfs.server</value>
</property>
<property>
  <name>mapreduce.jobtracker.kerberos.https.principal</name>
  <value>host/_HOST@hdfs.server</value>
</property>
<property>
  <name>mapreduce.jobtracker.keytab.file</name>
  <value>/home/nachuang/Workspace/Hadoop/hadoop-2.0.3-alpha/etc/nachuang.keytab</value> 
</property>
<property>
  <name>mapreduce.tasktracker.kerberos.principal</name>
  <value>nachuang/_HOST@hdfs.server</value>
</property>
<property>
  <name>mapreduce.tasktracker.kerberos.https.principal</name>
  <value>host/_HOST@hdfs.server</value>
</property>
<property>
  <name>mapreduce.tasktracker.keytab.file</name>
  <value>/home/nachuang/Workspace/Hadoop/hadoop-2.0.3-alpha/etc/nachuang.keytab</value> 
</property>
<property>
  <name>mapred.task.tracker.task-controller</name>
  <value>org.apache.hadoop.mapred.DefaultTaskController</value>
</property>
<property>
  <name>mapreduce.tasktracker.group</name>
  <value>nachuang</value>
</property>
</configuration>  

yarn-site.xml:

<?xml version="1.0"?>
<configuration>  
<property>  
<name>yarn.resourcemanager.address</name>  
<value>mcw-cc-nachuang:9080</value>  
</property>  
<property>  
<name>yarn.resourcemanager.scheduler.address</name>  
<value>mcw-cc-nachuang:9081</value>  
</property>  
<property>  
<name>yarn.resourcemanager.resource-tracker.address</name>  
<value>mcw-cc-nachuang:9082</value>  
</property>  
<property>
<name>yarn.nodemanager.vmem-pmem-ratio</name>
<value>6</value>
</property>
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>10240</value>
</property>
<!-- ResourceManager security configs -->
<property>
  <name>yarn.resourcemanager.keytab</name>
  <value>/home/nachuang/Workspace/Hadoop/hadoop-2.0.3-alpha/etc/nachuang.keytab</value>   
</property>
<property>
  <name>yarn.resourcemanager.principal</name>  
  <value>nachuang/_HOST@hdfs.server</value>
</property>
<!-- NodeManager security configs -->
<property>
  <name>yarn.nodemanager.keytab</name>
  <value>/home/nachuang/Workspace/Hadoop/hadoop-2.0.3-alpha/etc/nachuang.keytab</value>   
</property>
<property>
  <name>yarn.nodemanager.principal</name>  
  <value>nachuang/_HOST@hdfs.server</value>
</property>  
<property>  
<name>yarn.nodemanager.aux-services</name>  
<value>mapreduce.shuffle</value>  
</property>  
<property>  
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>  
<value>org.apache.hadoop.mapred.ShuffleHandler</value>  
</property>  
</configuration> 

yarn-env.sh:

export HADOOP_YARN_USER=nachuang

hadoop-policy.xml:

${HADOOP_YARN_USER} 改为 *

验证：

      启动Hadoop：

      sbin/start-all.sh  

      sudo sbin/start-secure-dns.sh(注意： datanode要用sudo执行)