container-executor.cfg文件及其各级父路径问题解决方案-CSDN博客

一、背景

为了增加Hadoop CGroup来限制部分作业对CPU的占用，近日在搭建Hadoop-2.6.0的CGroup过程中，碰到了以下一个问题，提示如下（注：500为用户hadoop的id号）：

 
           File /opt/hadoop/hadoop-2.6.0/etc/hadoop/container 
           -executor 
           .cfg must be owned by root, but is owned by 500

于是，就将container-executor.cfg文件的所有者修改为root，继而又出现了以下问题：

 
           File /opt/hadoop/hadoop-2.6.0/etc/hadoop must be owned by root, but is owned by 500

这是怎么回事呢？

二、分析

原来，LinuxContainerExecutor通过container-executor来启动容器，但是出于安全的考虑，要求其所依赖的配置文件container-executor.cfg及其各级父路径所有者必须是root用户。源码中的判断如下：

/**
 * Ensure that the configuration file and all of the containing directories
 * are only writable by root. Otherwise, an attacker can change the
 * configuration and potentially cause damage.
 * returns 0 if permissions are ok
 */
int check_configuration_permissions(const char* file_name) {
  // copy the input so that we can modify it with dirname
  char* dir = strdup(file_name);
  char* buffer = dir;
  do {
    if (!is_only_root_writable(dir)) {
      free(buffer);
      return -1;
    }
    dir = dirname(dir);
  } while (strcmp(dir, "/") != 0);
  free(buffer);
  return 0;
}
  
/**
 * Is the file/directory only writable by root.
 * Returns 1 if true
 */
static int is_only_root_writable(const char *file) {
  struct stat file_stat;
  if (stat(file, &file_stat) != 0) {
    fprintf(ERRORFILE, "Can't stat file %s - %s\n", file, strerror(errno));
    return 0;
  }
  if (file_stat.st_uid != 0) {
    fprintf(ERRORFILE, "File %s must be owned by root, but is owned by %d\n",
            file, file_stat.st_uid);
    return 0;
  }
  if ((file_stat.st_mode & (S_IWGRP | S_IWOTH)) != 0) {
    fprintf(ERRORFILE,
        "File %s must not be world or group writable, but is %03o\n",
        file, file_stat.st_mode & (~S_IFMT));
    return 0;
  }
  return 1;
}

在check_configuration_permissions函数中，对配置文件container-executor.cfg路径dir，在一个do循环内循环调用is_only_root_writable函数检测及所有者必须是root用户，否则不予启动容器。

而Hadoop-2.6.0在编译时，是通过如下方式确定配置文件container-executor.cfg的位置的，首先，在hadoop-yarn-server-nodemanager的pom.xml中，设置了一个名为container-executor.conf.dir的properties，其值为yarn.basedir/etc/hadoop，实际上就是$HADOOP_HOME/etc/hadoop/，如下：

 
           < 
           properties 
           > 
          
           <!-- Basedir eeded for generating FindBugs warnings using parent pom --> 
          
           < 
           yarn.basedir 
           >${project.parent.parent.basedir}</ 
           yarn.basedir 
           > 
          
           < 
           container-executor.conf.dir 
           >../etc/hadoop</ 
           container-executor.conf.dir 
           > 
          
           < 
           container-executor.additional_cflags 
           ></ 
           container-executor.additional_cflags 
           > 
          
           </ 
           properties 
           >

而在cmake的编译时用到了这个路径，将其赋值给DHADOOP_CONF_DIR并传入cmake编译环境，如下：

 
      
           < 
           exec  
           executable 
           = 
           "cmake"  
           dir 
           = 
           "${project.build.directory}/native"  
           failonerror 
           = 
           "true" 
           > 
          
 
                 
           < 
           arg  
           line 
           = 
           "${basedir}/src/ -DHADOOP_CONF_DIR=${container-executor.conf.dir} -DJVM_ARCH_DATA_MODEL=${sun.arch.data.model}" 
           /> 
          
 
                 
           < 
           env  
           key 
           = 
           "CFLAGS"  
           value 
           = 
           "${container-executor.additional_cflags}" 
           /> 
          
 
           </ 
           exec 
           > 
          
 
    

这就有一个问题，要么我们把$HADOOP_HOME各级父目录及其到container-executor.cfg的各级子目录设置其所有者为root，要么我们就得修改源码，重设路径，然后重新编译Hadoop-2.6.0，命令如下：

 
      
           mvn package  
           -Pdist 
           ,native  
           -DskipTests  
           -Dtar  
           -Dcontainer 
           -executor 
           .conf. 
           dir 
           =/etc/hadoop 
          
 
    

这两种方案都不是很好，前者隐患太大，后者比较麻烦，需要maven、protobuf等环境。

三、方案

通过cmake -DHADOOP_CONF_DIR=/etc/hadoop重新编译container-executor即可，步骤如下：

 
      
           cd  
           /tmp/lp_test 
          
 
           tar  
           -zxf  
           hadoop-2.6.0 
           -src 
           .tar.gz 
          
 
           chown  
           -R  
           root:root hadoop-2.6.0 
           -src 
          
 
           cd  
           /tmp/lp_test/hadoop-2.6.0 
           -src 
           /hadoop 
           -yarn 
           -project 
           /hadoop 
           -yarn 
           /hadoop 
           -yarn 
           -server 
           /hadoop 
           -yarn 
           -server 
           -nodemanager 
           / 
          
 
           cmake src -DHADOOP_CONF_DIR=/etc/hadoop 
          
 
           make 
          
 
    