ambari-server启动流程分析

最新推荐文章于 2024-04-03 05:05:20 发布

编程小王子啊

最新推荐文章于 2024-04-03 05:05:20 发布

阅读量2.4k

点赞数 2

文章标签： ambari 大数据 hadoop spark

本文链接：https://blog.csdn.net/u012361112/article/details/84304048

版权

一、启动脚本分析

1. ambari-server 服务启动有两种方式：service ambari-server start 和 ambari-server start．分别对应脚本文件/etc/init.d/ambari-server 和 /usr/sbin/ambari-server，其中/usr/sbin/ambari-server文件是一个快捷方式，指向/etc/init.d/ambari-server：

脚本文件/etc/init.d/ambari-server，对应工程源码ambari-server/sbin/ambari-server，启动关键代码如下：

AMBARI_PYTHON_EXECUTABLE="$ROOT/usr/sbin/ambari-server.py"

case "$1" in
  start)
        echo -e "Starting ambari-server"
        $PYTHON "$AMBARI_PYTHON_EXECUTABLE" $@
        ;;

2. 从上图中可以看出，执行/etc/init.d/ambari-server start 命令，脚本会调用python执行/usr/sbin/ambari-server.py 脚本文件，对应源码ambari-server/src/main/python/ambari-server.py，ambari-server.py 脚本解析传入的命令行参数，然后执行　main(options, args, parser) 方法，核心代码如下：

@OsFamilyFuncImpl(OsFamilyImpl.DEFAULT)
def create_user_action_map(args, options):
  action_map = {
        SETUP_ACTION: UserAction(setup, options),
        START_ACTION: UserAction(start, options),
        STOP_ACTION: UserAction(stop, options),
  }
  return action_map

def main(options, args, parser):
  action_map = create_user_action_map(args, options)
  action = args[0]

  try:
    action_obj = action_map[action]
  except KeyError:
    parser.error("Invalid action: " + action)

　action_obj.execute()

此时args值是['start']，options值为空，START_ACTION是一个常量，其值在setupActions.py文件中定义：START_ACTION = "start"．调用action_obj.execute()方法实际是调用了 UserAction(start, options) 对象的方法：

class UserActionPossibleArgs(object):
  def __init__(self, i_fn, i_possible_args_numbers, *args, **kwargs):
    self.fn = i_fn
    self.possible_args_numbers = i_possible_args_numbers
    self.args = args
    self.kwargs = kwargs
    self.need_restart = False

  def execute(self):
    self.fn(*self.args, **self.kwargs)

class UserAction(UserActionPossibleArgs):
  def __init__(self, i_fn, *args, **kwargs):
    super(UserAction, self).__init__(i_fn, [1], *args, **kwargs)

从上面代码中可以看出，execute()方法去调用fn()方法，但是fn变量是由外部传入的，传入值为start，因此会去调用start()方法，start()方法代码如下：

@OsFamilyFuncImpl(OsFamilyImpl.DEFAULT)
def start(args):
  status, pid = is_server_runing()
  if status:
    err = "Ambari Server is already running."
    raise FatalException(1, err)

  server_process_main(args)

3. server_proecess_main()方法在/usr/sbin/ambari-server-main.py文件中定义，对应源码ambari-server/src/main/python/ambari-server-main.py。server_process_main()方法源码如下：

CHECK_DATABASE_HELPER_CMD = "{0} -cp {1} org.apache.ambari.server.checks.DatabaseConsistencyChecker"

def server_process_main(options, scmStatus=None):
   jdk_path = find_jdk()
　　properties = get_ambari_properties()　
　　ensure_jdbc_driver_is_installed(options, properties)
　　ensure_dbms_is_running(options, properties, scmStatus)
　　refresh_stack_hash(properties)
　　java_exe = get_java_exe_path()
　　environ = generate_env(options, ambari_user, current_user)
　　class_path=serverClassPath.get_full_ambari_classpath_escaped_for_shell(
　　　　validate_classpath=True)

　　command = CHECK_DATABASE_HELPER_CMD.format(java_exe, class_path)
   (retcode, stdout, stderr) = run_os_command(command, env=environ)

   update_properties(properties)
   param_list = generate_child_process_param_list(ambari_user, java_exe, 
     class_path, debug_start, suspend_mode,options)

   print_info_msg("Running server: " + str(param_list))
   procJava = subprocess.Popen(param_list, env=environ, 
        preexec_fn=make_process_independent)

　　pidfile = os.path.join(configDefaults.PID_DIR, PID_NAME)
   save_pid(pidJava, pidfile)　
　　wait_for_server_start(pidfile, scmStatus)
　　return procJava

get_ambari_properties()方法去加载ambari.properties文件，默认位于/etc/ambari-server/conf目录下，将其转化为 Properties 对象．

4. refresh_stack_hash()方法是去刷新stack目录下文件夹hash值：在 /var/lib/ambari-server/resources/stacks/xxx/xxx/services 目录下，集成了关于各个组件的元信息，控制脚本及配置文件，以HDFS服务为例，其文件夹内容如下：

该目录下的package文件夹存放了组件启停控制脚本文件，在agent端执行组件启停命令时，也是先将该文件夹拷贝到自己节点的cache目录下，然后执行对应脚本，package文件夹内容如下：

该文件夹有两个特别的文件archive.zip和.hash，archive.zip文件是对alerts，files，scripts，templates４个文件夹压缩而产生的压缩包，.hash文件存放archive.zip文件的hash值．agent端拷贝package文件夹时，只拷贝了archive,zip文件和.hash文件．当修改了package文件夹下的文件内容时，ambari-server进程重启会重新生成archive.zip文件和.hash文件，而agent端在执行命令时，会先去比对两个.hash文件内容是否一致，如果不一致，则重新拷贝这两个文件到cache目录下．

5. 接下来的代码是去执行数据库的数据一致性检测：

command = CHECK_DATABASE_HELPER_CMD.format(java_exe, class_path)
(retcode, stdout, stderr) = run_os_command(command, env=environ)

它调用了org.apache.ambari.server.checks.DatabaseConsistencyChecker类的main()方法进行数据库一致性检测，防止数据错乱的问题．

6. 接着方法调用了generate_child_param_list()方法生成命令参数列表，该方法主要逻辑如下：

SERVER_START_CMD = "{0} " \
    "-server -XX:NewRatio=3 " \
    "-XX:+UseConcMarkSweepGC " + \
    "-XX:-UseGCOverheadLimit -XX:CMSInitiatingOccupancyFraction=60 " \
    "-XX:+CMSClassUnloadingEnabled " \
    "-Dsun.zip.disableMemoryMapping=true " + \
    "{1} {2} " \
    "-cp {3} "\
    "org.apache.ambari.server.controller.AmbariServer {4}" \
    "> {5} 2>&1 || echo $? > {6}"

@OsFamilyFuncImpl(OsFamilyImpl.DEFAULT)
def generate_child_process_param_list(ambari_user, java_exe, class_path,
                                      debug_start, suspend_mode,options=None):
  from ambari_commons.os_linux import ULIMIT_CMD

  properties = get_ambari_properties()

  command_base = SERVER_START_CMD_DEBUG if debug_start else SERVER_START_CMD

  command = command_base.format(java_exe,
          ambari_provider_module_option,
          jvm_args,
          class_path,
          startMode,
          configDefaults.SERVER_OUT_FILE,
          os.path.join(configDefaults.PID_DIR, EXITCODE_NAME),
          suspend_mode)
  param_list.append(cmd)
  return param_list

generate_child_param_list()方法里最主要的是格式化SERVER_START_CMD命令来生成启动子进程的param_list，从该命令中得出Java程序入口类是org.apache.ambari.server.controller.AmbariServer。

接下来便是启动java子进程的代码：

print_info_msg("Running server: " + str(param_list))
procJava = subprocess.Popen(param_list, env=environ, 
        preexec_fn=make_process_independent)

pidfile = os.path.join(configDefaults.PID_DIR, PID_NAME)
save_pid(pidJava, pidfile)　
wait_for_server_start(pidfile, scmStatus)
return procJava

方法调用python的subprocess模块启动子进程，然后保存pid到文件中，等待进程成功启动后返回，至此整个启动流程结束。

二、java 启动流程

1. 由上面的启动流程分析得出Java程序启动类是org.apache.ambari.server.controller.AmbariServer，先来看看它的main()方法：

public static void main(String[] args) throws Exception {
    Injector injector = Guice.createInjector(new ControllerModule(), new 
    AuditLoggerModule());

    AmbariServer server = null;
    try {
      LOG.info("Getting the controller");

      setupProxyAuth();
      injector.getInstance(ExceptionMessageI18nHandler.class);
      injector.getInstance(GuiceJpaInitializer.class);
      DatabaseConsistencyCheckHelper.checkDBVersionCompatible();
      server = injector.getInstance(AmbariServer.class);
      CertificateManager certMan = injector.getInstance(CertificateManager.class);
      certMan.initRootCert();
      KerberosChecker.checkJaasConfiguration();
      ViewRegistry.initInstance(server.viewRegistry);
      ComponentSSLConfiguration.instance().init(server.configs);

      server.run();
   }
}

main()方法首先使用google guice框架实例化了一个injector对象，ControllerModule类构造方法：

public ControllerModule() throws Exception {
    configuration = new Configuration();
    hostsMap = new HostsMap(configuration);
    os_family = new OsFamily(configuration);
}


public class Configuration {
    public Configuration() {
        this(readConfigFile());
    }

    public Configuration(Properties properties) {
        this.properties = properties;

        agentConfigsMap = new HashMap<String, String>();
        agentConfigsMap.put(CHECK_REMOTE_MOUNTS_KEY, properties.getProperty(
          CHECK_REMOTE_MOUNTS_KEY, CHECK_REMOTE_MOUNTS_DEFAULT));
        agentConfigsMap.put(CHECK_MOUNTS_TIMEOUT_KEY, properties.getProperty(
            CHECK_MOUNTS_TIMEOUT_KEY, CHECK_MOUNTS_TIMEOUT_DEFAULT));
    } 

    private static Properties readConfigFile() {
        Properties properties = new Properties();
        InputStream inputStream =         
         Configuration.class.getClassLoader().getResourceAsStream(CONFIG_FILE);

        if (inputStream == null) {
          throw new RuntimeException(CONFIG_FILE + " not found in classpath");
        }
        return properties;
    }  

}

ControllerModule()方法里面首先调用Configuration()构造方法，而Configuration()构造方法会去读取程序配置文件---位于classpath下的CONFIG_FILE文件，而CONFIG_FILE常量被定义为ambari.properties，也就是说会去读取classpath下的ambari.properties文件，然后转换为Properies对象。
通过server = injector.getInstance(AmbariServer.class)初始化了一个AmbariServer对象，AmbariServer类没有显示声明构造方法，类的成员属性如下：

public class AmbariServer {
  
  private Server server = null;
  public volatile boolean running = true; // true while controller runs
  @Inject
  Configuration configs;
  @Inject
  CertificateManager certMan;
  @Inject
  Injector injector;
  @Inject
  AmbariMetaInfo ambariMetaInfo;
  @Inject
  MetainfoDAO metainfoDAO;
  @Inject
  private ServiceManager serviceManager;
  @Inject
  ViewRegistry viewRegistry;
  @Inject
  AmbariHandlerList handlerList;
}

可以看出，AmbariServer类很多属性都是由guice框架自动注入。来看一下MetainfoDao类的构造方法：

public AmbariMetaInfo(Configuration conf) throws Exception {
    this.conf = conf;
    String stackPath = conf.getMetadataPath();
    stackRoot = new File(stackPath);

    String commonServicesPath = conf.getCommonServicesPath();
    if(commonServicesPath != null && !commonServicesPath.isEmpty()) {
      commonServicesRoot = new File(commonServicesPath);
    }

    String extensionsPath = conf.getExtensionsPath();
    if (extensionsPath != null && !extensionsPath.isEmpty()) {
      extensionsRoot = new File(extensionsPath);
    }

    String serverVersionFilePath = conf.getServerVersionFilePath();
    serverVersionFile = new File(serverVersionFilePath);

    customActionRoot = new File(conf.getCustomActionDefinitionPath());
  }
}

public void init() throws Exception {
    // Need to be initialized before all actions
    ALL_SUPPORTED_OS = new ArrayList<String>(osFamily.os_list());

    readServerVersion();

    stackManager = stackManagerFactory.create(stackRoot, commonServicesRoot, 
    extensionsRoot,
        osFamily, false);

    getCustomActionDefinitions(customActionRoot);
}

AmbariMetaInfo()方法会去寻找stack，commonservice的文件根路径。接着会由guice框架自动调用init()方法。init()方法会去读取服务器版本，然后初始化StackManager类，StackManager()构造方法如下：

public StackManager(@Assisted("stackRoot") File stackRoot,
      @Assisted("commonServicesRoot") @Nullable File commonServicesRoot,
      @Assisted("extensionRoot") @Nullable File extensionRoot,
      @Assisted OsFamily osFamily, @Assisted boolean validate,
      MetainfoDAO metaInfoDAO, ActionMetadata actionMetadata, StackDAO stackDao,
      ExtensionDAO extensionDao, ExtensionLinkDAO linkDao)
      throws AmbariException {

    LOG.info("Initializing the stack manager...");

    if (validate) {
      validateStackDirectory(stackRoot);
      validateCommonServicesDirectory(commonServicesRoot);
      validateExtensionDirectory(extensionRoot);
    }

    stackMap = new HashMap<String, StackInfo>();
    stackContext = new StackContext(metaInfoDAO, actionMetadata, osFamily);
    extensionMap = new HashMap<String, ExtensionInfo>();

    Map<String, ServiceModule> commonServiceModules =     
     parseCommonServicesDirectory(commonServicesRoot);
    Map<String, StackModule> stackModules = parseStackDirectory(stackRoot);
    LOG.info("About to parse extension directories");
    Map<String, ExtensionModule> extensionModules = null;
    extensionModules = parseExtensionDirectory(extensionRoot);
}

其中比较重要的是parseStackDirectory(stackRoot)方法：

 private Map<String, StackModule> parseStackDirectory(File stackRoot) throws AmbariException {
    Map<String, StackModule> stackModules = new HashMap<String, StackModule>();

    File[] stackFiles = stackRoot.listFiles(AmbariMetaInfo.FILENAME_FILTER);
    for (File stack : stackFiles) {
      if (stack.isFile()) {
        continue;
      }
      for (File stackFolder : stack.listFiles(AmbariMetaInfo.FILENAME_FILTER)) {
        if (stackFolder.isFile()) {
          continue;
        }
        String stackName = stackFolder.getParentFile().getName();
        String stackVersion = stackFolder.getName();

        StackModule stackModule = new StackModule(new StackDirectory(stackFolder.getPath()), stackContext);
        String stackKey = stackName + StackManager.PATH_DELIMITER + stackVersion;
        stackModules.put(stackKey, stackModule);
        stackMap.put(stackKey, stackModule.getModuleInfo());
      }
    }

    if (stackMap.isEmpty()) {
      throw new AmbariException("Unable to find stack definitions under " +
          "stackRoot = " + stackRoot.getAbsolutePath());
    }
    return stackModules;
  }

该方法会将stack文件夹转换为一个个StackModule对象，StackModule类构造方法如下：

public StackModule(StackDirectory stackDirectory, StackContext stackContext) {
    this.stackDirectory = stackDirectory;
    this.stackContext = stackContext;
    this.stackInfo = new StackInfo();
    populateStackInfo();
}

在AmbariServer类歌属性初始化完成后，便会调用server.run()方法，对于其run()方法分析，我在后面的文章中再分析。