Ansible源码解析：配置的读取和使用

（本文基于Ansible 2.7）
写本文的想法是在写Ansible 源码解析：forks并发机制的实现一文时萌生的。Ansible的配置管理现状是比较复杂的，支持ini格式的配置文件，支持yaml格式的配置文件，也支持环境变量配置或在运行时加入命令选项配置，在调用Ansible Python API时传入配置项等。本文尝试理清一个配置项起作用的方式，如觉得代码冗长，可直接到文末看结论。

本文将仍以forks的配置为例。通过上面提到的这篇文章，我们已经知道在 lib/ansible/cli/__init__.py中有接收和处理ansible命令选项的代码：

        if fork_opts:
            parser.add_option('-f', '--forks', dest='forks', default=C.DEFAULT_FORKS, type='int',
                              help="specify number of parallel processes to use (default=%s)" % C.DEFAULT_FORKS)

其中C是lib/ansible/constants.py,在:lib/ansible/cli/__init__.py 的第36行有声明。

from ansible import constants as C

但constants中并没有显式声明DEFAULT_FORKS常量。我们通过阅读文档，或查看ansible的帮助，可以知道默认的并发进程数是5，这个默认值可以通过编辑lib/ansible/config/base.yml 中配置，也可以通过修改环境变量或ini配置文件类配置：

lib/ansible/config/base.yml ，566-573行：

DEFAULT_FORKS:
  name: Number of task forks
  default: 5
  description: Maximum number of forks Ansible will use to execute tasks on target hosts.
  env: [{name: ANSIBLE_FORKS}]
  ini:
  - {key: forks, section: defaults}
  type: integer

那么默认值5是如何读取到constant中的呢？
我们知道ConfigManager类用来管理Ansible的配置信息，在constants.py的末尾，创建了ConfigManager对象，读取base.yml并export到constant中：

lib/ansible/constants.py , 183-201行：

# POPULATE SETTINGS FROM CONFIG ###
config = ConfigManager()

# Generate constants from config
for setting in config.data.get_settings():

    value = setting.value
    if setting.origin == 'default' and \
       isinstance(setting.value, string_types) and \
       (setting.value.startswith('{{') and setting.value.endswith('}}')):
        try:
            t = Template(setting.value)
            value = t.render(vars())
            try:
                value = literal_eval(value)
            except ValueError:
                pass  # not a python data structure
        except Exception:
            pass  # not templatable

        value = ensure_type(value, setting.type)

    set_constant(setting.name, value)

set_constant在66-68行：

def set_constant(name, value, export=vars()):
    ''' sets constants and returns resolved options dict '''
    export[name] = value

而ConfigManager.data是ConfigData对象（lib/ansible/config/data.py）：

# Copyright: (c) 2017, Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)

from __future__ import (absolute_import, division, print_function)
__metaclass__ = type


class ConfigData(object):

    def __init__(self):
        self._global_settings = {}
        self._plugins = {}

    def get_setting(self, name, plugin=None):

        setting = None
        if plugin is None:
            setting = self._global_settings.get(name)
        elif plugin.type in self._plugins and plugin.name in self._plugins[plugin.type]:
            setting = self._plugins[plugin.type][plugin.name].get(name)

        return setting

    def get_settings(self, plugin=None):

        settings = []
        if plugin is None:
            settings = [self._global_settings[k] for k in self._global_settings]
        elif plugin.type in self._plugins and plugin.name in self._plugins[plugin.type]:
            settings = [self._plugins[plugin.type][plugin.name][k] for k in self._plugins[plugin.type][plugin.name]]

        return settings

    def update_setting(self, setting, plugin=None):

        if plugin is None:
            self._global_settings[setting.name] = setting
        else:
            if plugin.type not in self._plugins:
                self._plugins[plugin.type] = {}
            if plugin.name not in self._plugins[plugin.type]:
                self._plugins[plugin.type][plugin.name] = {}
            self._plugins[plugin.type][plugin.name][setting.name] = setting

此类中并没有对 _global_settings 赋值，必然是从外部通过调用update_setting方法填充的config数据。
lib/ansible/config/manager.py，464-502行：

def update_config_data(self, defs=None, configfile=None):
        ''' really: update constants '''

        if defs is None:
            defs = self._base_defs

        if configfile is None:
            configfile = self._config_file

        if not isinstance(defs, dict):
            raise AnsibleOptionsError("Invalid configuration definition type: %s for %s" % (type(defs), defs))

        # update the constant for config file
        self.data.update_setting(Setting('CONFIG_FILE', configfile, '', 'string'))

        origin = None
        # env and config defs can have several entries, ordered in list from lowest to highest precedence
        for config in defs:
            if not isinstance(defs[config], dict):
                raise AnsibleOptionsError("Invalid configuration definition '%s': type is %s" % (to_native(config), type(defs[config])))

            # get value and origin
            try:
                value, origin = self.get_config_value_and_origin(config, configfile)
            except Exception as e:
                # Printing the problem here because, in the current code:
                # (1) we can't reach the error handler for AnsibleError before we
                #     hit a different error due to lack of working config.
                # (2) We don't have access to display yet because display depends on config
                #     being properly loaded.
                #
                # If we start getting double errors printed from this section of code, then the
                # above problem #1 has been fixed.  Revamp this to be more like the try: except
                # in get_config_value() at that time.
                sys.stderr.write("Unhandled error:\n %s\n\n" % traceback.format_exc())
                raise AnsibleError("Invalid settings supplied for %s: %s\n%s" % (config, to_native(e), traceback.format_exc()))

            # set the constant
            self.data.update_setting(Setting(config, value, origin, defs[config].get('type', 'string')))

update_config_data仅在ConfigManager构造函数中有一次调用，且未传递配置文件名参数，而默认使用的配置项集合是self._base_defs，它是在update_config_data被调用之前赋值的（230行）：

        self._base_defs = self._read_config_yaml_file(defs_file or ('%s/base.yml' % os.path.dirname(__file__)))

update_config_data里最核心的一部分是get_config_value_and_origin方法的调用，它负责实施注释中提到的，对来自多个来源的配置项的值进行优先级排序。
这个方法在lib/ansible/config/manager.py的357-447行：

    def get_config_value_and_origin(self, config, cfile=None, plugin_type=None, plugin_name=None, keys=None, variables=None, direct=None):
        ''' Given a config key figure out the actual value and report on the origin of the settings '''
        if cfile is None:
            # use default config
            cfile = self._config_file

        # Note: sources that are lists listed in low to high precedence (last one wins)
        value = None
        origin = None
        defs = {}
        if plugin_type is None:
            defs = self._base_defs
        elif plugin_name is None:
            defs = self._plugins[plugin_type]
        else:
            defs = self._plugins[plugin_type][plugin_name]

        if config in defs:

            # direct setting via plugin arguments, can set to None so we bypass rest of processing/defaults
            if direct and config in direct:
                value = direct[config]
                origin = 'Direct'

            else:
                # Use 'variable overrides' if present, highest precedence, but only present when querying running play
                if variables and defs[config].get('vars'):
                    value, origin = self._loop_entries(variables, defs[config]['vars'])
                    origin = 'var: %s' % origin

                # use playbook keywords if you have em
                if value is None and keys and defs[config].get('keywords'):
                    value, origin = self._loop_entries(keys, defs[config]['keywords'])
                    origin = 'keyword: %s' % origin

                # env vars are next precedence
                if value is None and defs[config].get('env'):
                    value, origin = self._loop_entries(py3compat.environ, defs[config]['env'])
                    origin = 'env: %s' % origin

                # try config file entries next, if we have one
                if self._parsers.get(cfile, None) is None:
                    self._parse_config_file(cfile)

                if value is None and cfile is not None:
                    ftype = get_config_type(cfile)
                    if ftype and defs[config].get(ftype):
                        if ftype == 'ini':
                            # load from ini config
                            try:  # FIXME: generalize _loop_entries to allow for files also, most of this code is dupe
                                for ini_entry in defs[config]['ini']:
                                    temp_value = get_ini_config_value(self._parsers[cfile], ini_entry)
                                    if temp_value is not None:
                                        value = temp_value
                                        origin = cfile
                                        if 'deprecated' in ini_entry:
                                            self.DEPRECATED.append(('[%s]%s' % (ini_entry['section'], ini_entry['key']), ini_entry['deprecated']))
                            except Exception as e:
                                sys.stderr.write("Error while loading ini config %s: %s" % (cfile, to_native(e)))
                        elif ftype == 'yaml':
                            # FIXME: implement, also , break down key from defs (. notation???)
                            origin = cfile

                # set default if we got here w/o a value
                if value is None:
                    if defs[config].get('required', False):
                        entry = ''
                        if plugin_type:
                            entry += 'plugin_type: %s ' % plugin_type
                            if plugin_name:
                                entry += 'plugin: %s ' % plugin_name
                        entry += 'setting: %s ' % config
                        if not plugin_type or config not in INTERNAL_DEFS.get(plugin_type, {}):
                            raise AnsibleError("No setting was provided for required configuration %s" % (entry))
                    else:
                        value = defs[config].get('default')
                        origin = 'default'
                        # skip typing as this is a temlated default that will be resolved later in constants, which has needed vars
                        if plugin_type is None and isinstance(value, string_types) and (value.startswith('{{') and value.endswith('}}')):
                            return value, origin

            # ensure correct type, can raise exceptoins on mismatched types
            value = ensure_type(value, defs[config].get('type'), origin=origin)

            # deal with deprecation of the setting
            if 'deprecated' in defs[config] and origin != 'default':
                self.DEPRECATED.append((config, defs[config].get('deprecated')))
        else:
            raise AnsibleError('Requested option %s was not defined in configuration' % to_native(config))

        return value, origin

优先级的规则看注释即可。

至此为止，一个配置项从配置文件中读取到加载的过程大致可以归纳为：

1. 配置文件
2. ConfigManager读取配置文件
3. ConfigManager.get_config_value_and_origin将读取到的配置文件内容根据固定的规则排好优先级（会决定最终取值）
4. ConfigManager.data.update_setting将配置更新到ConfigData对象
5. constants通过ConfigManager对象找到ConfigData对象，并将其中已加载的配置export到constants中
6. CLI判断是否有输入选项，如无，使用（从多个配置来源，经过优先级比较取得并加入constant中的）默认配置。