(本文基于Ansible 2.7)
写本文的想法是在写Ansible 源码解析:forks并发机制的实现一文时萌生的。Ansible的配置管理现状是比较复杂的,支持ini格式的配置文件,支持yaml格式的配置文件,也支持环境变量配置或在运行时加入命令选项配置,在调用Ansible Python API时传入配置项等。本文尝试理清一个配置项起作用的方式,如觉得代码冗长,可直接到文末看结论。
本文将仍以forks的配置为例。通过上面提到的这篇文章,我们已经知道在 lib/ansible/cli/__init__.py中有接收和处理ansible命令选项的代码:
if fork_opts:
parser.add_option('-f', '--forks', dest='forks', default=C.DEFAULT_FORKS, type='int',
help="specify number of parallel processes to use (default=%s)" % C.DEFAULT_FORKS)
其中C是lib/ansible/constants.py,在:lib/ansible/cli/__init__.py 的第36行有声明。
from ansible import constants as C
但constants中并没有显式声明DEFAULT_FORKS常量。我们通过阅读文档,或查看ansible的帮助,可以知道默认的并发进程数是5,这个默认值可以通过编辑lib/ansible/config/base.yml 中配置,也可以通过修改环境变量或ini配置文件类配置:
lib/ansible/config/base.yml ,566-573行:
DEFAULT_FORKS:
name: Number of task forks
default: 5
description: Maximum number of forks Ansible will use to execute tasks on target hosts.
env: [{name: ANSIBLE_FORKS}]
ini:
- {key: forks, section: defaults}
type: integer
那么默认值5是如何读取到constant中的呢?
我们知道ConfigManager类用来管理Ansible的配置信息,在constants.py的末尾,创建了ConfigManager对象,读取base.yml并export到constant中:
lib/ansible/constants.py , 183-201行:
# POPULATE SETTINGS FROM CONFIG ###
config = ConfigManager()
# Generate constants from config
for setting in config.data.get_settings():
value = setting.value
if setting.origin == 'default' and \
isinstance(setting.value, string_types) and \
(setting.value.startswith('{{') and setting.value.endswith('}}')):
try:
t = Template(setting.value)
value = t.render(vars())
try:
value = literal_eval(value)
except ValueError:
pass # not a python data structure
except Exception:
pass # not templatable
value = ensure_type(value, setting.type)
set_constant(setting.name, value)
set_constant在66-68行:
def set_constant(name, value, export=vars()):
''' sets constants and returns resolved options dict '''
export[name] = value
而ConfigManager.data是ConfigData对象(lib/ansible/config/data.py):
# Copyright: (c) 2017, Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
class ConfigData(object):
def __init__(self):
self._global_settings = {}
self._plugins = {}
def get_setting(self, name, plugin=None):
setting = None
if plugin is None:
setting = self._global_settings.get(name)
elif plugin.type in self._plugins and plugin.name in self._plugins[plugin.type]:
setting = self._plugins[plugin.type][plugin.name].get(name)
return setting
def get_settings(self, plugin=None):
settings = []
if plugin is None:
settings = [self._global_settings[k] for k in self._global_settings]
elif plugin.type in self._plugins and plugin.name in self._plugins[plugin.type]:
settings = [self._plugins[plugin.type][plugin.name][k] for k in self._plugins[plugin.type][plugin.name]]
return settings
def update_setting(self, setting, plugin=None):
if plugin is None:
self._global_settings[setting.name] = setting
else:
if plugin.type not in self._plugins:
self._plugins[plugin.type] = {}
if plugin.name not in self._plugins[plugin.type]:
self._plugins[plugin.type][plugin.name] = {}
self._plugins[plugin.type][plugin.name][setting.name] = setting
此类中并没有对 _global_settings 赋值,必然是从外部通过调用update_setting方法填充的config数据。
lib/ansible/config/manager.py,464-502行:
def update_config_data(self, defs=None, configfile=None):
''' really: update constants '''
if defs is None:
defs = self._base_defs
if configfile is None:
configfile = self._config_file
if not isinstance(defs, dict):
raise AnsibleOptionsError("Invalid configuration definition type: %s for %s" % (type(defs), defs))
# update the constant for config file
self.data.update_setting(Setting('CONFIG_FILE', configfile, '', 'string'))
origin = None
# env and config defs can have several entries, ordered in list from lowest to highest precedence
for config in defs:
if not isinstance(defs[config], dict):
raise AnsibleOptionsError("Invalid configuration definition '%s': type is %s" % (to_native(config), type(defs[config])))
# get value and origin
try:
value, origin = self.get_config_value_and_origin(config, configfile)
except Exception as e:
# Printing the problem here because, in the current code:
# (1) we can't reach the error handler for AnsibleError before we
# hit a different error due to lack of working config.
# (2) We don't have access to display yet because display depends on config
# being properly loaded.
#
# If we start getting double errors printed from this section of code, then the
# above problem #1 has been fixed. Revamp this to be more like the try: except
# in get_config_value() at that time.
sys.stderr.write("Unhandled error:\n %s\n\n" % traceback.format_exc())
raise AnsibleError("Invalid settings supplied for %s: %s\n%s" % (config, to_native(e), traceback.format_exc()))
# set the constant
self.data.update_setting(Setting(config, value, origin, defs[config].get('type', 'string')))
update_config_data仅在ConfigManager构造函数中有一次调用,且未传递配置文件名参数,而默认使用的配置项集合是self._base_defs,它是在update_config_data被调用之前赋值的(230行):
self._base_defs = self._read_config_yaml_file(defs_file or ('%s/base.yml' % os.path.dirname(__file__)))
update_config_data里最核心的一部分是get_config_value_and_origin方法的调用,它负责实施注释中提到的,对来自多个来源的配置项的值进行优先级排序。
这个方法在lib/ansible/config/manager.py的357-447行:
def get_config_value_and_origin(self, config, cfile=None, plugin_type=None, plugin_name=None, keys=None, variables=None, direct=None):
''' Given a config key figure out the actual value and report on the origin of the settings '''
if cfile is None:
# use default config
cfile = self._config_file
# Note: sources that are lists listed in low to high precedence (last one wins)
value = None
origin = None
defs = {}
if plugin_type is None:
defs = self._base_defs
elif plugin_name is None:
defs = self._plugins[plugin_type]
else:
defs = self._plugins[plugin_type][plugin_name]
if config in defs:
# direct setting via plugin arguments, can set to None so we bypass rest of processing/defaults
if direct and config in direct:
value = direct[config]
origin = 'Direct'
else:
# Use 'variable overrides' if present, highest precedence, but only present when querying running play
if variables and defs[config].get('vars'):
value, origin = self._loop_entries(variables, defs[config]['vars'])
origin = 'var: %s' % origin
# use playbook keywords if you have em
if value is None and keys and defs[config].get('keywords'):
value, origin = self._loop_entries(keys, defs[config]['keywords'])
origin = 'keyword: %s' % origin
# env vars are next precedence
if value is None and defs[config].get('env'):
value, origin = self._loop_entries(py3compat.environ, defs[config]['env'])
origin = 'env: %s' % origin
# try config file entries next, if we have one
if self._parsers.get(cfile, None) is None:
self._parse_config_file(cfile)
if value is None and cfile is not None:
ftype = get_config_type(cfile)
if ftype and defs[config].get(ftype):
if ftype == 'ini':
# load from ini config
try: # FIXME: generalize _loop_entries to allow for files also, most of this code is dupe
for ini_entry in defs[config]['ini']:
temp_value = get_ini_config_value(self._parsers[cfile], ini_entry)
if temp_value is not None:
value = temp_value
origin = cfile
if 'deprecated' in ini_entry:
self.DEPRECATED.append(('[%s]%s' % (ini_entry['section'], ini_entry['key']), ini_entry['deprecated']))
except Exception as e:
sys.stderr.write("Error while loading ini config %s: %s" % (cfile, to_native(e)))
elif ftype == 'yaml':
# FIXME: implement, also , break down key from defs (. notation???)
origin = cfile
# set default if we got here w/o a value
if value is None:
if defs[config].get('required', False):
entry = ''
if plugin_type:
entry += 'plugin_type: %s ' % plugin_type
if plugin_name:
entry += 'plugin: %s ' % plugin_name
entry += 'setting: %s ' % config
if not plugin_type or config not in INTERNAL_DEFS.get(plugin_type, {}):
raise AnsibleError("No setting was provided for required configuration %s" % (entry))
else:
value = defs[config].get('default')
origin = 'default'
# skip typing as this is a temlated default that will be resolved later in constants, which has needed vars
if plugin_type is None and isinstance(value, string_types) and (value.startswith('{{') and value.endswith('}}')):
return value, origin
# ensure correct type, can raise exceptoins on mismatched types
value = ensure_type(value, defs[config].get('type'), origin=origin)
# deal with deprecation of the setting
if 'deprecated' in defs[config] and origin != 'default':
self.DEPRECATED.append((config, defs[config].get('deprecated')))
else:
raise AnsibleError('Requested option %s was not defined in configuration' % to_native(config))
return value, origin
优先级的规则看注释即可。
至此为止,一个配置项从配置文件中读取到加载的过程大致可以归纳为:
- 配置文件
- ConfigManager读取配置文件
- ConfigManager.get_config_value_and_origin将读取到的配置文件内容根据固定的规则排好优先级(会决定最终取值)
- ConfigManager.data.update_setting将配置更新到ConfigData对象
- constants通过ConfigManager对象找到ConfigData对象,并将其中已加载的配置export到constants中
- CLI判断是否有输入选项,如无,使用(从多个配置来源,经过优先级比较取得并加入constant中的)默认配置。