Ansible-python

1. Ansible优势

1. Ansible是python的一套完整的自动化执行任务模块
2. Ansible的play_book模式，采用yaml配置，对于自动化任务执行一目了然
3. 自动化场景支持丰富（shell,定时，用户管理或者第三方模块）

2. Ansible安装

1. 通过系统的方式yum, apt-get等
2. 通过python方式

3. Python ./setup.py install

   Easy_insatll ansible

   Pip install ansible

3. Ansible的配置文件

/etc/ansible/

        |____ansible.cfg  执行需求的全局性、默认的配置文件。

        |____hosts 默认的主机资产清单文件

        可以使用export ANSIBLE_CONFIG=路径方式修改默认配置文件

Ansible.cfg

        Ansible.cfg是ansible自动化任务所用的一个核心配置文件，大部分的配置文件都集中在defualts配置项目中。我的自动化实战课程讲解到ansble配置的时候，提供如下的问题，给大家作为参考阅读，defaults下的配置项，下面列出常用的配置参数：

1）inventory

        该参数表示资源清单inventory文件的位置，资源清单就是一些Ansible需要连接管理的主机列表

        inventory = /root/ansible/hosts

2）library

        Ansible的操作动作，无论是本地或远程，都使用一小段代码来执行，这小段代码称为模块，这个library参数就是指向存放Ansible模块的目录

        library = /usr/share/ansible

3）forks

        设置默认情况下Ansible最多能有多少个进程同时工作，默认设置最多5个进程并行处理。具体需要设置多少个，可以根据控制主机的性能和被管理节点的数量来确定。

        forks = 5

4）sudo_user

        这是设置默认执行命令的用户，也可以在playbook中重新设置这个参数

        sudo_user = root

        //注意:新版本已经作了修改，如ansible2.4.1下已经为：

        default_sudo_user = root

5）remote_port

        这是指定连接被关节点的管理端口，默认是22，除非设置了特殊的SSH端口，不然这个参数一般是不需要修改的

        remote_port = 22

6）host_key_checking

        这是设置是否检查SSH主机的密钥。可以设置为True或False

        host_key_checking = False

7）timeout

        这是设置SSH连接的超时间隔，单位是秒。

        timeout = 20

8）log_path

        Ansible系统默认是不记录日志的，如果想把Ansible系统的输出记录到人i治稳健中，需要设置log_path来指定一个存储Ansible日志的文件

        log_path = /var/log/ansible.log

        另外需要注意，执行Ansible的用户需要有写入日志的权限，模块将会调用被管节点的syslog来记录，口令是不会出现的日志中的

9）private_key_file

        在使用ssh公钥私钥登录系统时候，使用的密钥路径。

        private_key_file=/path/to/file.pem

        还有更多的配置项，大家可以参考官方文档，如下：

http://docs.ansible.com/ansible/latest/intro_configuration.html#private-key-file#private-key-file

Hosts

[group_name]

10.10.10.10:22 ansible_ssh_user=root ansible_ssh_pass=Y131231312

10.10.10.11:22 ansible_ssh_user=root ansible_ssh_pass=Hua1213

10.10.10.11:22 ansible_ssh_user=root ansible_ssh_private_key_file=/root/.ssh/id_rsa

[group_name]

test1 ansible_ssh_host = 192.168.1.13 ansible_ssh_port=22

ansible_ssh_user=root ansible_ssh_private_key_file=idrsapath

别名 + ssh用户 + ssh密钥

4. Ad-hoc模式和playbook

• 区别

        ad-hoc简而言之，就是临时命令，playbook是保存成一个配置文件下次还可以执行。

• Ad-hoc

1. Ad-hoc使用的场景（临时性）

        在多台机器上，查看某个进程是否启动。

        在多台机器上，拷贝指定日志文件到本地。

2. Ad-hoc模式命令的使用

        Ansible <host-patter组，正则表达式匹配，全部用于匹配主机名或者主机组名> [options]

        Ansible 192.168.1.* -a ‘ls /tmp’ 匹配192.168.1段

        Ansible group1 -a ‘ls /tmp’ 匹配192.168.1段

        Ansible test1 -a ‘ls /tmp’ 匹配192.168.1段 (别名的方式)

3. ansible命令的参数，如下：

        -v, --verbose：输出更详细的执行过程信息，-vvv可得到所有执行过程信息。

        -i PATH, --inventory=PATH：指定inventory信息，默认/etc/ansible/hosts。

        -f NUM, --forks=NUM：并发线程数，默认5个线程。

        --private-key=PRIVATE_KEY_FILE：指定密钥文件。

        -m NAME, --module-name=NAME：指定执行使用的模块。

        -M DIRECTORY, --module-path=DIRECTORY：指定模块存放路径，默认/usr/share/ansible，也可以通过ANSIBLE_LIBRARY设定默认路径。

        -a 'ARGUMENTS', --args='ARGUMENTS'：模块参数。

        -k, --ask-pass SSH：认证密码。

        -K, --ask-sudo-pass sudo：用户的密码（—sudo时使用）。

        -o, --one-line：标准输出至一行。

        -s, --sudo：相当于Linux系统下的sudo命令。

        -t DIRECTORY, --tree=DIRECTORY：输出信息至DIRECTORY目录下，结果文件以远程主机名命名。

        -T SECONDS, --timeout=SECONDS：指定连接远程主机的最大超时，单位是：秒。

        -B NUM, --background=NUM：后台执行命令，超NUM秒后kill正在执行的任务。

        -P NUM, --poll=NUM：定期返回后台任务进度。

        -u USERNAME, --user=USERNAME：指定远程主机以USERNAME运行命令。

        -U SUDO_USERNAME, --sudo-user=SUDO_USERNAM：E使用sudo，相当于Linux下的sudo命令。

        -c CONNECTION, --connection=CONNECTION：指定连接方式，可用选项paramiko (SSH), ssh, local。Local方式常用于crontab 和 kickstarts。

        -l SUBSET, --limit=SUBSET：指定运行主机。

        -l ~REGEX, --limit=~REGEX：指定运行主机（正则）。

        --list-hosts：列出符合条件的主机列表，不执行任何其他命令

• ansible-playbook模式常用命令的使用

        执行方式：ansible-playbook playbook.yml [options]

        -u REMOTE_USER, --user=REMOTE_USER  ＃ ssh 连接的用户名

         -k, --ask-pass    ＃ssh登录认证密码

         -s, --sudo           ＃sudo 到root用户，相当于Linux系统下的sudo命令

         -U SUDO_USER, --sudo-user=SUDO_USER    ＃sudo 到对应的用户

         -K, --ask-sudo-pass     ＃用户的密码（—sudo时使用）

         -T TIMEOUT, --timeout=TIMEOUT ＃ ssh 连接超时，默认 10 秒

         -C, --check      ＃ 指定该参数后，执行 playbook 文件不会真正去执行，而是模拟执行一遍，然后输出本次执行会对远程主机造成的修改

         -e EXTRA_VARS, --extra-vars=EXTRA_VARS    ＃ 设置额外的变量如：key=value 形式 或者 YAML or JSON，以空格分隔变量，或用多个-e

         -f FORKS, --forks=FORKS    ＃ 进程并发处理，默认 5

         -i INVENTORY, --inventory-file=INVENTORY   ＃ 指定 hosts 文件路径，默认 default=/etc/ansible/hosts

         -l SUBSET, --limit=SUBSET    ＃ 指定一个 pattern，对- hosts:匹配到的主机再过滤一次

         --list-hosts  ＃ 只打印有哪些主机会执行这个 playbook 文件，不是实际执行该 playbook

         --list-tasks   ＃ 列出该 playbook 中会被执行的 task

         --private-key=PRIVATE_KEY_FILE   ＃ 私钥路径

         --step    ＃ 同一时间只执行一个 task，每个 task 执行前都会提示确认一遍

         --syntax-check  ＃ 只检测 playbook 文件语法是否有问题，不会执行该 playbook

         -t TAGS, --tags=TAGS   ＃当 play 和 task 的 tag 为该参数指定的值时才执行，多个 tag 以逗号分隔

         --skip-tags=SKIP_TAGS   ＃ 当 play 和 task 的 tag 不匹配该参数指定的值时，才执行

         -v, --verbose   ＃输出更详细的执行过程信息，-vvv可得到所有执行过程信息。

5. Python-ansible的使用

• 资产配置清单

from ansible.parsing.dataloader import DataLoader
from ansible.inventory.manager import InventoryManager
from ansible.vars.manager import VariableManager


# InventoryManager类
loader = DataLoader()
inventory = InventoryManager(loader=loader, sources=['hosts'])
# 获取主机组资源
print(inventory.get_groups_dict())
# 获取指定的主机对象
print(inventory.get_hosts())
print(inventory.get_host(hostname='10.10.10.10'))

#　添加主机到指定主机组
inventory.add_host(host='127.0.0.1', port=22, group='test_group1')
print(inventory.get_groups_dict())

# VariableManager类 变量管理
variable_manager = VariableManager(loader=loader, inventory=inventory)
# 查看变量方法 get_vars
variable_manager.get_vars(host=inventory.get_host(hostname='10.10.10.10'))
# 设置主机变量方法 set_host_variable
variable_manager.set_host_variable(host=inventory.get_host(hostname='10.10.10.10'),
                                   varname='ansible_ssh_user',
                                   value='root')

# 添加扩展变量 extra_vars
variable_manager.extra_vars={'myweb': 'imocc.com', 'myname':'ribbon'}

• Ad-hoc

from ansible.playbook.play import Play
from ansible.executor.task_queue_manager import TaskQueueManager
from ansible.plugins.callback import CallbackBase
from ansible.parsing.dataloader import DataLoader
from ansible.inventory.manager import InventoryManager
from ansible.vars.manager import VariableManager
from collections import namedtuple


loader = DataLoader()
inventory = InventoryManager(loader=loader, sources=['hosts'])
variable_manager = VariableManager(loader=loader, inventory=inventory)

# 执行选项
Options = namedtuple('Options', ['connection', 'remote_user', 'forks', 'become',
                                 'become_method', 'become_user', 'check', 'module_path',
                                 'diff'])
options = Options(connection='smart', remote_user='root', forks=100, become=None,
                  become_method=None, become_user=None, check=False, module_path=None,
                  diff=None)


# Play执行对象和模块
play_source = dict(
    name='Ansible PLAY ad-hoc test',
    hosts='10.10.10.10',
    gather_facts='no',
    tasks = [
        dict(action=dict(module='shell', args='touch adhoc_test.txt'))
    ]
)
play = Play().load(play_source, variable_manager=variable_manager, loader=loader)


# 任务队列
passwords = dict()
tqm = TaskQueueManager(
    inventory=inventory,
    variable_manager=variable_manager,
    loader=loader,
    options=options,
    passwords=passwords
)

result = tqm.run(play)

• Playbook

.py

# coding=utf-8
import sys


reload(sys)
sys.setdefaultencoding("utf-8")
from ansible.plugins.callback import CallbackBase
from ansible.parsing.dataloader import DataLoader
from ansible.inventory.manager import InventoryManager
from ansible.vars.manager import VariableManager
from collections import namedtuple
from ansible.executor.playbook_executor import PlaybookExecutor


loader = DataLoader()
inventory = InventoryManager(loader=loader, sources=['hosts'])
variable_manager = VariableManager(loader=loader, inventory=inventory)

# 执行选项
Options = namedtuple('Options', ['connection', 'remote_user',
                                 'ack_pass', 'sudo_user',
                                 'forks', 'sudo',
                                 'ask_sudo_pass', 'verbosity',
                                 'module_path', 'become',
                                 'become_method', 'become_user',
                                 'check', 'diff', 'listhosts',
                                 'listtasks', 'listtags', 'syntax'])

options = Options(connection='smart', remote_user='root', forks=100, ack_pass=None,
                  sudo_user=None, sudo=None, ask_sudo_pass=False, verbosity=5,
                  module_path=None,become=None,become_method=None,   become_user=None,
                  check=False, diff=False, listhosts=None, listtasks=None, listtags=None,
                  syntax=None)


# 任务队列
passwords = dict()
playbook = PlaybookExecutor(
    playbooks=['f1.yaml'],
    inventory=inventory,
    variable_manager=variable_manager,
    loader=loader,
    options=options,
    passwords=passwords
)

playbook.run()

f1.yaml

---
- hosts : 10.10.10.10
  remote_user : root
  vars :
      touch_file : imooc.file
  tasks :
      - name: touch file
        shell: "touch /tmp/{{touch_file}}"

hosts

[test_group1]
10.10.10.11:22 ansible_ssh_user=root ansible_ssh_private_key_file=/root/.ssh/id_rsa

[test_group2]
10.10.10.10:22 ansible_ssh_user=root ansible_ssh_pass='Yanwentao123'

• Callback类重写

Adhoc callback改写.py

# coding=utf-8
from ansible.playbook.play import Play
from ansible.executor.task_queue_manager import TaskQueueManager
from ansible.plugins.callback import CallbackBase
from ansible.parsing.dataloader import DataLoader
from ansible.inventory.manager import InventoryManager
from ansible.vars.manager import VariableManager
from collections import namedtuple
import datetime
import json


class ModelResultsCollector(CallbackBase):
    def __init__(self, *args, **kwargs):
        super(ModelResultsCollector, self).__init__(*args, **kwargs)
        self.host_ok = {}
        self.host_unreachable = {}
        self.host_failed = {}

    def v2_runner_on_ok(self, result, **kwargs):  # 定义成功
        """Print a json representation of the result
        This method could store the result in an instance attribute for retrieval later
        """
        self.host_ok[result._host.get_name()] = result
        host = result._host
        currt_tiem = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
        logs = json.dumps({currt_tiem: {host.name: result._result['msg']}}, indent=4)
        # with open('/opt/aaaa.json', 'a') as f:
        #     f.write(logs + '\n')
        self.result_log = result._result['msg']

    def v2_runner_on_unreachable(self, result, **kwargs):
        """Print a json representation of the result
        This method could store the result in an instance attribute for retrieval later
        """
        self.host_ok[result._host.get_name()] = result
        host = result._host
        currt_tiem = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
        self.result_log = result._result['msg']
        logs = json.dumps({currt_tiem: {host.name: self.result_log}}, indent=4)
        # with open('/opt/aaaa.json', 'a') as f:
        #     f.write(logs + '\n')

    def v2_runner_on_failed(self, result, **kwargs):
        """Print a json representation of the result
        This method could store the result in an instance attribute for retrieval later
        """
        self.host_failed[result._host.get_name()] = result
        host = result._host
        currt_tiem = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
        self.result_log = result._result['msg']
        logs = json.dumps({currt_tiem: {host.name: self.result_log}}, indent=4)
        # with open('/opt/aaaa.json', 'a') as f:
        #     f.write(logs + '\n')

loader = DataLoader()
inventory = InventoryManager(loader=loader, sources=['hosts'])
variable_manager = VariableManager(loader=loader, inventory=inventory)

# 执行选项
Options = namedtuple('Options', ['connection', 'remote_user',
                                 'ack_pass', 'sudo_user',
                                 'forks', 'sudo',
                                 'ask_sudo_pass', 'verbosity',
                                 'module_path', 'become',
                                 'become_method', 'become_user',
                                 'check', 'diff', 'listhosts',
                                 'listtasks', 'listtages', 'systax'])
options = Options(connection='smart', remote_user='root', forks=100, ack_pass=None,
                  sudo_user=None, sudo=None, ask_sudo_pass=False, verbosity=5,
                  module_path=None,become=None, become_method=None, become_user=None,
                  check=False, diff=False, listhosts=None, listtasks=None, 
                  listtages=None, systax=None)


# Play执行对象和模块
play_source = dict(
    name='Ansible PLAY ad-hoc test',
    hosts='10.10.10.10',
    gather_facts='no',
    tasks = [
        dict(action=dict(module='shell', args='touch adhoc_test.txt'))
    ]
)

play = Play().load(play_source, variable_manager=variable_manager, loader=loader)

# callback
callback = ModelResultsCollector()

# 任务队列
passwords = dict()
tqm = TaskQueueManager(
    inventory=inventory,
    variable_manager=variable_manager,
    loader=loader,
    options=options,
    passwords=passwords,
    stdout_callback=callback
)
result = tqm.run(play)

# 输出callback状态
print callback.host_ok.items()
result_raw = {'success':{},
              'failed':{},
              'unreach':{}}
for host, result in callback.host_failed.items():
    result_raw['failed'][host] = result._result
for host, result in callback.host_ok.items():
    result_raw['success'][host] = result._result
for host, result in callback.host_unreachable.items():
    result_raw['unreach'][host] = result._result
print(result_raw)

• Playbook callback改写.py

# coding=utf-8
import sys
reload(sys)
sys.setdefaultencoding("utf-8")
import json
import datetime
from ansible.plugins.callback import CallbackBase
from ansible.parsing.dataloader import DataLoader
from ansible.inventory.manager import InventoryManager
from ansible.vars.manager import VariableManager
from collections import namedtuple
from ansible.executor.playbook_executor import PlaybookExecutor


class PlayBookResultsCollector(CallbackBase):
    CALLBACK_VERSION = 2.0
    def __init__(self, *args, **kwargs):
        super(PlayBookResultsCollector, self).__init__(*args, **kwargs)
        self.task_ok = {}
        self.task_skipped = {}
        self.task_failed = {}
        self.task_status = {}
        self.task_unreachable = {}

    def v2_runner_on_ok(self, result, **kwargs):  # 定义成功
        self.task_ok[result._host.get_name()] = result

    def v2_runner_on_unreachable(self, result, **kwargs):
        self.task_unreachable[result._host.get_name()] = result

    def v2_runner_on_failed(self, result, **kwargs):
        self.task_failed[result._host.get_name()] = result

    def v2_runner_on_skipped(self, result):
        self.task_skipped[result._host.get_name()] = result

    def v2_playbook_on_stats(self, stats):
        hosts = sorted(stats.processed.keys())
        for h in hosts:
            t = stats.summarize(h)
            self.task_status[h] = {
                'ok':t['ok'],
                'changed': t['changed'],
                'unreachable':t['unreachable'],
                'skipped':t['skipped'],
                'failed':t['failures']
            }

loader = DataLoader()
inventory = InventoryManager(loader=loader, sources=['hosts'])
variable_manager = VariableManager(loader=loader, inventory=inventory)

#Options 执行选项
Options = namedtuple('Options',
                     ['connection',
                      'remote_user',
                      'ask_sudo_pass',
                      'verbosity',
                      'ack_pass',
                      'module_path',
                      'forks',
                      'become',
                      'become_method',
                      'become_user',
                      'check',
                      'listhosts',
                      'listtasks',
                      'listtags',
                      'syntax',
                      'sudo_user',
                      'sudo',
                      'diff'])

options = Options(connection='smart',
                  remote_user=None,
                  ack_pass=None,
                  sudo_user=None,
                  forks=5,
                  sudo=None,
                  ask_sudo_pass=False,
                  verbosity=5,
                  module_path=None,
                  become=None,
                  become_method=None,
                  become_user=None,
                  check=False,
                  diff=False,
                  listhosts=None,
                  listtasks=None,
                  listtags=None,
                  syntax=None)

callback = PlayBookResultsCollector()
# 任务队列
passwords = dict()
playbook = PlaybookExecutor(
    playbooks=['f1.yaml'],
    inventory=inventory,
    variable_manager=variable_manager,
    loader=loader,
    options=options,
    passwords=passwords
)

playbook._tqm._stdout_callback = callback
playbook.run()

result_raw = {'ok':{},
              'failed':{},
              'unreach':{},
              'status':{},
              'skipped':{}}

for host, result in callback.task_ok.items():
    result_raw['ok'][host] = result._result

print(result_raw)

• Django_Ansible