return，job cache和salt-ssh运用

最新推荐文章于 2021-11-02 19:32:08 发布

qixian123456

最新推荐文章于 2021-11-02 19:32:08 发布

阅读量204

点赞数

文章标签： linux

本文链接：https://blog.csdn.net/yeqixian123456/article/details/108230189

版权

文章目录

1. return和job cache
2. salt-ssh

1. return和job cache

在这里插入图片描述

1.1 SaltStack组件之return

return组件可以理解为SaltStack系统对执行Minion返回后的数据进行存储或者返回给其他程序，它支持多种存储方式，比如用MySQL、MongoDB、Redis、Memcache等，通过return我们可以对SaltStack的每次操作进行记录，对以后日志审计提供了数据来源。目前官方已经支持30种return数据存储与接口，我们可以很方便的配置与使用它。当然也支持自己定义的return，自定义的return需由python来编写。在选择和配置好要使用的return后，只需在salt命令后面指定return即可。

[root@xian ~]# salt '*' sys.list_returners
yeqixian:
    - carbon
    - couchdb
    - elasticsearch
    - etcd
    - highstate
    - local
    - local_cache
    - mattermost
    - multi_returner
    - pushover
    - rawfile_json
    - slack
    - slack_webhook
    - smtp
    - splunk
    - sqlite3
    - syslog
    - telegram

在这里插入图片描述

还有很多没有列出来的，可以在官网查看

1.1.1 return流程

return是在Master端触发任务，然后Minion接受处理任务后直接与return存储服务器建立连接，然后把数据return存到存储服务器。关于这点一定要注意，因为此过程都是Minion端操作存储服务器，所以要确保Minion端的配置跟依赖包是正确的，这意味着我们将必须在每个Minion上安装指定的return方式依赖包，假如使用Mysql作为return存储方式，那么我们将在每台Minion上安装python-mysql模块。

1.1.2 使用mysql作为return存储方式

在所有minion上安装Mysql-python模块

[root@xian ~]# salt '*' pkg.install MySQL-python
yeqixian:
    ----------
    MySQL-python:
        ----------
        new:
            1.2.5-1.el7
        old:
[root@xian ~]# salt '*' cmd.run 'rpm -qa|grep MySQL-python'
yeqixian:
    MySQL-python-1.2.5-1.el7.x86_64

部署一台mysql服务器用作存储服务器，此处就直接在192.168.116.180这台主机上部署

//部署mysql

[root@xian ~]# yum -y install mariadb-server
[root@xian ~]# systemctl enable --now mariadb

//创建数据库和表结构

在这里插入图片描述

[root@xian ~]# mysql -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 5.7.30 MySQL Community Server (GPL)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.


mysql> CREATE DATABASE  `salt`
    ->   DEFAULT CHARACTER SET utf8
    ->   DEFAULT COLLATE utf8_general_ci;
Query OK, 1 row affected (0.01 sec)

mysql> 
mysql> USE `salt`;
Database changed
mysql> DROP TABLE IF EXISTS `jids`;
Query OK, 0 rows affected, 1 warning (0.01 sec)

mysql> CREATE TABLE `jids` (
    ->   `jid` varchar(255) NOT NULL,
    ->   `load` mediumtext NOT NULL,
    ->   UNIQUE KEY `jid` (`jid`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.03 sec)

mysql> DROP TABLE IF EXISTS `salt_returns`;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> CREATE TABLE `salt_returns` (
    ->   `fun` varchar(50) NOT NULL,
    ->   `jid` varchar(255) NOT NULL,
    ->   `return` mediumtext NOT NULL,
    ->   `id` varchar(255) NOT NULL,
    ->   `success` varchar(10) NOT NULL,
    ->   `full_ret` mediumtext NOT NULL,
    ->   `alter_time` TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    ->   KEY `id` (`id`),
    ->   KEY `jid` (`jid`),
    ->   KEY `fun` (`fun`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.01 sec)
mysql> DROP TABLE IF EXISTS `salt_events`;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> CREATE TABLE `salt_events` (
    -> `id` BIGINT NOT NULL AUTO_INCREMENT,
    -> `tag` varchar(255) NOT NULL,
    -> `data` mediumtext NOT NULL,
    -> `alter_time` TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    -> `master_id` varchar(255) NOT NULL,
    -> PRIMARY KEY (`id`),
    -> KEY `tag` (`tag`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.00 sec)
mysql> show tables;
+----------------+
| Tables_in_salt |
+----------------+
| jids           |
| salt_events    |
| salt_returns   |
+----------------+
3 rows in set (0.01 sec)
mysql> grant all on salt.* to salt@'%' identified by 'salt';
Query OK, 0 rows affected, 1 warning (0.01 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

配置minion

To enable this returner, the minion will need the python client for mysql installed and the following values configured in the minion or master config. These are the defaults:

mysql.host: 'salt'
mysql.user: 'salt'
mysql.pass: 'salt'
mysql.db: 'salt'
mysql.port: 3306
[root@yeqixian ~]# vim /etc/salt/minion
mysql.host: '192.168.116.180'(ip为数据库主机的IP)
mysql.user: 'salt'
mysql.pass: 'salt'
mysql.db: 'salt'
mysql.port: 3306

You have mail in /var/spool/mail/root
[root@yeqixian ~]# systemctl restart salt-minion

在Master上测试存储到mysql中

[root@xian ~]# salt '*' test.ping --return mysql
yeqixian:
    True

在数据库中查询

mysql> use salt;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select * from salt_returns\G
*************************** 1. row ***************************
       fun: test.ping
       jid: 20200826021114505713
    return: true
        id: yeqixian
   success: 1
  full_ret: {"fun_args": [], "jid": "20200826021114505713", "return": true, "retcode": 0, "success": true, "fun": "test.ping", "id": "yeqixian"}
alter_time: 2020-08-26 10:11:14
1 row in set (0.00 sec)

1.2 job cache

1.2.1 job cache流程

return时是由Minion直接与存储服务器进行交互，因此需要在每台Minion上安装指定的存储方式的模块，比如python-mysql，那么我们能否直接在Master上就把返回的结果给存储到存储服务器呢？

答案是肯定的，这种方式被称作 job cache 。意思是当Minion将结果返回给Master后，由Master将结果给缓存在本地，然后将缓存的结果给存储到指定的存储服务器，比如存储到mysql中。

先把上面的minion配置文件修改回来（在master端下载MySQL-python）

开启master端的master_job_cache

[root@xian ~]# vim /etc/salt/master
master_job_cache: mysql
mysql.host: '192.168.116.180'(ip为数据库主机的IP)
mysql.user: 'salt'
mysql.pass: 'salt'
mysql.db: 'salt'
mysql.port: 3306
[root@xian ~]# systemctl restart salt-master

在数据库服务器中清空表内容

[root@xian ~]# mysql -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11
Server version: 5.7.30 MySQL Community Server (GPL)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> delete from salt.salt_returns;
Query OK, 1 row affected (0.00 sec)

mysql> select * from salt.salt_returns;
Empty set (0.00 sec)

在master上再次测试能否存储至数据库

[root@xian ~]# salt '*' cmd.run 'df -h'
[ERROR   ] Message timed out
Salt request timed out. The master is not responding. You may need to run your command with `--async` in order to bypass the congested event bus. With `--async`, the CLI tool will print the job id (jid) and exit immediately without listening for responses. You can then use `salt-run jobs.lookup_jid` to look up the results of the job in the job cache later.
[root@xian ~]# yum -y install MySQL-python
[root@xian ~]# salt '*' cmd.run 'df -h' 
yeqixian:
    Filesystem               Size  Used Avail Use% Mounted on
    /dev/mapper/centos-root   17G  5.9G   12G  35% /
    devtmpfs                 476M     0  476M   0% /dev
    tmpfs                    488M   40K  488M   1% /dev/shm
    tmpfs                    488M  7.7M  480M   2% /run
    tmpfs                    488M     0  488M   0% /sys/fs/cgroup
    /dev/sda1               1014M  130M  885M  13% /boot
    tmpfs                     98M     0   98M   0% /run/user/0

在数据库中查询

mysql> select * from salt.salt_returns\G
*************************** 1. row ***************************
       fun: cmd.run
       jid: 20200826022854607955
    return: "Filesystem               Size  Used Avail Use% Mounted on\n/dev/mapper/centos-root   17G  5.9G   12G  35% /\ndevtmpfs                 476M     0  476M   0% /dev\ntmpfs                    488M   40K  488M   1% /dev/shm\ntmpfs                    488M  7.7M  480M   2% /run\ntmpfs                    488M     0  488M   0% /sys/fs/cgroup\n/dev/sda1               1014M  130M  885M  13% /boot\ntmpfs                     98M     0   98M   0% /run/user/0"
        id: yeqixian
   success: 1
  full_ret: {"fun_args": ["df -h"], "jid": "20200826022854607955", "return": "Filesystem               Size  Used Avail Use% Mounted on\n/dev/mapper/centos-root   17G  5.9G   12G  35% /\ndevtmpfs                 476M     0  476M   0% /dev\ntmpfs                    488M   40K  488M   1% /dev/shm\ntmpfs                    488M  7.7M  480M   2% /run\ntmpfs                    488M     0  488M   0% /sys/fs/cgroup\n/dev/sda1               1014M  130M  885M  13% /boot\ntmpfs                     98M     0   98M   0% /run/user/0", "retcode": 0, "success": true, "cmd": "_return", "_stamp": "2020-08-26T02:28:54.817452", "fun": "cmd.run", "id": "yeqixian"}
alter_time: 2020-08-26 10:28:54
1 row in set (0.00 sec)

1.2.2 job管理

获取任务的jid

[root@xian ~]# salt '*' cmd.run 'uptime' -v
Executing job with jid 20200826023437286734
-------------------------------------------

yeqixian:
     18:34:34 up  1:51,  1 user,  load average: 0.00, 0.01, 0.05
[root@xian ~]# mysql -p

mysql> select * from salt.salt_returns\G
full_ret: {"fun_args": ["df -h"], "jid": "20200826022854607955", "return": "Filesystem

通过jid获取此任务的返回结果

[root@xian ~]# salt-run jobs.lookup_jid 20200826022854607955
yeqixian:
    Filesystem               Size  Used Avail Use% Mounted on
    /dev/mapper/centos-root   17G  5.9G   12G  35% /
    devtmpfs                 476M     0  476M   0% /dev
    tmpfs                    488M   40K  488M   1% /dev/shm
    tmpfs                    488M  7.7M  480M   2% /run
    tmpfs                    488M     0  488M   0% /sys/fs/cgroup
    /dev/sda1               1014M  130M  885M  13% /boot
    tmpfs                     98M     0   98M   0% /run/user/0
[root@xian ~]# salt-run jobs.lookup_jid 20200826023437286734
yeqixian:
     18:34:34 up  1:51,  1 user,  load average: 0.00, 0.01, 0.05

2. salt-ssh

2.1 salt-ssh介绍

salt-ssh可以让我们不需要在受控机上安装salt-minion客户端也能够实现管理操作。

2.1.1 salt-ssh的特点

远程系统需要Python支持，除非使用-r选项发送原始ssh命令
salt-ssh是一个软件包，需安装之后才能使用，命令本身也是salt-ssh
salt-ssh不会取代标准的Salt通信系统，它只是提供了一个基于SSH的替代方案，不需要ZeroMQ和agent

请注意，由于所有与Salt SSH的通信都是通过SSH执行的，因此它比使用ZeroMQ的标准Salt慢得多

2.1.2 salt-ssh远程管理的方式

salt-ssh有两种方式实现远程管理，一种是在配置文件中记录所有客户端的信息，诸如 IP 地址、端口号、用户名、密码以及是否支持sudo等；另一种是使用密钥实现远程管理，不需要输入密码。

2.2 salt-ssh管理

在 master 上安装 salt-ssh

[root@xian ~]# yum -y install salt-ssh

2.2.1 通过使用用户名密码的SSH实现远程管理

修改配置文件，添加受控机信息

[root@master ~]# vim /etc/salt/roster
vm1:
  host: 192.168.116.145
  user: root
  passwd: 123456

测试连通性

[root@xian ~]# salt-ssh '*' test.ping
vm1:
    ----------
    retcode:
        254
    stderr:
    stdout:
        The host key needs to be accepted, to auto accept run salt-ssh with the -i flag:
        The authenticity of host '192.168.116.145 (192.168.116.145)' can't be established.
        ECDSA key fingerprint is SHA256:y4X6+wLvLHe4Tpb6y9wEO7OtZcktxaRB9RlBzAeegh4.
        ECDSA key fingerprint is MD5:c9:23:71:a1:a0:8a:c5:61:db:20:d6:00:5d:f2:3f:88.
        Are you sure you want to continue connecting (yes/no)?

从上面的信息可以看出，第一次访问时需要输入 yes/no ，但是 saltstack 是不支持交互式操作的，所以为了解决这个问题，我们需要对其进行设置，让系统不进行主机验证。

[root@xian ~]# vim ~/.ssh/config
StrictHostKeyChecking no
[root@xian ~]# salt-ssh '*' test.ping
Permission denied for host vm1, do you want to deploy the salt-ssh key? (password required):
[Y/n] Y
Password for root@vm1: 
vm1:
    True

2.2.2 通过salt-ssh初始化系统安装salt-minion

执行状态命令，初始化系统，安装salt-minion

[root@xian ~]# mkdir -p /srv/salt/base/{repo,files}
[root@xian ~]# \cp /etc/yum.repos.d/salt-latest.repo /srv/salt/base/repo/salt-latest.repo
[root@xian ~]# cp /etc/salt/minion /srv/salt/base/files/
[root@xian ~]# vim /srv/salt/base/repo.sls
salt-repo:
  file.managed:
    - name: /etc/yum.repos.d/salt-latest.repo
    - source: salt://repo/salt-latest.repo
    - user: root
    - group: root
    - mode: 644
[root@xian ~]# vim /srv/salt/base/minion.sls
salt-minion-install:
  pkg.installed:
    - name: salt-minion

salt-minion-conf:
  file.managed:
    - name: /etc/salt/minion
    - source: salt://files/minion
    - user: root
    - group: root
    - mode: 644
    - template: jinja
    - default:
      ID: {{ grains['ipv4'] [1] }}
    - require:
      - pkg: salt-minion-install

salt-minion-service:
  service.running:
    - name: salt-minion
    - enable: True
    - start: True
    - watch:
       - file: /etc/salt/minion
[root@xian ~]# salt-ssh '*' state.sls repo
vm1:
----------
          ID: salt-repo
    Function: file.managed
        Name: /etc/yum.repos.d/salt-latest.repo
      Result: True
     Comment: File /etc/yum.repos.d/salt-latest.repo is in the correct state
     Started: 19:04:44.339391
    Duration: 87.7 ms
     Changes:   

Summary for vm1
------------
Succeeded: 1
Failed:    0
------------
Total states run:     1
Total run time:  87.700 ms
[root@xian ~]# salt-ssh '*' state.sls minion
vm1:
----------
          ID: salt-minion-install
    Function: pkg.installed
        Name: salt-minion
      Result: True
     Comment: All specified packages are already installed
     Started: 19:05:06.348396
    Duration: 1139.422 ms
     Changes:   
----------
          ID: salt-minion-conf
    Function: file.managed
        Name: /etc/salt/minion
      Result: True
     Comment: File /etc/salt/minion updated
     Started: 19:05:07.490907
    Duration: 58.33 ms
     Changes:   
              ----------
              diff:
                  --- 
                  +++ 
                  @@ -905,11 +905,3 @@
                   ############################################
                   # Default match type for filtering events tags: startswith, endswith, find, regex, fnmatch
                   #event_match_type: startswith
                  -
                  -#mysql.host: '192.168.116.180'
                  -#mysql.user: 'salt'
                  -#mysql.pass: 'salt'
                  -#mysql.db: 'salt'
                  -#mysql.port: 3306
                  -
                  -
              mode:
                  0644
----------
          ID: salt-minion-service
    Function: service.running
        Name: salt-minion
      Result: True
     Comment: Service restarted
     Started: 19:05:07.598064
    Duration: 730.9 ms
     Changes:   
              ----------
              salt-minion:
                  True

Summary for vm1
------------
Succeeded: 3 (changed=2)
Failed:    0
------------
Total states run:     3
Total run time:   1.929 s
[root@xian ~]# salt-ssh '*' cmd.run 'systemctl restart salt-minion'
[root@xian ~]# salt-key -ya yeqixian
The following keys are going to be accepted:
Unaccepted Keys:
yeqixian
Key for minion yeqixian accepted.
[root@xian ~]# salt-key -L
Accepted Keys:
yeqixian
Denied Keys:
Unaccepted Keys:
xian
yei
Rejected Keys: