satlstack之return与job管理

最新推荐文章于 2023-12-28 07:30:00 发布

Zhangrl_

最新推荐文章于 2023-12-28 07:30:00 发布

阅读量287

点赞数

分类专栏： linux作业

本文链接：https://blog.csdn.net/Zhangrl_/article/details/104563567

版权

linux作业专栏收录该内容

36 篇文章 0 订阅

订阅专栏

使用mysql作为return存储方式

1.在minion上查看yum源里有Mysql-python模块

[root@node02 ~]# yum list all |grep -i mysql
MySQL-python.x86_64                     1.2.5-1.el7                myrepo       
akonadi-mysql.x86_64                    1.9.2-4.el7                myrepo       
dovecot-mysql.x86_64                    1:2.2.10-8.el7             myrepo       
libdbi-dbd-mysql.x86_64                 0.8.3-16.el7               myrepo       
mysql-connector-java.noarch             1:5.1.25-3.el7             myrepo       
mysql-connector-odbc.x86_64             5.2.5-6.el7                myrepo       
pcp-pmda-mysql.x86_64                   3.11.8-7.el7               myrepo       
perl-DBD-MySQL.x86_64                   4.023-5.el7                myrepo       
php-mysql.x86_64                        5.4.16-42.el7              myrepo       
qt-mysql.i686                           1:4.8.5-13.el7             myrepo       
qt-mysql.x86_64                         1:4.8.5-13.el7             myrepo       
qt3-MySQL.i686                          3.3.8b-51.el7              myrepo       
qt3-MySQL.x86_64                        3.3.8b-51.el7              myrepo       
qt5-qtbase-mysql.i686                   5.6.2-1.el7                myrepo       
qt5-qtbase-mysql.x86_64                 5.6.2-1.el7                myrepo       
rsyslog-mysql.x86_64                    8.24.0-12.el7              myrepo

2. 然后在minion上安装Mysql-python模块

[root@node01 ~]# salt '*' cmd.run 'rpm -qa|grep MySQL-python'
node02:
    MySQL-python-1.2.5-1.el7.x86_64
node01:
    MySQL-python-1.2.5-1.el7.x86_64

3.查看minion上的MYSQL-python是否安装完成

[root@node01 ~]# salt '*' cmd.run 'rpm -qa|grep MySQL-python'
node02:
    MySQL-python-1.2.5-1.el7.x86_64
node01:
    MySQL-python-1.2.5-1.el7.x86_64

4.另起一台新的服务器部署mysql来用作存储服务器

[root@mysql yum.repos.d]# yum -y install mariadb mariadb-server
已加载插件：product-id, search-disabled-repos, subscription-manager
This system is not registered with an entitlement server. You can use subscription-manager to register.
myrepo                                                           | 4.1 kB  00:00:00     
(1/2): myrepo/group_gz                                           | 137 kB  00:00:00     
(2/2): myrepo/primary_db                                         | 4.0 MB  00:00:00     
正在解决依赖关系
--> 正在检查事务
---> 软件包 mariadb.x86_64.1.5.5.56-2.el7 将被 安装
--> 正在处理依赖关系 /usr/bin/perl，它被软件包 1:mariadb-5.5.56-2.el7.x86_64 需要
--> 正在处理依赖关系 perl(Exporter)，它被软件包 1:mariadb-5.5.56-2.el7.x86_64 需要
--> 正在处理依赖关系 perl(Fcntl)，它被软件包 1:mariadb-5.5.56-2.el7.x86_64 需要
--> 正在处理依赖关系 perl(File::Temp)，它被软件包 1:mariadb-5.5.56-2.el7.x86_64 需要
--> 正在处理依赖关系 perl(Getopt::Long)，它被软件包 1:mariadb-5.5.56-2.el7.x86_64 需要
--> 正在处理依赖关系 perl(IPC::Open3)，它被软件包 1:mariadb-5.5.56-2.el7.x86_64 需要
--> 正在处理依赖关系 perl(Sys::Hostname)，它被软件包 1:mariadb-5.5.56-2.el7.x86_64 需要
---> 软件包 mariadb-server.x86_64.1.5.5.56-2.el7 将被 安装
--> 正在处理依赖关系 perl(DBI)，它被软件包 1:mariadb-server-5.5.56-2.el7.x86_64 需要
--> 正在处理依赖关系 perl(Data::Dumper)，它被软件包 1:mariadb-server-5.5.56-2.el7.x86_64 需要
--> 正在处理依赖关系 perl(File::Path)，它被软件包 1:mariadb-server-5.5.56-2.el7.x86_64 需要
--> 正在处理依赖关系 perl-DBD-MySQL，它被软件包 1:mariadb-server-5.5.56-2.el7.x86_64 需要
--> 正在处理依赖关系 perl-DBI，它被软件包 1:mariadb-server-5.5.56-2.el7.x86_64 需要

开启mysql服务

[root@mysql ~]# systemctl restart mariadb
[root@mysql ~]# systemctl enable mariadb
Created symlink from /etc/systemd/system/multi-user.target.wants/mariadb.service to /usr/lib/systemd/system/mariadb.service.
[root@mysql ~]# systemctl status mariadb
● mariadb.service - MariaDB database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
   Active: active (running) since 五 2020-02-28 20:47:18 CST; 16s ago
 Main PID: 2803 (mysqld_safe)
   CGroup: /system.slice/mariadb.service
           ├─2803 /bin/sh /usr/bin/mysqld_safe --basedir=/usr
           └─2965 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugi...

2月 28 20:47:16 mysql mariadb-prepare-db-dir[2725]: MySQL manual for more instructions.
2月 28 20:47:16 mysql mariadb-prepare-db-dir[2725]: Please report any problems at h...a
2月 28 20:47:16 mysql mariadb-prepare-db-dir[2725]: The latest information about Ma....
2月 28 20:47:16 mysql mariadb-prepare-db-dir[2725]: You can find additional informa...:
2月 28 20:47:16 mysql mariadb-prepare-db-dir[2725]: http://dev.mysql.com
2月 28 20:47:16 mysql mariadb-prepare-db-dir[2725]: Consider joining MariaDB's stro...:
2月 28 20:47:16 mysql mariadb-prepare-db-dir[2725]: https://mariadb.org/get-involved/
2月 28 20:47:16 mysql mysqld_safe[2803]: 200228 20:47:16 mysqld_safe Logging to '/...'.
2月 28 20:47:16 mysql mysqld_safe[2803]: 200228 20:47:16 mysqld_safe Starting mysq...ql
2月 28 20:47:18 mysql systemd[1]: Started MariaDB database server.
Hint: Some lines were ellipsized, use -l to show in full.
[root@mysql ~]# ss -antl
State      Recv-Q Send-Q Local Address:Port               Peer Address:Port              
LISTEN     0      128              *:22                           *:*                  
LISTEN     0      100      127.0.0.1:25                           *:*                  
LISTEN     0      50               *:3306                         *:*                  
LISTEN     0      128             :::22                          :::*                  
LISTEN     0      100            ::1:25                          :::*

进入mysql,创建数据库和表结构

[root@mysql ~]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 5
Server version: 5.5.56-MariaDB MariaDB Server

Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> CREATE DATABASE  `salt`  DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> use 'salt';
Database changed
MariaDB [salt]> CREATE TABLE `jids` (
    ->   `jid` varchar(255) NOT NULL,
    ->   `load` mediumtext NOT NULL,
    ->   UNIQUE KEY `jid` (`jid`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.00 sec)

MariaDB [salt]> CREATE TABLE `salt_returns` (
    ->   `fun` varchar(50) NOT NULL,
    ->   `jid` varchar(255) NOT NULL,
    ->   `return` mediumtext NOT NULL,
    ->   `id` varchar(255) NOT NULL,
    ->   `success` varchar(10) NOT NULL,
    ->   `full_ret` mediumtext NOT NULL,
    ->   `alter_time` TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    ->   KEY `id` (`id`),
    ->   KEY `jid` (`jid`),
    ->   KEY `fun` (`fun`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.00 sec)

MariaDB [salt]> CREATE TABLE `salt_events` (
    -> `id` BIGINT NOT NULL AUTO_INCREMENT,
    -> `tag` varchar(255) NOT NULL,
    -> `data` mediumtext NOT NULL,
    -> `alter_time` TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    -> `master_id` varchar(255) NOT NULL,
    -> PRIMARY KEY (`id`),
    -> KEY `tag` (`tag`)
    -> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.01 sec)
MariaDB [salt]> show tables;
+----------------+
| Tables_in_salt |
+----------------+
| jids           |
| salt_events    |
| salt_returns   |
+----------------+
3 rows in set (0.00 sec)

MariaDB [salt]> grant all on salt.* to salt@'%' identified by 'salt';        //授权访问
Query OK, 0 rows affected (0.00 sec)

MariaDB [salt]> flush privileges;
Query OK, 0 rows affected (0.00 sec)

MariaDB [salt]> quit
Bye

5.在minion上安装mariadb使用mysql命令来测试远程的mysql服务器是否可以连接

[root@node01 ~]# salt '*' pkg.install mariadb
node02:
    ----------
    mariadb:
        ----------
        new:
            1:5.5.56-2.el7
        old:
    perl:
        ----------
        new:
            4:5.16.3-292.el7
        old:
    perl-Carp:
        ----------
        new:
            1.26-244.el7
        old:
    perl-Encode:
        ----------
        new:
            2.51-7.el7
        old:
    perl-Exporter:
        ----------
[root@node02 ~]# mysql -usalt -psalt -h192.168.100.130
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 6
Server version: 5.5.56-MariaDB MariaDB Server

Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> 
MariaDB [(none)]> 
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| salt               |
| test               |
+--------------------+
3 rows in set (0.00 sec)

MariaDB [(none)]>

6.配置minion

现在其中一台的minion上配置

[root@node01 ~]# vi /etc/salt/minion
[root@node01 ~]# head -893 /etc/salt/minion|tail -6
#return: mysql
mysql.host: '192.168.100.130'
mysql.user: 'salt'
mysql.pass: 'salt'
mysql.db: 'salt'
mysql.port: 3306

将配置好的配置文件复制到另一台minion上

[root@node01 ~]# salt-cp 'node02' /etc/salt/minion  /etc/salt/
node02:
    ----------
    /etc/salt/minion:
        True
[root@node01 ~]# salt 'node02' cmd.run 'grep mysql /etc/salt/minion'
node02:
    #return: mysql
    mysql.user: 'salt'
    mysql.pass: 'salt'
    mysql.db: 'salt'
    mysql.port: 3306
    #return: mysql,slack,redis
    #  - mysql

7.重启minion上的minion服务

[root@node01 ~]# salt '*' service.restart salt-minion
[root@node01 ~]# salt '*' test.ping
node01:
    True
node02:
    True

8.在 Maste上测试存储到nysql中

[root@node01 ~]# salt '*' test.ping --return mysql
node01:
    True
node02:
    True

9.在数据库中查询数据已经成功写入进去

[root@mysql ~]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 19
Server version: 5.5.56-MariaDB MariaDB Server

Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> use salt;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
MariaDB [salt]> show tables;
+----------------+
| Tables_in_salt |
+----------------+
| jids           |
| salt_events    |
| salt_returns   |
+----------------+
MariaDB [salt]> select * from salt_returns\G;
*************************** 1. row ***************************
       fun: test.ping
       jid: 20200228214856929152
    return: true
        id: node02
   success: 1
  full_ret: {"fun_args": [], "jid": "20200228214856929152", "return": true, "retcode": 0, "success": true, "fun": "test.ping", "id": "node02"}
alter_time: 2020-02-28 21:48:57
*************************** 2. row ***************************
       fun: test.ping
       jid: 20200228214856929152
    return: true
        id: node01
   success: 1
  full_ret: {"fun_args": [], "jid": "20200228214856929152", "return": true, "retcode": 0, "success": true, "fun": "test.ping", "id": "node01"}
alter_time: 2020-02-28 21:48:57

2.job cache

2.1 开启master端的master_job_cache

[root@node01 ~]# vim /etc/salt/master
[root@node01 ~]# head -140 /etc/salt/master|tail -6
master_job_cache: mysql
mysql.host: '192.168.100.130'
mysql.user: 'salt'
mysql.pass: 'salt'
mysql.db: 'salt'
mysql.port: 3306

2.2在数据库服务器中清空之前生成的表内容

[root@mysql ~]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 22
Server version: 5.5.56-MariaDB MariaDB Server

Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> delete from salt.salt_returns;
Query OK, 6 rows affected (0.01 sec)


MariaDB [(none)]> select '*' from salt.salt_returns;
Empty set (0.00 sec)

2.3 在master上再次测试是否能够够存储到数据库

[root@node01 ~]# salt '*' cmd.run 'ls /root'
node01:
    123.sql
    2019.sql
    a
    anaconda-ks.cfg
    haha
    kkkk
    lala
    ll
    pass
node02:
    a
    anaconda-ks.cfg
    haha

在数据库中查询

[root@mysql ~]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 28
Server version: 5.5.56-MariaDB MariaDB Server

Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> use salt;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
MariaDB [salt]> select * from salt.salt_returns\G
*************************** 1. row ***************************
       fun: cmd.run
       jid: 20200229183605615414
    return: "123.sql\n2019.sql\na\nanaconda-ks.cfg\nhaha\nkkkk\nlala\nll\npass"
        id: node01
   success: 1
  full_ret: {"fun_args": ["ls /root"], "jid": "20200229183605615414", "return": "123.sql\n2019.sql\na\nanaconda-ks.cfg\nhaha\nkkkk\nlala\nll\npass", "retcode": 0, "success": true, "cmd": "_return", "_stamp": "2020-02-29T10:36:05.744654", "fun": "cmd.run", "id": "node01"}
alter_time: 2020-02-29 18:36:05
*************************** 2. row ***************************
       fun: cmd.run
       jid: 20200229183629089972
    return: "123.sql\n2019.sql\na\nanaconda-ks.cfg\nhaha\nkkkk\nlala\nll\npass"
        id: node01
   success: 1
  full_ret: {"fun_args": ["ls /root"], "jid": "20200229183629089972", "return": "123.sql\n2019.sql\na\nanaconda-ks.cfg\nhaha\nkkkk\nlala\nll\npass", "retcode": 0, "success": true, "cmd": "_return", "_stamp": "2020-02-29T10:36:29.199421", "fun": "cmd.run", "id": "node01"}
alter_time: 2020-02-29 18:36:29
*************************** 3. row ***************************
       fun: cmd.run
       jid: 20200229183629089972
    return: "a\nanaconda-ks.cfg\nhaha"
        id: node02
   success: 1
  full_ret: {"fun_args": ["ls /root"], "jid": "20200229183629089972", "return": "a\nanaconda-ks.cfg\nhaha", "retcode": 0, "success": true, "cmd": "_return", "_stamp": "2020-02-29T10:36:29.255025", "fun": "cmd.run", "id": "node02"}
alter_time: 2020-02-29 18:36:29
3 rows in set (0.00 sec)

3.job管理

获取任务的jid

[root@node01 ~]# salt '*' cmd.run 'uptime' -v
Executing job with jid 20200229183929818990
-------------------------------------------

node01:
     18:39:29 up  9:14,  2 users,  load average: 0.02, 0.08, 0.07
node02:
     18:39:29 up  9:18,  2 users,  load average: 0.00, 0.01, 0.05

通过jid来获取执行命令的返回结果，这里使用之前ls /root的jid来查询返回结果

[root@node01 ~]# salt-run jobs.lookup_jid 20200229183629089972
node01:
    123.sql
    2019.sql
    a
    anaconda-ks.cfg
    haha
    kkkk
    lala
    ll
    pass
node02:
    a
    anaconda-ks.cfg
    haha