[root@salt-master salt]# salt-key 
Accepted Keys:
hddcluster1
hddcluster2
hddcluster3
hddcluster4
salt-master
salt-minion01
Denied Keys:
Unaccepted Keys:
Rejected Keys:
查看操作系统版本以及版本号
[root@salt-master salt]# salt '*' grains.item  os osrelease 
salt-master:
    ----------
    os:
        CentOS
    osrelease:
        7.2.1511
hddcluster3:
    ----------
    os:
        RedHat
    osrelease:
        7.3
hddcluster2:
    ----------
    os:
        RedHat
    osrelease:
        7.2
hddcluster4:
    ----------
    os:
        RedHat
    osrelease:
        7.2
salt-minion01:
    ----------
    os:
        CentOS
    osrelease:
        6.7
hddcluster1:
    ----------
    os:
        RedHat
    osrelease:
        7.2
[root@salt-master salt]#

使用cmd.run 命令，查看是否所有主机是否有ntp

[root@salt-master salt]# salt '*' cmd.run 'rpm -q ntp'         
salt-master:
    ntp-4.2.6p5-22.el7.centos.x86_64
hddcluster3:
    ntp-4.2.6p5-25.el7.x86_64
hddcluster4:
    ntp-4.2.6p5-22.el7.x86_64
hddcluster1:
    package ntp is not installed
hddcluster2:
    ntp-4.2.6p5-22.el7.x86_64
salt-minion01:
    package ntp is not installed
ERROR: Minions returned with non-zero exit code
[root@salt-master salt]#

其中一台hddcluster1没有安装ntp，批量进行安装

[root@salt-master salt]# salt '*' cmd.run 'yum install -y ntp'
hddcluster3:
    Loaded plugins: fastestmirror, product-id, search-disabled-repos, subscription-
                  : manager
    Bad id for repo: comments abridged, byte =   8
    Loading mirror speeds from cached hostfile
    Package ntp-4.2.6p5-25.el7.x86_64 already installed and latest version
    Nothing to do
hddcluster4:
    Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-
                  : manager
    This system is registered to Red Hat Subscription Management, but is not receiving updates. You can use subscription-manager to assign subscriptions.
    Nothing to do
hddcluster1:
    Loaded plugins: fastestmirror, product-id, search-disabled-repos, subscription-
                  : manager
    This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
    Bad id for repo: comments abridged, byte =   8
    Loading mirror speeds from cached hostfile
    Resolving Dependencies
    --> Running transaction check
    ---> Package ntp.x86_64 0:4.2.6p5-22.el7.centos.2 will be installed
    --> Processing Dependency: ntpdate = 4.2.6p5-22.el7.centos.2 for package: ntp-4.2.6p5-22.el7.centos.2.x86_64
    --> Running transaction check
    ---> Package ntpdate.x86_64 0:4.2.6p5-22.el7 will be updated
    ---> Package ntpdate.x86_64 0:4.2.6p5-22.el7.centos.2 will be an update
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ================================================================================
     Package       Arch         Version                         Repository     Size
    ================================================================================
    Installing:
     ntp           x86_64       4.2.6p5-22.el7.centos.2         updates       544 k
    Updating for dependencies:
     ntpdate       x86_64       4.2.6p5-22.el7.centos.2         updates        84 k
    
    Transaction Summary
    ================================================================================
    Install  1 Package
    Upgrade             ( 1 Dependent package)
    
    Total download size: 628 k
    Downloading packages:
    Not downloading deltainfo for updates, MD is 786 k and rpms are 84 k
    --------------------------------------------------------------------------------
    Total                                              648 kB/s | 628 kB  00:00     
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Updating   : ntpdate-4.2.6p5-22.el7.centos.2.x86_64                       1/3 
      Installing : ntp-4.2.6p5-22.el7.centos.2.x86_64                           2/3 
      Cleanup    : ntpdate-4.2.6p5-22.el7.x86_64                                3/3 
      Verifying  : ntp-4.2.6p5-22.el7.centos.2.x86_64                           1/3 
      Verifying  : ntpdate-4.2.6p5-22.el7.centos.2.x86_64                       2/3 
      Verifying  : ntpdate-4.2.6p5-22.el7.x86_64                                3/3 
    
    Installed:
      ntp.x86_64 0:4.2.6p5-22.el7.centos.2                                          
    
    Dependency Updated:
      ntpdate.x86_64 0:4.2.6p5-22.el7.centos.2                                      
    
    Complete!
salt-master:
    Loaded plugins: fastestmirror, langpacks
    Loading mirror speeds from cached hostfile
     * base: mirrors.163.com
     * epel: mirror01.idc.hinet.net
     * extras: mirrors.163.com
     * updates: centos.ustc.edu.cn
    Resolving Dependencies
    --> Running transaction check
    ---> Package ntp.x86_64 0:4.2.6p5-22.el7.centos will be updated
    ---> Package ntp.x86_64 0:4.2.6p5-22.el7.centos.2 will be an update
    --> Processing Dependency: ntpdate = 4.2.6p5-22.el7.centos.2 for package: ntp-4.2.6p5-22.el7.centos.2.x86_64
    --> Running transaction check
    ---> Package ntpdate.x86_64 0:4.2.6p5-22.el7.centos will be updated
    ---> Package ntpdate.x86_64 0:4.2.6p5-22.el7.centos.2 will be an update
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ================================================================================
     Package       Arch         Version                         Repository     Size
    ================================================================================
    Updating:
     ntp           x86_64       4.2.6p5-22.el7.centos.2         updates       544 k
    Updating for dependencies:
     ntpdate       x86_64       4.2.6p5-22.el7.centos.2         updates        84 k
    
    Transaction Summary
    ================================================================================
    Upgrade  1 Package (+1 Dependent package)
    
    Total size: 628 k
    Downloading packages:
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Updating   : ntpdate-4.2.6p5-22.el7.centos.2.x86_64                       1/4 
      Updating   : ntp-4.2.6p5-22.el7.centos.2.x86_64                           2/4 
      Cleanup    : ntp-4.2.6p5-22.el7.centos.x86_64                             3/4 
      Cleanup    : ntpdate-4.2.6p5-22.el7.centos.x86_64                         4/4 
      Verifying  : ntp-4.2.6p5-22.el7.centos.2.x86_64                           1/4 
      Verifying  : ntpdate-4.2.6p5-22.el7.centos.2.x86_64                       2/4 
      Verifying  : ntp-4.2.6p5-22.el7.centos.x86_64                             3/4 
      Verifying  : ntpdate-4.2.6p5-22.el7.centos.x86_64                         4/4 
    
    Updated:
      ntp.x86_64 0:4.2.6p5-22.el7.centos.2                                          
    
    Dependency Updated:
      ntpdate.x86_64 0:4.2.6p5-22.el7.centos.2                                      
    
    Complete!
hddcluster2:
    Loaded plugins: fastestmirror, product-id, search-disabled-repos, subscription-
                  : manager
    This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
    Bad id for repo: comments abridged, byte =   8
    Loading mirror speeds from cached hostfile
     * epel: ftp.cuhk.edu.hk
    Resolving Dependencies
    --> Running transaction check
    ---> Package ntp.x86_64 0:4.2.6p5-22.el7 will be updated
    ---> Package ntp.x86_64 0:4.2.6p5-22.el7.centos.2 will be an update
    --> Processing Dependency: ntpdate = 4.2.6p5-22.el7.centos.2 for package: ntp-4.2.6p5-22.el7.centos.2.x86_64
    --> Running transaction check
    ---> Package ntpdate.x86_64 0:4.2.6p5-22.el7 will be updated
    ---> Package ntpdate.x86_64 0:4.2.6p5-22.el7.centos.2 will be an update
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ================================================================================
     Package       Arch         Version                         Repository     Size
    ================================================================================
    Updating:
     ntp           x86_64       4.2.6p5-22.el7.centos.2         updates       544 k
    Updating for dependencies:
     ntpdate       x86_64       4.2.6p5-22.el7.centos.2         updates        84 k
    
    Transaction Summary
    ================================================================================
    Upgrade  1 Package (+1 Dependent package)
    
    Total download size: 628 k
    Downloading packages:
    Not downloading deltainfo for updates, MD is 786 k and rpms are 628 k
    --------------------------------------------------------------------------------
    Total                                              729 kB/s | 628 kB  00:00     
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Updating   : ntpdate-4.2.6p5-22.el7.centos.2.x86_64                       1/4 
      Updating   : ntp-4.2.6p5-22.el7.centos.2.x86_64                           2/4 
      Cleanup    : ntp-4.2.6p5-22.el7.x86_64                                    3/4 
      Cleanup    : ntpdate-4.2.6p5-22.el7.x86_64                                4/4 
      Verifying  : ntp-4.2.6p5-22.el7.centos.2.x86_64                           1/4 
      Verifying  : ntpdate-4.2.6p5-22.el7.centos.2.x86_64                       2/4 
      Verifying  : ntpdate-4.2.6p5-22.el7.x86_64                                3/4 
      Verifying  : ntp-4.2.6p5-22.el7.x86_64                                    4/4 
    
    Updated:
      ntp.x86_64 0:4.2.6p5-22.el7.centos.2                                          
    
    Dependency Updated:
      ntpdate.x86_64 0:4.2.6p5-22.el7.centos.2                                      
    
    Complete!
salt-minion01:
    Loaded plugins: fastestmirror
    Setting up Install Process
    Loading mirror speeds from cached hostfile
     * base: mirrors.cn99.com
     * extras: mirrors.cn99.com
     * updates: centos.ustc.edu.cn
    Resolving Dependencies
    There are unfinished transactions remaining. You might consider running yum-complete-transaction first to finish them.
    --> Running transaction check
    ---> Package ntp.x86_64 0:4.2.6p5-10.el6.centos.1 will be installed
    --> Processing Dependency: ntpdate = 4.2.6p5-10.el6.centos.1 for package: ntp-4.2.6p5-10.el6.centos.1.x86_64
    --> Running transaction check
    ---> Package ntpdate.x86_64 0:4.2.6p5-10.el6.centos.1 will be installed
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ================================================================================
     Package       Arch         Version                         Repository     Size
    ================================================================================
    Installing:
     ntp           x86_64       4.2.6p5-10.el6.centos.1         updates       598 k
    Installing for dependencies:
     ntpdate       x86_64       4.2.6p5-10.el6.centos.1         updates        78 k
    
    Transaction Summary
    ================================================================================
    Install       2 Package(s)
    
    Total download size: 677 k
    Installed size: 1.8 M
    Downloading Packages:
    --------------------------------------------------------------------------------
    Total                                            42 kB/s | 677 kB     00:16     
    Running rpm_check_debug
    Running Transaction Test
    Transaction Test Succeeded
    Running Transaction
    
      Installing : ntpdate-4.2.6p5-10.el6.centos.1.x86_64                       1/2 
    
      Installing : ntp-4.2.6p5-10.el6.centos.1.x86_64                           2/2 
    
      Verifying  : ntpdate-4.2.6p5-10.el6.centos.1.x86_64                       1/2 
    
      Verifying  : ntp-4.2.6p5-10.el6.centos.1.x86_64                           2/2 
    
    Installed:
      ntp.x86_64 0:4.2.6p5-10.el6.centos.1                                          
    
    Dependency Installed:
      ntpdate.x86_64 0:4.2.6p5-10.el6.centos.1                                      
    
    Complete!
[root@salt-master salt]# 


[root@salt-master salt]# salt '*' cmd.run 'rpm -q ntp'        
salt-master:
    ntp-4.2.6p5-22.el7.centos.2.x86_64
salt-minion01:
    ntp-4.2.6p5-10.el6.centos.1.x86_64
hddcluster4:
    ntp-4.2.6p5-22.el7.x86_64
hddcluster1:
    ntp-4.2.6p5-22.el7.centos.2.x86_64
hddcluster3:
    ntp-4.2.6p5-25.el7.x86_64
hddcluster2:
    ntp-4.2.6p5-22.el7.centos.2.x86_64
[root@salt-master salt]# salt '*' cmd.run 'rpm -q ntp'        
salt-master:
    ntp-4.2.6p5-22.el7.centos.2.x86_64
salt-minion01:
    ntp-4.2.6p5-10.el6.centos.1.x86_64
hddcluster4:
    ntp-4.2.6p5-22.el7.x86_64
hddcluster1:
    ntp-4.2.6p5-22.el7.centos.2.x86_64
hddcluster3:
    ntp-4.2.6p5-25.el7.x86_64
hddcluster2:
    ntp-4.2.6p5-22.el7.centos.2.x86_64

已经全部安装完成


下面查看一下未同步前的时间

[root@salt-master salt]# salt '*' cmd.run 'date'      
salt-master:
    Tue Nov 22 23:15:59 HKT 2016
hddcluster4:
    Tue Nov 22 15:38:33 HKT 2016
hddcluster3:
    Tue Nov 22 15:32:47 HKT 2016
hddcluster2:
    Tue Nov 22 15:39:42 HKT 2016
salt-minion01:
    Tue Nov 22 07:14:59 HKT 2016
hddcluster1:
    Tue Nov 22 15:38:23 HKT 2016

查看时间，发现所有的机器时间不同步


执行下面命令同步所有时间，cn.pool.ntp.org  是阿里云的时间同步

[root@salt-master salt]# salt '*' cmd.run 'ntpdate -u cn.pool.ntp.org'
salt-minion01:
    22 Nov 15:15:49 ntpdate[2377]: step time server 120.25.108.11 offset 28798.690332 sec
salt-master:
    22 Nov 15:15:55 ntpdate[5931]: step time server 120.25.108.11 offset -28860.874730 sec
hddcluster2:
    22 Nov 15:15:55 ntpdate[5572]: step time server 120.25.108.11 offset -1484.484047 sec
hddcluster3:
    22 Nov 15:15:55 ntpdate[25012]: step time server 120.25.108.11 offset -1069.462786 sec
hddcluster1:
    22 Nov 15:15:55 ntpdate[13243]: step time server 120.25.108.11 offset -1404.857463 sec
hddcluster4:
    22 Nov 15:15:55 ntpdate[30399]: step time server 120.25.108.11 offset -1415.696896 sec
再次查看一下，会有惊喜：
[root@salt-master salt]# salt '*' cmd.run 'date'
salt-master:
    Tue Nov 22 15:17:18 HKT 2016
hddcluster4:
    Tue Nov 22 15:17:18 HKT 2016
hddcluster2:
    Tue Nov 22 15:17:18 HKT 2016
hddcluster3:
    Tue Nov 22 15:17:18 HKT 2016
salt-minion01:
    Tue Nov 22 15:17:18 HKT 2016
hddcluster1:
    Tue Nov 22 15:17:18 HKT 2016

设置开机启动，发现centos6系列的salt-minion01主机没有找到命令，返回一个非0的退出代码

[root@salt-master salt]# salt '*' cmd.run 'systemctl enable ntpd'     
salt-minion01:
    /bin/bash: systemctl: command not found
salt-master:
    Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
hddcluster3:
    Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
hddcluster2:
    Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
hddcluster1:
    Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
hddcluster4:
    Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
ERROR: Minions returned with non-zero exit code

批量启动ntpd服务器，发现再次没有找到命令并返回非0退出代码，说有某个命令没有正常运行

[root@salt-master salt]# salt '*' cmd.run 'systemctl start ntpd' 
salt-minion01:
    /bin/bash: systemctl: command not found
salt-master:
hddcluster3:
hddcluster2:
hddcluster4:
hddcluster1:
ERROR: Minions returned with non-zero exit code

单独运行一个salt-minion01主机使用centos6系列下的命令，返回0正常。

[root@salt-master salt]# salt 'salt-minion01' cmd.run 'service ntpd start'
salt-minion01:
    Starting ntpd: ?[60G[?[0;32m  OK  ?[0;39m]
[root@salt-master salt]# salt 'salt-minion01' cmd.run 'chkconfig ntpd on' 
salt-minion01:
[root@salt-master salt]# salt 'salt-minion01' cmd.run 'chkconfig --list|grep ntpd'
salt-minion01:
    ntpd                0:off   1:off   2:on    3:on    4:on    5:on    6:off
    ntpdate             0:off   1:off   2:off   3:off   4:off   5:off   6:off
[root@salt-master salt]#