xsync 集群同步工具

前言

在配置集群时，往往需要将文件拷贝到各个机器，一来二去就很麻烦；我们可以使用 xsync 工具同时进行多台机器同步数据。

环境准备

我们准备三台虚拟机，他们的 IP 分别为 192.168.56.2、192.168.56.3、192.168.56.4，先修改它们的主机名，在不同的机器下执行以下命令：

# 192.168.56.2
echo > kafka2 /etc/hostname
# 192.168.56.3
echo > kafka3 /etc/hostname
# 192.168.56.4
echo > kafka4 /etc/hostname

完成后将每台机器添加到 hosts 文件：

vim /etc/hosts
# 添加以下内容到文件中
192.168.56.2 kafka2
192.168.56.3 kafka3
192.168.56.4 kafka4

保存后可以执行 ping 命令测试是否能通：

[root@kafka2 ~]# ping -c 3 kafka3
PING kafka3 (192.168.56.3) 56(84) bytes of data.
64 bytes from kafka3 (192.168.56.3): icmp_seq=1 ttl=64 time=0.407 ms
64 bytes from kafka3 (192.168.56.3): icmp_seq=2 ttl=64 time=0.425 ms
64 bytes from kafka3 (192.168.56.3): icmp_seq=3 ttl=64 time=0.531 ms

--- kafka3 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.407/0.454/0.531/0.057 ms

在其他机器上完成同样的步骤，前期工作就准备好了。

SSH 配置

生成 RSA 公钥

现在 192.168.56.2这台机器完成一下命令。输入ssh-keygen 直接全部按回车即可，由于我之前生成过，因此会提示覆盖

[root@kafka2 ~]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
/root/.ssh/id_rsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:O/y8hCL2Q/ltzkckr6p7f/GC+IC8PaoUhumX0azaYdQ root@kafka2
The key's randomart image is:
+---[RSA 2048]----+
|                 |
|                 |
|                 |
|   o +    . .    |
|  o = E.S  +     |
| . o *oo o  +    |
|  . X.+.*o.+ o   |
|   B +.+=B= + .  |
|  . o.**oBO+ .   |
+----[SHA256]-----+

将公钥拷贝到其他机器，输入命令 ssh-copy-id -i ~/.ssh/id_rsa.pub remote_ip remote_ip 为远程主机的 IP 地址，我们需要同时输入 .2 .3 .4 三台机器。

[root@kafka2 ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub 192.168.56.2
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.56.2's password: 
[root@kafka2 ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub 192.168.56.3
...
[root@kafka2 ~]# ssh-copy-id -i ~/.ssh/id_rsa.pub 192.168.56.4
...

上述步骤完成后，再在 192.168.56.3 与 192.168.56.4 的机器上重复上述步骤。

xsync 脚本编写

由于 xsync 是对 rsync 的再封装，因此需要先安装 rsync

[root@kafka2 ~]# yum install -y rsync

在 /usr/bin 下新建 xsync.sh

[root@kafka2 bin]# vim /usr/bin/xsync.sh

将以下内容粘贴到 xsync.sh 脚本中

#!/bin/bash

#1. 判断参数个数
if [ $# -lt 1 ]
then
    echo Not Enough Arguement!
    exit;
fi

#2. 遍历集群所有机器
for host in kafka2 kafka3 kafka4
do
    echo ====================  $host  ====================
    #3. 遍历所有目录，挨个发送

    for file in $@
    do
        #4. 判断文件是否存在
        if [ -e $file ]
            then
                #5. 获取父目录
                pdir=$(cd -P $(dirname $file); pwd)

                #6. 获取当前文件的名称
                fname=$(basename $file)
                ssh $host "mkdir -p $pdir"
                rsync -av $pdir/$fname $host:$pdir
            else
                echo $file does not exists!
        fi
    done
done

修改文件权限

[root@kafka2 bin]# chmod 777 /usr/bin/xsync.sh

测试

新建 a.txt 并且将其同步到其他服务器：

[root@kafka2 bin]# touch /usr/local/a.txt
[root@kafka2 bin]# xsync.sh /usr/local/a.txt 
==================== kafka2 ====================
sending incremental file list

sent 44 bytes  received 12 bytes  112.00 bytes/sec
total size is 0  speedup is 0.00
==================== kafka3 ====================
sending incremental file list

sent 44 bytes  received 12 bytes  112.00 bytes/sec
total size is 0  speedup is 0.00
==================== kafka4 ====================
sending incremental file list

sent 44 bytes  received 12 bytes  112.00 bytes/sec
total size is 0  speedup is 0.00

至此，xsync 工具已完成。