RSYNC实现数据同步（运维笔记）

最新推荐文章于 2024-05-03 12:35:43 发布

Rkun18

最新推荐文章于 2024-05-03 12:35:43 发布

阅读量885

点赞数

分类专栏：运维文章标签：运维笔记 linux

本文链接：https://blog.csdn.net/weixin_51882166/article/details/130167412

版权

运维专栏收录该内容

46 篇文章 0 订阅

订阅专栏

RSYNC实现数据同步（运维笔记）

线上MIS系统部分代码备份到另一台主机上
线上MIS系统服务器的代码存放目录为/app/code_project

备份机器要求每天12：00定时同步MIS服务器的/app/code_project目录下所有文件

要求同步日志，方便同步失败分析原因

备份的本质就是复制

rsync可以实现增量拷贝（只拷贝改变过的数据）

rsync本地同步文件

[root@server1 ~]# mkdir /dir1 /dir2
[root@server1 ~]# cd /dir1
[root@server1 dir1]# touch file{1..3}
[root@server1 dir1]# mkdir aa{1..3}
[root@server1 dir1]# ll
total 12
drwxr-xr-x 2 root root 4096 Apr 14 15:14 aa1
drwxr-xr-x 2 root root 4096 Apr 14 15:14 aa2
drwxr-xr-x 2 root root 4096 Apr 14 15:14 aa3
-rw-r--r-- 1 root root    0 Apr 14 15:14 file1
-rw-r--r-- 1 root root    0 Apr 14 15:14 file2
-rw-r--r-- 1 root root    0 Apr 14 15:14 file3
[root@server1 dir1]# ll /dir2
total 0

改变其中一个文件所属组，并将/dir1下的文件同步到/dir2下

[root@server1 dir1]# chgrp admin aa1
[root@server1 dir1]# ll
total 12
drwxr-xr-x 2 root admin 4096 Apr 14 15:14 aa1
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa2
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa3
-rw-r--r-- 1 root root     0 Apr 14 15:14 file1
-rw-r--r-- 1 root root     0 Apr 14 15:14 file2
-rw-r--r-- 1 root root     0 Apr 14 15:14 file3
[root@server1 dir1]# rsync -av /dir1 /dir2/
sending incremental file list
dir1/
dir1/file1
dir1/file2
dir1/file3
dir1/aa1/
dir1/aa2/
dir1/aa3/

sent 310 bytes  received 89 bytes  798.00 bytes/sec
total size is 0  speedup is 0.00
[root@server1 dir1]# ll /dir2
total 4
drwxr-xr-x 5 root root 4096 Apr 14 15:14 dir1

上述同步先将/dir1同步过来，再递归同步下面的文件

[root@server1 dir1]# cd /dir2/dir1
[root@server1 dir1]# ll
total 12
drwxr-xr-x 2 root admin 4096 Apr 14 15:14 aa1
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa2
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa3
-rw-r--r-- 1 root root     0 Apr 14 15:14 file1
-rw-r--r-- 1 root root     0 Apr 14 15:14 file2
-rw-r--r-- 1 root root     0 Apr 14 15:14 file3

发现修改属性后的文件同时也同步了属性

如果只想把dir1/下的文件同步过去

[root@server1 dir1]# rsync -av /dir1/ /dir2/
sending incremental file list
./
file1
file2
file3
aa1/
aa2/
aa3/

sent 300 bytes  received 88 bytes  776.00 bytes/sec
total size is 0  speedup is 0.00
[root@server1 dir1]# cd /dir2
[root@server1 dir2]# ll
total 16
drwxr-xr-x 2 root admin 4096 Apr 14 15:14 aa1
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa2
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa3
drwxr-xr-x 5 root root  4096 Apr 14 15:14 dir1
-rw-r--r-- 1 root root     0 Apr 14 15:14 file1
-rw-r--r-- 1 root root     0 Apr 14 15:14 file2
-rw-r--r-- 1 root root     0 Apr 14 15:14 file3

增加一个文件

[root@server1 dir2]# cd /dir1
[root@server1 dir1]# ll
total 12
drwxr-xr-x 2 root admin 4096 Apr 14 15:14 aa1
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa2
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa3
-rw-r--r-- 1 root root     0 Apr 14 15:14 file1
-rw-r--r-- 1 root root     0 Apr 14 15:14 file2
-rw-r--r-- 1 root root     0 Apr 14 15:14 file3
[root@server1 dir1]# echo hello >test1
[root@server1 dir1]# ll
total 16
drwxr-xr-x 2 root admin 4096 Apr 14 15:14 aa1
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa2
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa3
-rw-r--r-- 1 root root     0 Apr 14 15:14 file1
-rw-r--r-- 1 root root     0 Apr 14 15:14 file2
-rw-r--r-- 1 root root     0 Apr 14 15:14 file3
-rw-r--r-- 1 root root     6 Apr 14 15:45 test1

同步到/dir2目录下

[root@server1 dir1]# rsync -av /dir1/ /dir2/
sending incremental file list
./
test1

sent 250 bytes  received 41 bytes  582.00 bytes/sec
total size is 6  speedup is 0.02

只同步了test1文件

先删除一些文件并且增加一些文件同步文件

[root@server1 dir1]# rm -f file*
[root@server1 dir1]# touch test{2..5}
[root@server1 dir1]# ll
total 16
drwxr-xr-x 2 root admin 4096 Apr 14 15:14 aa1
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa2
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa3
-rw-r--r-- 1 root root     6 Apr 14 15:45 test1
-rw-r--r-- 1 root root     0 Apr 14 15:50 test2
-rw-r--r-- 1 root root     0 Apr 14 15:50 test3
-rw-r--r-- 1 root root     0 Apr 14 15:50 test4
-rw-r--r-- 1 root root     0 Apr 14 15:50 test5
[root@server1 dir1]# rsync -av /dir1/ /dir2/
sending incremental file list
./
test2
test3
test4
test5

sent 371 bytes  received 98 bytes  938.00 bytes/sec
total size is 6  speedup is 0.01

查看/dir2下的文件

[root@server1 dir1]# ll /dir2
total 20
drwxr-xr-x 2 root admin 4096 Apr 14 15:14 aa1
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa2
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa3
drwxr-xr-x 5 root root  4096 Apr 14 15:14 dir1
-rw-r--r-- 1 root root     0 Apr 14 15:14 file1
-rw-r--r-- 1 root root     0 Apr 14 15:14 file2
-rw-r--r-- 1 root root     0 Apr 14 15:14 file3
-rw-r--r-- 1 root root     6 Apr 14 15:45 test1
-rw-r--r-- 1 root root     0 Apr 14 15:50 test2
-rw-r--r-- 1 root root     0 Apr 14 15:50 test3
-rw-r--r-- 1 root root     0 Apr 14 15:50 test4
-rw-r--r-- 1 root root     0 Apr 14 15:50 test5

发现之前的文件并没有被删除，只是增添了新的文件

我的目的是同步/dir1/下的文件 /dir2/下的多余的文件被删除

可以查看man文档或者rsync --hlep

   --delete                delete extraneous files from destination dirs

查看命令，这句是删除多余的文件从目标目录

使用命令同步文件，删除多余文件

[root@server1 dir1]# rsync -av --delete /dir1/ /dir2/  
sending incremental file list
deleting dir1/aa3/
deleting dir1/aa2/
deleting dir1/aa1/
deleting dir1/file3
deleting dir1/file2
deleting dir1/file1
deleting dir1/
deleting file3
deleting file2
deleting file1

sent 208 bytes  received 132 bytes  680.00 bytes/sec
total size is 6  speedup is 0.02
[root@server1 dir1]# ll /dir2
total 16
drwxr-xr-x 2 root admin 4096 Apr 14 15:14 aa1
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa2
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa3
-rw-r--r-- 1 root root     6 Apr 14 15:45 test1
-rw-r--r-- 1 root root     0 Apr 14 15:50 test2
-rw-r--r-- 1 root root     0 Apr 14 15:50 test3
-rw-r--r-- 1 root root     0 Apr 14 15:50 test4
-rw-r--r-- 1 root root     0 Apr 14 15:50 test5

rsync远程同步文件

将server1上的/dir1上的文件远程同步到server2 /backup上

[root@server1 /]# ll /dir1
total 16
drwxr-xr-x 2 root admin 4096 Apr 14 15:14 aa1
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa2
drwxr-xr-x 2 root root  4096 Apr 14 15:14 aa3
-rw-r--r-- 1 root root     6 Apr 14 15:45 test1
-rw-r--r-- 1 root root     0 Apr 14 15:50 test2
-rw-r--r-- 1 root root     0 Apr 14 15:50 test3
-rw-r--r-- 1 root root     0 Apr 14 15:50 test4
-rw-r--r-- 1 root root     0 Apr 14 15:50 test5
[root@server1 /]# rsync -av /dir1 120.55.65.27:/backup
The authenticity of host '120.55.65.27 (120.55.65.27)' can't be established.
ECDSA key fingerprint is SHA256:ELFCCafVHildP7YqcOfyiHz60tjEOyP9+JOVAA5nYvs.
ECDSA key fingerprint is MD5:ab:f2:49:c0:3d:0d:a9:f9:52:1e:3c:8a:fc:50:d5:40.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '120.55.65.27' (ECDSA) to the list of known hosts.
root@120.55.65.27's password: 
sending incremental file list
dir1/
dir1/test1
dir1/test2
dir1/test3
dir1/test4
dir1/test5
dir1/aa1/
dir1/aa2/
dir1/aa3/

sent 444 bytes  received 131 bytes  67.65 bytes/sec
total size is 6  speedup is 0.01

在server2上查看：

[root@server2 ~]# ll /backup
total 4
drwxr-xr-x 5 root root 4096 Apr 14 15:50 dir1
[root@server2 ~]# ll /backup/dir1
total 16
drwxr-xr-x 2 root code 4096 Apr 14 15:14 aa1
drwxr-xr-x 2 root root 4096 Apr 14 15:14 aa2
drwxr-xr-x 2 root root 4096 Apr 14 15:14 aa3
-rw-r--r-- 1 root root    6 Apr 14 15:45 test1
-rw-r--r-- 1 root root    0 Apr 14 15:50 test2
-rw-r--r-- 1 root root    0 Apr 14 15:50 test3
-rw-r--r-- 1 root root    0 Apr 14 15:50 test4
-rw-r--r-- 1 root root    0 Apr 14 15:50 test5

只同步/dir1/下的文件，不同步文件目录：

[root@server1 /]# rsync -av /dir1/ 120.55.65.27:/backup/
root@120.55.65.27's password: 
sending incremental file list
./
test1
test2
test3
test4
test5
aa1/
aa2/
aa3/

sent 433 bytes  received 130 bytes  86.62 bytes/sec
total size is 6  speedup is 0.01

查看：

[root@server2 ~]# ll /backup
total 20
drwxr-xr-x 2 root code 4096 Apr 14 15:14 aa1
drwxr-xr-x 2 root root 4096 Apr 14 15:14 aa2
drwxr-xr-x 2 root root 4096 Apr 14 15:14 aa3
drwxr-xr-x 5 root root 4096 Apr 14 15:50 dir1
-rw-r--r-- 1 root root    6 Apr 14 15:45 test1
-rw-r--r-- 1 root root    0 Apr 14 15:50 test2
-rw-r--r-- 1 root root    0 Apr 14 15:50 test3
-rw-r--r-- 1 root root    0 Apr 14 15:50 test4
-rw-r--r-- 1 root root    0 Apr 14 15:50 test5

使用免密登录就可以使rsync不使用密码把数据同步至远程主机

rsync作为后台程序使用

启动它（找启动脚本）
监听端口 873端口（默认）

[root@server1 ~]# netstat -nltp|grep 873

配置文件

#尝试后台程序方式启动 必须要有配置文件  如果没有必须创建一个/etc/rsyncd.conf
[root@server1 ~]# rsync --daemon
[root@server1 ~]# netstat -nltp|grep 873
tcp        0      0 0.0.0.0:873             0.0.0.0:*               LISTEN      5174/rsync          
tcp6       0      0 :::873                  :::*                    LISTEN      5174/rsync

#寻找rsync位置

[root@server1 ~]# which rsync
/usr/bin/rsync

#查找软件包
[root@server1 ~]# rpm -qf /usr/bin/rsync
rsync-3.1.2-12.el7_9.x86_64

#寻找启动脚本

[root@server1 ~]# rpm -ql rsync
/etc/rsyncd.conf
/etc/sysconfig/rsyncd
/usr/bin/rsync   #程序本身
/usr/lib/systemd/system/rsyncd.service
/usr/lib/systemd/system/rsyncd.socket
/usr/lib/systemd/system/rsyncd@.service
/usr/share/doc/rsync-3.1.2
/usr/share/doc/rsync-3.1.2/COPYING
/usr/share/doc/rsync-3.1.2/NEWS
/usr/share/doc/rsync-3.1.2/OLDNEWS
/usr/share/doc/rsync-3.1.2/README
/usr/share/doc/rsync-3.1.2/support
/usr/share/doc/rsync-3.1.2/support/Makefile
/usr/share/doc/rsync-3.1.2/support/atomic-rsync
/usr/share/doc/rsync-3.1.2/support/cvs2includes
/usr/share/doc/rsync-3.1.2/support/deny-rsync
/usr/share/doc/rsync-3.1.2/support/file-attr-restore
/usr/share/doc/rsync-3.1.2/support/files-to-excludes
/usr/share/doc/rsync-3.1.2/support/git-set-file-times
/usr/share/doc/rsync-3.1.2/support/instant-rsyncd
/usr/share/doc/rsync-3.1.2/support/logfilter
/usr/share/doc/rsync-3.1.2/support/lsh
/usr/share/doc/rsync-3.1.2/support/lsh.sh
/usr/share/doc/rsync-3.1.2/support/mapfrom
/usr/share/doc/rsync-3.1.2/support/mapto
/usr/share/doc/rsync-3.1.2/support/mnt-excl
/usr/share/doc/rsync-3.1.2/support/munge-symlinks
/usr/share/doc/rsync-3.1.2/support/rrsync
/usr/share/doc/rsync-3.1.2/support/rsync-no-vanished
/usr/share/doc/rsync-3.1.2/support/rsync-slash-strip
/usr/share/doc/rsync-3.1.2/support/rsyncstats
/usr/share/doc/rsync-3.1.2/support/savetransfer.c
/usr/share/doc/rsync-3.1.2/tech_report.tex
/usr/share/man/man1/rsync.1.gz
/usr/share/man/man5/rsyncd.conf.5.gz 
#启动脚本一般在/etc/下 这里没有找到


#查询rsync配置文档
man rsyncd.conf

#看到这一段
The rsync daemon is launched by specifying the --daemon option to rsync

#rsync可以作为守护进程启动

实施步骤

RSYNC实现数据同步 （运维笔记）

- 线上MIS系统部分代码备份到另一台主机上

- 线上MIS系统服务器的代码存放目录为/app/code_project


备份机器要求每天24：00定时同步MIS服务器的/app/code_project目录下所有文件

要求同步日志，方便同步失败分析原因

#server2作为MIS系统服务器
[root@server2 ~]# mkdir /app/code_project -p
[root@server2 ~]# touch /app/code_project/file{1..5}


#进入配置文件 填写代码路径和日志路径

vim /etc/rsyncd.conf


[app1]
path = /app/code_project
log file = /var/log/rsync.log


#启动rsync
[root@server2 ~]# rsync --daemon
[root@server2 ~]# netstat -ntlp|grep 873
tcp        0      0 0.0.0.0:873             0.0.0.0:*               LISTEN      13172/rsync         
tcp6       0      0 :::873                  :::*                    LISTEN      13172/rsync

man rsync文档查看语法

Pull: rsync [OPTION...] [USER@]HOST::SRC... [DEST]
               rsync [OPTION...] rsync://[USER@]HOST[:PORT]/SRC... [DEST]
         Push: rsync [OPTION...] SRC... [USER@]HOST::DEST
               rsync [OPTION...] SRC... rsync://[USER@]HOST[:PORT]/DEST

server1作为备份机器

[root@server1 /]# mkdir /backup

#从server2上拉取文件到/backup
[root@server1 /]# rsync -a 120.55.65.27::
app1               
[root@server1 /]# rsync -av 120.55.65.27::app1 /backup/
receiving incremental file list
./
file1
file2
file3
file4
file5

sent 122 bytes  received 314 bytes  872.00 bytes/sec
total size is 0  speedup is 0.00
[root@server1 /]# ll /backup
total 0
-rw-r--r-- 1 root root 0 Apr 15 10:14 file1
-rw-r--r-- 1 root root 0 Apr 15 10:14 file2
-rw-r--r-- 1 root root 0 Apr 15 10:14 file3
-rw-r--r-- 1 root root 0 Apr 15 10:14 file4
-rw-r--r-- 1 root root 0 Apr 15 10:14 file5

创建脚本，交给计划任务去执行，实现自动拉取

备份机器要求每天12：00定时同步MIS服务器的/app/code_project目录下所有文件

要求同步日志，方便同步失败分析原因

root@server1 ~]# vim test.sh
[root@server1 ~]# ll
total 896
drwxr-xr-x 9 ecs-assist-user ecs-assist-user   4096 Apr 11 21:42 nginx-1.10.1
-rw-r--r-- 1 root            root            909077 Jun  1  2016 nginx-1.10.1.tar.gz
-rw-r--r-- 1 root            root                50 Apr 15 10:49 test.sh
[root@server1 ~]# chmod +x test.sh
[root@server1 ~]# cat test.sh
#!/bin/bash
rsync -av 120.55.65.27::app1 /backup/

设置计划任务

[root@server1 ~]# crontab -e
[root@server1 ~]# crontab -l
00 12 * * * /root/test.sh &>/dev/null

删除/backup文件修改系统时间测试

[root@server1 ~]# cd /backup
[root@server1 backup]# ll
total 0
-rw-r--r-- 1 root root 0 Apr 15 10:14 file1
-rw-r--r-- 1 root root 0 Apr 15 10:14 file2
-rw-r--r-- 1 root root 0 Apr 15 10:14 file3
-rw-r--r-- 1 root root 0 Apr 15 10:14 file4
-rw-r--r-- 1 root root 0 Apr 15 10:14 file5
[root@server1 backup]# rm -rf file*
[root@server1 backup]# ll
total 0

[root@server1 backup]# date -s 11:59:00
Sat Apr 15 11:59:00 CST 2023
[root@server1 backup]# clock -w
[root@server1 backup]# date
Sat Apr 15 11:59:09 CST 2023
[root@server1 backup]# date
Sat Apr 15 11:59:55 CST 2023
[root@server1 backup]# ll
total 0
-rw-r--r-- 1 root root 0 Apr 15 10:14 file1
-rw-r--r-- 1 root root 0 Apr 15 10:14 file2
-rw-r--r-- 1 root root 0 Apr 15 10:14 file3
-rw-r--r-- 1 root root 0 Apr 15 10:14 file4
-rw-r--r-- 1 root root 0 Apr 15 10:14 file5

Rkun18

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
RSYNC实现数据同步（运维笔记）

线上MIS系统部分代码备份到另一台主机上线上MIS系统服务器的代码存放目录为/app/code_project备份机器要求每天12：00定时同步MIS服务器的/app/code_project目录下所有文件要求同步日志，方便同步失败分析原因备份的本质就是复制rsync可以实现增量拷贝（只拷贝改变过的数据）
复制链接

扫一扫