rsync的配置应用

高可用软件：keepalived
负载均衡：nginx/lvs/haproxy
web服务器：nginx/apache
数据库：MySQL/MongoDB/Oracle
存储：NFS/FastdFS/GFS/MFS
备份：Rsync+定时任务
实时复制：Rsync+inotify/sersync
缓存：Redis/Memcached
秘密通道：Open×××/×××
跳板机：Shell/JumpServer/CreazyEYE/GateONE
批量管理：Ansible/SaltStack/Pssh
监控：Zabbix/Nagios+cacti
网站的更新过程（代码上线流程）：办公室测试环境-线上测试环境-线上环境
1.从安装系统开始准备
安装虚拟机过程中添加网卡（网络适配器2 NAT）

网络适配器2，LAN区段：172.16.1.0/24

eth0 ip 10.0.0.210 netmask 24
gateway 10.0.0.254
DNS 223.5.5.5

eth1 ip 172.16.1.210 netmask 24

磁盘分区如下：

安装包选择如下：

2.统一网络环境网关都是 10.0.0.254

本机我的电脑===>网络连接

2.5 服务器ip地址规划服务器说明外网IP（NAT）内网IP（LAN区段/Host-only）主机名规划
A1-nginx负载服务器 01 10.0.0.5/24 172.16.1.5/24 lb01
A1-nginx负载服务器 02 10.0.0.6/24 172.16.1.6/24 lb02
B2-nginx web服务器10.0.0.7/24 172.16.1.7/24 web02
B2-nginx web服务器10.0.0.8/24 172.16.1.8/24 web01
C3-mysql数据库服务器10.0.0.51/24 172.16.1.51/24 db01
C1-NFS存储服务器10.0.0.31/24 172.16.1.31/24 nfs01
C2-rsync存储服务器10.0.0.41/24 172.16.1.41/24 backup
X-管理服务器10.0.0.61/24 172.16.1.61/24 m01

3.系统优化

#所有服务器模板机的基本优化
cp /etc/hosts{,.bak}
cat >/etc/hosts<<EOF
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.1.5 lb01
172.16.1.6 lb02
172.16.1.7 web02
172.16.1.8 web01
172.16.1.51 db01 db01.etiantian.org
172.16.1.31 nfs01
172.16.1.41 backup
172.16.1.61 m01
EOF
#0、更改yum源
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
mv /etc/yum.repos.d/epel.repo /etc/yum.repos.d/epel.repo.backup
mv /etc/yum.repos.d/epel-testing.repo /etc/yum.repos.d/epel-testing.repo.backup
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-6.repo
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-6.repo
#yum makecache
#1、关闭selinux
sed -i.bak 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
grep SELINUX=disabled /etc/selinux/config
setenforce 0
getenforce
#2、关闭iptables
/etc/init.d/iptables stop
/etc/init.d/iptables stop
chkconfig iptables off
#3、精简开机自启动服务
export LANG=en
chkconfig|egrep -v "crond|sshd|network|rsyslog|sysstat"|awk '{print"chkconfig",$1,"off"}'|bash
chkconfig --list|grep 3:on
#4、提权oldboy可以sudo
useradd oldboy
echo 123456|passwd --stdin oldboy
\cp /etc/sudoers /etc/sudoers.ori
echo "oldboy ALL=(ALL) NOPASSWD: ALL " >>/etc/sudoers
visudo -c
#5、英文字符集
cp /etc/sysconfig/i18n /etc/sysconfig/i18n.ori
echo 'LANG="en_US.UTF-8"' >/etc/sysconfig/i18n
source /etc/sysconfig/i18n
echo $LANG
#6、时间同步
echo '#time sync by lidao at 2017-03-08' >>/var/spool/cron/root
echo '/5 /usr/sbin/ntpdate ntp1.aliyun.com >/dev/null 2>&1' >>/var/spool/cron/root
crontab -l
#8、加大文件描述
echo ' -nofile 65535 ' >>/etc/security/limits.conf
tail -1 /etc/security/limits.conf
#9、内核优化
cat >>/etc/sysctl.conf<<EOF
net.ipv4.tcp_fin_timeout = 2
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_keepalive_time = 600
net.ipv4.ip_local_port_range = 4000 65000
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_max_tw_buckets = 36000
net.ipv4.route.gc_timeout = 100
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_synack_retries = 1
net.core.somaxconn = 16384
net.core.netdev_max_backlog = 16384
net.ipv4.tcp_max_orphans = 16384
#以下参数是对iptables防火墙的优化，防火墙不开会提示，可以忽略不理。
net.nf_conntrack_max = 25000000
net.netfilter.nf_conntrack_max = 25000000
net.netfilter.nf_conntrack_tcp_timeout_established= 180
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120
EOF
sysctl -p
#10、安装其他小软件
yum install lrzsz nmap tree dos2unix nc telnet sl -y
#11、ssh连接速度慢优化
sed -i.bak 's@#UseDNS yes@UseDNS no@g;s@^GSSAPIAuthentication yes@GSSAPIAuthentication no@g' /etc/ssh/sshd_config
/etc/init.d/sshd reload

4.学会克隆虚拟机
!!!!!!!!!!
克隆前准备
1清2删
1清

/etc/udev/rules.d/70-persistent-net.rules

2删
sed -ri '/UUID|HWADDR/d' /etc/sysconfig/network-scripts/ifcfg-eth*

修改IP地址、修改主机名、重启网卡
sed -i '/IPADDR/s#210$#31#g' /etc/sysconfig/network-scripts/ifcfg-eth*
vi /etc/sysconfig/network
/etc/init.d/network restart
20180526 完成

Rsync 的企业工作场景说明

1.）定时备份
1.1.生产场景集群架构服务器备份方案项目
借助cron+rsync把所有客户服务器数据同步到备份服务器

1.针对公司重要数据备份混乱状况和领导提出备份全网数据的解决方案。
2.通过本地打包备份，然后rsync结合inotify应用把全网数据统一备份到一个固定存储服务器，然后在存储服务器上通过脚本检查并报警管理员备份结果。
3.定期将IDC机房的数据备份公司的内部不服务器，防止机房地震及火灾问题导致数据丢失。

Rsync的工作方式：
最常用的：以守护进程（socket）的方式传输数据（这个是rsync自身的重要功能）。
一般用推送

2.）实时复制

本地数据传输模式（local-only mode）
/oldboy/ 与 /oldboy
/oldboy 表示oldboy目录及oldboy目录下面的内容
/oldboy/ 只表示oldboy目录下面的内容
rsync /etc/hosts /opt/
rsync -avz /etc /opt/
--delete 复制的时候保持目标与源目录的内容一模一样
例子：# rsync -avz --delete /opt/ /tmp/

通过远程shell进行数据传输
scp命令
例子：# scp -rP 22 /etc/ 172.16.1.31:/tmp
rsync命令 -v 详细模式输出 -z 传输时进行压缩 -a 以递归方式传输文件，并保持所有文件属性

例子：# rsync -avz /etc/sysconfig 172.16.1.31:/tmp/ （推送，用的最多）

--bwlimit=RATE limit socket I/O bandwidth（案例：某DBA做数据同步，带宽占满，导致用户无法访问网站）

rsync服务端backup配置

[root@backup ~]# mkdir /backup
[root@backup ~]# useradd -s /sbin/nologin -M rsync
[root@backup ~]# chown -R rsync.rsync /backup
[root@backup ~]# ls -ld /backup
drwxr-xr-x 2 rsync rsync 4096 Jan 18 23:18 /backup
[root@backup ~]# echo 'rsync_backup:123456' >/etc/rsync.password
[root@backup ~]# cat /etc/rsync.password
rsync_backup:123456
[root@backup ~]# chmod 600 /etc/rsync.password
[root@backup ~]# ll /etc/rsync.password
-rw------- 1 root root 20 Jan 18 23:32 /etc/rsync.password

[root@backup ~]# vim /etc/rsyncd.conf
######rsync_config___start
#created by hkping 15:01 2018-5-27
##rsyncd.conf start##
uid = rsync
gid = rsync
use chroot = no
max connections = 200
timeout = 300
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log
[backup]
path = /backup/
ignore errors
read only = false
list = false
hosts allow = 172.16.1.0/24
#hosts deny = 0.0.0.0/32
auth users = rsync_backup
secrets file = /etc/rsync.password

[root@backup /]# rsync --daemon
[root@backup ~]# ps -ef |grep rsync
root 24946 1 0 23:34 ? 00:00:00 rsync --daemonroot 24948 1787 0 23:34 pts/1 00:00:00 grep rsync
[root@backup ~]# ss -lntup |grep rsync
tcp LISTEN 0 5 :::873 ::: users:(("rsync",24946,5))
tcp LISTEN 0 5 :873 : users:(("rsync",24946,4))

客户端
[root@nfs01 tmp]# echo '123456' > /etc/rsync.password
[root@nfs01 tmp]# chmod 600 /etc/rsync.password
[root@nfs01 tmp]# rsync -avzP /etc/hosts rsync_backup@172.16.1.41::backup --password-file=/etc/rsync.password
sending incremental file list
sent 26 bytes received 8 bytes 68.00 bytes/sec
total size is 349 speedup is 10.26
[root@nfs01 /]# rsync -avzP --delete /tmp/ rsync_backup@172.16.1.41::backup --password-file=/etc/rsync.password #--delete 无差异同步

更改完/etc/rsyncd.conf配置文件后，重启rsync --daemon
[root@backup ~]# killall rsync
[root@backup ~]# killall rsync
rsync: no process killed
[root@backup ~]# ps -ef|grep rsync
root 26693 26671 0 19:13 pts/0 00:00:00 grep rsync
[root@backup ~]# rsync --daemon
[root@backup ~]# ps -ef|grep rsync
root 26695 1 0 19:13 ? 00:00:00 rsync --daemon
root 26697 26671 0 19:13 pts/0 00:00:00 grep rsync

rsync配置多个模块
[root@backup ~]# cat /etc/rsyncd.conf
######rsync_config___start
#created by oldboy 15:01 2007-6-5
#QQ 31333741 blog:http://oldboy.blog.51cto.com
##rsyncd.conf start##
uid = rsync
gid = rsync
use chroot = no
max connections = 200
timeout = 300
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log

ignore errors
read only = false
list = false
hosts allow = 172.16.1.0/24
#hosts deny = 0.0.0.0/32
auth users = rsync_backup
secrets file = /etc/rsync.password

[backup]
path = /backup/
[nfsbackup]
path = /nfsbackup/

更改完/etc/rsyncd.conf配置文件后，重启rsync --daemon，新建nfsbackup，更改目录权限
[root@backup backup]# killall rsync
rsync: no process killed
[root@backup backup]# ps -ef|grep rsync
root 25681 1844 0 23:21 pts/0 00:00:00 grep rsync
[root@backup backup]# rsync --daemon
[root@backup /]# mkdir -p /nfsbackup
[root@backup /]# chown -R rsync.rsync /nfsbackup/
[root@backup /]# ll -d /nfsbackup/
drwxr-xr-x 2 rsync rsync 4096 May 27 23:35 /nfsbackup/

某公司里有一台Web服务器，里面的数据很重要，但是如果硬盘坏了，数据就会丢失，现在领导要求你把数据在其他机器上做一个周期性定时备份。要求如下：每天晚上00点整在Web服务器A(web01)上打包备份网站程序目录并通过rsync命令推送到服务器B(backup)上备份保留（备份思路可以是先在本地按日期打包，然后再利用rsync推到备份服务器上）。具体要求如下：
1)Web服务器A和备份服务器B的备份目录必须都为/backup。2)Web服务器站点目录假定为(/var/www/html)。3)Web服务器本地仅保留7天内的备份。4)备份服务器上每周六的数据都保留，其他备份仅保留180天备份。
5)备份服务器上检查备份结果是否正常，并将每天的备份结果发给管理员信箱。
rsync服务端
[root@backup ~]# killall rsync
[root@backup ~]# killall rsyncrsync
: no process killed
[root@backup ~]# rsync --daemon
[root@backup ~]#[root@backup ~]#[root@backup ~]# ps -ef |grep rsyncroot 26755 1 0 19:45 ? 00:00:00 rsync --daemonroot 26757 26671 0 19:45 pts/0 00:00:00 grep rsync
[root@backup ~]# cat /etc/rsyncd.conf######rsync_config___start#created by oldboy 15:01 2007-6-5 #QQ 31333741 blog:http://oldboy.blog.51cto.com##rsyncd.conf start##uid = rsyncgid = rsyncuse chroot = no max connections = 200 timeout = 300 pid file = /var/run/rsyncd.pidlock file = /var/run/rsync.locklog file = /var/log/rsyncd.log
ignore errors
read only = false
list = false
hosts allow = 172.16.1.0/24
#hosts deny = 0.0.0.0/32
auth users = rsync_backup
secrets file = /etc/rsync.password

[backup]
path = /backup/

[root@backup ~]# ll /etc/rsync.password
-rw------- 1 root root 20 Jan 18 23:32 /etc/rsync.password
[root@backup ~]# cat /etc/rsync.password
rsync_backup:123456
[root@backup ~]# ll /backup/
total 0
[root@backup ~]#

[root@web01 ~]# echo '123456'>/etc/rsync.password
[root@web01 ~]# chmod 600 /etc/rsync.password
[root@web01 ~]# ll /etc/rsync.password
-rw------- 1 root root 7 Jan 19 19:50 /etc/rsync.password
[root@web01 ~]#
[root@web01 ~]#
[root@web01 ~]#
[root@web01 ~]#
[root@web01 ~]# rsync -avz /etc/hosts rsync_backup@172.16.1.41::backup
--password-file=/etc/rsync.password
sending incremental file list
hosts

sent 196 bytes received 27 bytes 446.00 bytes/sectotal size is 349 speedup is 1.57
web01脚本
#backup file && conf
tar zchf /backup/web01-$(date +%F_%w).tar.gz /var/www/html/ /etc/rc.local /var/spool/cron/

#make md5sum
cd /backup/
md5sum web01-$(date +%F_%w).tar.gz >check.log

#rsync file to backup
rsync -avz /backup/ rsync_backup@172.16.1.41::backup --password-file=/etc/rsync.password

#delete 7 day ago
find /backup/ -type f -name "*.tar.gz" -mtime +7|xargs rm

#!/bin/bash
#desc: backup html && conf

Date=$(date +%F_%w)

#backup file && conf
cd / && \
tar zchf /backup/web01-${Date}.tar.gz var/www/html/ etc/rc.local var/spool/cron/

#make md5sum
cd /backup/ && \
md5sum web01-${Date}.tar.gz >check.log

#rsync file to backup
rsync -az /backup/ rsync_backup@172.16.1.41::backup --password-file=/etc/rsync.password

#delete 7 day ago
find /backup/ -type f -name "*.tar.gz" -mtime +7|xargs rm

backup 脚本
[root@backup backup]# cat /server/scripts/check.sh
#!/bin/bash

#del 180 day file keep every 6
find /backup/ -type f -name ".tar.gz" -mtime +180 ! -name "_6.tar.gz"|xargs rm

#check
cd /backup/ && md5sum -c check.log >/backup/result.log
if [ $? -ne 0 ];then

mail -s "md5sum wrong" 918391635@qq.com </backup/result.log
fi
#需要开启postfix[root@backup backup]# /etc/init.d/postfix startStarting postfix: [ OK ][root@backup backup]# chkconfig postfix on
mail -s "md5sum wrong" kang50055@163.com </etc/hosts
[root@backup backup]# crontab -l |tail -2
#check & send result lee at 2017-01-01
00 03 * /bin/sh /server/scripts/check.sh >/dev/null 2>&1
图片集合
中小型网站搭建---大酒店https://www.processon.com/view/link/5a5dfca1e4b05a8ff305c36f
老男孩教育-链接克隆与完整克隆区别https://www.processon.com/view/link/5a609837e4b0abe85d60af61
老男孩教育-rsync守护进程模式部署流程https://www.processon.com/view/link/5a60b7f8e4b010a6e7231d10
老男孩教育-全网备份项目逻辑图https://www.processon.com/view/link/5a61d763e4b05a8ff30e5df7

转载于:https://blog.51cto.com/kang50055/2122282