脚本实现全网数据备份搭建(rsync)

目录

1 全网数据备份搭建整体思路

1.1 拓扑图

这里写图片描述

1.2 需求分析

1)3台服务器主机名分别为A(web01)、B(backup)、C(nfs01)

2)要求每晚00点整在Web服务器上打包备份系统配置文件、网站程序目录及访问日志并通过rsync命令推送备份服务器B上备份保留(备份思路可以是现在本地按日期打包,然后再推送到备份服务器B上)。

1.3 具体要求

Web01:

1)Web服务器A和备份服务器B的备份目录必须都为/backup

2)要求备份的系统配置文件包括但不限于: 
a.定时任务服务的配置文件(/var/spool/cron/root)

b.开机自动启动的配置文件(/etc/rc.local)

c.日常脚本的目录(/server/scripts/)

d.防火墙iptables的配置文件(/etc/sysconfig/iptables)

e.Web服务器站点目录(/var/www/html)

f.Web服务器A访问日志路径假定为(/app/logs)

3)Web服务器保留打包后的7天的备份数据即可(本地保留不能多于7天,因为太多硬盘会满)

backup备份服务器:

1)备份服务器B上保留每周一的所有数据副本,其它要保留6个月的数据副本。

2)备份服务器B上要按照备份数据服务器的IP为目录保存备份,备份的文件按照时间名字保存。

3)要确保备份的数据尽量完整正确,在备份服务器上对备份的数据进行检查,把备份的成功及失败结果信息发送给系统管理员邮箱中。

NFS服务器:

1)在NFS服务端C(nfs01)上共享/data/w_shared及/data/r_shared两个文件目录,允许从NFS客户端A(web01)、B(backup)上分别挂载共享目录后可实现从A(web01)、B(backup)上只读/data/r_shared,可写/data/w_shared。

2)NFS客户端A(web01)上的挂载点为/data/b_w(写),/data/b_r(读)

3)NFS客户端B(backup)上的挂载点为/data/w_你的名字英文(写),/data/r_你的名字英文(读)。

4)从NFS客户端B(backup)上的NFS可写挂载点目录创建任意文件,从NFS客户端A(web01)上可以删除这个创建的文件,反之也可以。

5)优化NFS服务。

实时数据同步:

当用户通过web服务器将数据写入到NFS服务器C(nfs01)时,同时复制到备份服务器B(backup)

1.4 IP地址规划

服务器名称内网IP外网IP主机名
Web服务172.16.1.8/24192.168.90.8/24web01
NFS服务172.16.1.31/24192.168.90.31/24nfs01
Backup备份服务172.16.1.41/24192.168.90.41/24backup

2 实施步骤

2.1 系统优化脚本

#每个新的服务器上最好执行一遍,下面这个脚本只是简单的优化下系统

cat /server/scripts/youhua.sh
#!/bin/bash
#
export PATH=/usr/local/sbin:/usr/1ocal/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

#Shudown iptables
/etc/init.d/iptables stop
chkconfig iptables off

#Shutdown SELinux
setenforce 0
sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux.config
grep SELINUX=disabled /etc/selinux/config

#Simplify system services
export LANG=en
chkconfig |egrep -v "sshd|crond|network|rsyslog|sysstat" |awk '{print "chkconfig",$1,"off"}' |bash
chkconfig --list | grep "3:on"

#Sudo config
useradd oldboy
cp /etc/sudoers /etc/sudoers.ori
echo "oldboy ALL=(ALL) NOPASSWD:ALL " >>/etc/sudoers
tail -1 /etc/sudoers
visudo -c

#Time sync
echo '#time sync by oldboy' >>/var/spool/cron/root
echo "*/5 * * * * /usr/sbin/ntpdate ntp1.aliyun.com > /dev/null 2>&1" >>/var/spool/cron/root
crontab -l

#Command line setting
echo 'export TMOUT=300' >>/etc/profile
echo 'export HISTSIZE=5' >>/etc/profile
echo 'export HISTFILESIZE=5' >>/etc/profile
tail -3 /etc/profile
source /etc/profile

#Increase the description of the file
echo '*     -       nofile          65535' >> /etc/security/limits.conf
tail -1 /etc/security/limits.conf

#Kernel tuning
cat >>/etc/sysctl.conf<<EOF
net.ipv4.tcp_fin_timeout = 2
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_keepalive_time = 600
net.ipv4.ip_local_port_range = 4000 65000
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_max_tw_buckets = 36000
net.ipv4.route.gc_timeout = 100
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_synack_retries = 1
net.core.somaxconn = 16384
net.core.netdev_max_backlog = 16384
net.ipv4.tcp_max_orphans = 16384
net.nf_conntrack_max = 25000000
net.netfilter.nf_conntrack_max = 25000000
net.netfilter.nf_conntrack_tcp_timeout_established = 180
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120
EOF
sysctl -p

#Update the yum source
wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-6.repo
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-6.repo
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49
  • 50
  • 51
  • 52
  • 53
  • 54
  • 55
  • 56
  • 57
  • 58
  • 59
  • 60
  • 61
  • 62
  • 63
  • 64
  • 65
  • 66
  • 67
  • 68
  • 69
  • 70
  • 71

2.2 主机名解析脚本

cat >>/etc/hosts<< EOF
172.16.1.5  lb01
172.16.1.6  lb02
172.16.1.7  web01
172.16.1.8  web02
172.16.1.51 db01 db01.etiantian.org
172.16.1.31 nfs01
172.16.1.41 backup
172.16.1.61 m01
EOF
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10

2.3 backup服务器

2.3.1 搭建rsync服务

cat /server/scripts/rsyncser.sh
#!/bin/bash
#Write by rsq at 2018/02/08

export PATH=/usr/local/sbin:/usr/1ocal/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

#Variable declaration
user=rsync
authuser=rsync_backup
passwd=123456
passwdfile=/etc/rsync.password
bakpath1=/backup
bakpath2=/nfsbackup

#Install rsync
yum -y install rsync


#Create user
useradd $user -s /sbin/nologin -M


#Create configfile
cat >>/etc/rsyncd.conf<<EOF
#rsync_config_________________start
#created by rsq
##rsyncd.conf start##
uid = rsync
gid = rsync
use chroot = no
max connections = 200
timeout = 300
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log
ignore errors
read only = false
list = false
hosts allow = 172.16.1.0/24
host deny = 0.0.0.0/32
auth users = rsync_backup
secrets file = /etc/rsync.password
[backup]
path = /backup/

[nfsbackup]
path = /nfsbackup
#rsync_config_________________end
EOF


#Create passwdfile
echo "$authuser:$passwd" > $passwdfile
chmod 600 $passwdfile


#Create backup path
mkdir -p $bakpath1
mkdir -p $bakpath2
chown -R $user. $bakpath1
chown -R $user. $bakpath2


#Onboot
echo "/usr/bin/rsync --daemon" >>/etc/rc.local


#Start daemon
rsync --daemon
lsof -i :873
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49
  • 50
  • 51
  • 52
  • 53
  • 54
  • 55
  • 56
  • 57
  • 58
  • 59
  • 60
  • 61
  • 62
  • 63
  • 64
  • 65
  • 66
  • 67
  • 68
  • 69
  • 70
  • 71

2.3.2 邮件搭建及检查脚本

#简单搭建邮件

cat >>/etc/mail.rc<<EOF
set from=cactirsq@163.com smtp=smtp.163.com smtp-auth-user=cactirsq smtp-auth-password=xxxxxxxx smtp-auth=login
EOF
  • 1
  • 2
  • 3
  • 4

#md5检查web01是否传送成功,没有丢包,有的话发邮件

cat >>/server/scripts/check.sh<<EOF
#!/bin/bash
#Check web01 backcup data
#Write by rsq at 2018/02/08

export PATH=/usr/local/sbin:/usr/1ocal/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

#Variable declaration
bakpath=/backup
find $bakpath -type f -name "flag_$(date +%F).log"|xargs md5sum -c $flagfile >>$bakpath/$(date +%F)_result.log 2>&1
mail -s "$(date +%F) backup result" 960503480@qq.com < $bakpath/$(date +%F)_result.log && \
find $bakpath -type f ! -name "*_1.tar.gz" -mtime +180|xargs rm -f
EOF
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14

2.3.3 backup添加定时任务

cat >>/var/spool/cron/root<<EOF
#check for web01 backup data
00 04 * * * /bin/bash /server/scripts/check.sh &> /dev/null
EOF
  • 1
  • 2
  • 3
  • 4
  • 5

2.4 web01服务器

2.4.1 web01 rsync客户端配置

cat /server/scripts/rsyncclient.sh
#!/bin/bash
#Write by rsq at 2018/02/08

export PATH=/usr/local/sbin:/usr/1ocal/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

#Variable declaration
authuer=rsync_backup
passwd=123456
passwdfile=/etc/rsync.password
bakpath=/backup

#Install rsync
yum -y install rsync


#Create passwdfile
echo "$passwd" >>$passwdfile
chmod 600 $passwdfile


#create backup path
mkdir -p $bakpath
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24

2.4.2 web01创建所需的目录

#由于没有安装web服务,只是做简单的测试,故需要创建一开始不存在的目录

mkdir -p /app/logs

mkdir -p /var/www/html
  • 1
  • 2
  • 3
  • 4

2.4.3 web01备份到backup的脚本

cat /server/scripts/backup.sh
#!/bin/bash
#Write by rsq at 2018/02/08

export PATH=/usr/local/sbin:/usr/1ocal/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

#Variable declaration
IP=$(ifconfig eth1 | awk -F'[ :]+' 'NR==2{print $4}')
bakpath=/backup
user=rsync_backup
bakserver=172.16.1.41
module=backup
passwdfile=/etc/rsync.password

if [ ! -e $bakpath/$IP ];then
    mkdir -p $bakpath/$IP
fi

if [ $(date +%w) -eq 1 ];then
    time=$(date +%F_%w)
else
    time=$(date +%F)
fi

cd / && \
tar zchf $bakpath/$IP/web_$time.tar.gz var/spool/cron/root etc/rc.local var/www/html app/logs etc/sysconfig/iptables server/scripts etc/sysctl.conf && \

md5sum $bakpath/$IP/web_$time.tar.gz > $bakpath/$IP/flag_$(date +%F).log &&\

rsync -az $bakpath $user@$bakserver::$module/ --password-file=$passwdfile

find $bakpath -type f -mtime +7 \( -name "*.tar.gz" -o -name "*.log" \)|xargs rm -f
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33

2.4.4 为web服务添加定时任务

cat >>/var/spool/cron/root<<EOF
#backup 
00 00 * * * /bin/bash /server/scripts/backup.sh
EOF
  • 1
  • 2
  • 3
  • 4
  • 5

2.5 NFS服务器

2.5.1 搭建NFS服务

cat /server/scripts/nfsser.sh 
#!/bin/bash
#Write by rsq at 2018/02/08

export PATH=/usr/local/sbin:/usr/1ocal/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

#Variable declaration
share1=/data/r_shared
share2=/data/w_shared
user=nfsnobody
group=nfsnobody
IP=$(ifconfig eth1 | awk -F'[ :]+' 'NR==2{print $4}')
net=$(ifconfig eth1 | awk -F'[ :]+' 'NR==2{print $4}'|cut -d"." -f1-3)
mask=".0/24"
opt1="ro,sync,all_squash,anonuid=65534,anongid=65534"
opt2="rw,sync,all_squash,anonuid=65534,anongid=65534"


#Install nfs
yum -y install rpcbind nfs-utils


#Share folder
mkdir -p $share1 $share2
chown -R $user.$group $share1
chown -R $user.$group $share2


#Kernel youhua
cat >>/etc/sysctl.conf<<EOF
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 1677216
net.core.wmem_max = 1677216
EOF
sysctl -p


#NFS config file
cat >>/etc/exports<<EOF
$share2 $net$mask($opt2)
$share1 $net$mask($opt1)
EOF


#start service and onboot
/etc/init.d/rpcbind start && \
/etc/init.d/nfs start
echo "/etc/init.d/rpcbind start" >>/etc/rc.local
echo "/etc/init.d/nfs start" >>/etc/rc.local


#Test
showmount -e $IP
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49
  • 50
  • 51
  • 52
  • 53
  • 54
  • 55

2.5.2 NFS客户端搭建(web01,nfs01)

cat /server/scripts/nfsmount.sh 
#!/bin/bash
#Write by rsq at 2018/02/08

export PATH=/usr/local/sbin:/usr/1ocal/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

#Variable declaration
nfsserver=172.16.1.31
user=nfsnobody
group=nfsnobody
mountpoint1=/data/b_r
mountpoint2=/data/b_w
mountopt="nosuid,noexec,nodev,noatime,rsize=131072,wsize=131072"


#Install nfs
yum install -y rpcbind nfs-utils


#Only start rpcbind service
/etc/init.d/rpcbind start


#mkdir 
mkdir -p $mountpoint1 $mountpoint2
chown -R $user.$group $mountpoint1
chown -R $user.$group $mountpoint2


#mount 
share1=$(showmount -e $nfsserver|awk 'NR==2{print $1}')
share2=$(showmount -e $nfsserver|awk 'NR==3{print $1}')

mount -t nfs -o $mountopt $nfsserver:$share1 $mountpoint1
mount -t nfs -o $mountopt $nfsserver:$share2 $mountpoint2


#mount onboot
echo "mount -t nfs -o $mountopt $nfsserver:$share1 $mountpoint1" >>/etc/rc.local
echo "mount -t nfs -o $mountopt $nfsserver:$share2 $mountpoint2" >>/etc/rc.local
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41

2.5.3 NFS实时同步

#安装inotify

yum -y install inotify-tools
rpm -qa | grep inotify-tools

#youhua
echo 655350 >/proc/sys/fs/inotify/max_user_watches
echo 655350 >/proc/sys/fs/inotify/max_queued_events

#Create passwd_file
echo 123456 > /etc/rsync.password
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10

inotify.sh

#!/bin/bash
PATH=/data
IP=172.16.1.41

/usr/bin/inotifywait -mrq --format '%w%f' -e close_write,delete $PATH \
|while read file
do
    if [ -f $file ];then
      /usr/bin/rsync -az $file --delete rsync_backup@$IP::nfsbackup --password-file=/etc/rsync.password
    else
      cd $PATH && \
      /usr/bin/rsync -az ./ --delete rsync_backup@$IP::nfsbackup --password-file=/etc/rsync.password
    fi
done
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15

#设置inotify服务脚本,即在/etc/init.d/目录下创建一个syncd文件

cat /etc/init.d/syncd
#!/bin/bash
#chkconfig: 2345 38 46
#############################################################################
#this scripts is inotify start or stop
#############################################################################
. /etc/rc.d/init.d/functions

function usage(){
    echo "$0 {start|stop}"
}

if [ $# -ne 1 ];then
    usage
    exit 1
fi

case "$1" in
  start)
    /bin/bash /server/scripts/inotify.sh &
    echo $$ >/var/run/inotify.pid
    if [ `ps -ef|grep inotify|wc -l` -gt 2 ];then
        action "inotify service is started" /bin/true
    else
        action "inotify service is started" /bin/false
        fi
        ;;

  stop)
    kill -9 `cat /var/run/inotify.pid` >/dev/null 2>&1
    pkill inotifywait
    sleep 2
    if [ `ps -ef|grep inotify|grep -v grep|wc -l` -eq 0 ];then
        action "inotify service is stoped" /bin/true
    else
        action "inotify service is stoped" /bin/false
    fi
    ;;

  *)
    usage
    exit 1

esac
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45

#启动服务脚本

chmod +x /etc/init.d/syncd
chkconfig --add syncd
chkconfig syncd on
chkconfig --list syncd

# start
/etc/init.d/syncd start
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8

3 故障排错

主要是脚本内容的原因,有IP地址写错、变量名字后边引用错误等。

客户端挂载nfs反了。


转载至https://blog.csdn.net/mr_rsq/article/details/79579247


评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值