编写了一个inotify实时同步脚本,用来检测nfs挂载目录是否有数据变化,如果有变化通过脚本里的rsync命令将数据传输的从的服务器上。
# cat /etc/nfschuanhsu.sh
#!/bin/bash
#编写 inotify 实时监控脚本
backup_Server=192.168.8.30
inotifywait=/usr/bin/inotifywait
bakData=/data/rsync/nfs-server
${inotifywait} -mrq --format '%w%f' -e create,close_write,delete ${bakData} | while read line
do
cd ${bakData}
rsync -az ./
rsyncuser1@${backup_Server}::nfs-master
--password-file=/etc/rsync/rsync33-nfs-master-passwd
done
首先,我先手动测试脚本是否可以同步成功,手动启动命令后输出了报错,开始排错并没有发现有什么问题,然后我查看了日志是否可以通过日志查看是否有什么报错信息。
# tail -f /var/log/rsync.log
2024/01/18 09:07:31 [9490] connect from NFS-5 (192.168.8.5)
2024/01/18 09:07:31 [9490] rsync allowed access on module nfs-master from NFS-5 (192.168.8.5)
2024/01/18 09:07:31 [9490] secrets file must be owned by root when running as root (see strict modes)
2024/01/18 09:07:31 [9490] auth failed on module nfs-master from NFS-5 (192.168.8.5) for rsyncuser1: ignoring secrets file
2024/01/18 09:10:33 [974] sent 0 bytes received 0 bytes total size 0
2024/01/18 09:11:04 [10318] rsyncd version 3.2.3 starting, listening on port 873
2024/01/18 09:11:33 [10433] connect from NFS-5 (192.168.8.5)
2024/01/18 09:11:33 [10433] rsync allowed access on module nfs-master from NFS-5 (192.168.8.5)
2024/01/18 09:11:33 [10433] secrets file must be owned by root when running as root (see strict modes)
2024/01/18 09:11:33 [10433] auth failed on module nfs-master from NFS-5 (192.168.8.5) for rsyncuser1: ignoring secrets file
tail: 无法使用 inotify 机制,回退为轮询(polling)机制: 打开的文件过多
查看了日志后,日志发聩的信息说tail: 无法使用 inotify 机制,回退为轮询(polling)机制: 打开的文件过多。说明了有多个进程在运行该脚本文件。然后我查看了编写的同步脚本的进程数量
查看inotify进程数量
[root@NFS-30 rsync]# crontab -l
*/1 * * * * sleep 5 && /opt/nfschuangshu.sh >>/var/log/nfsMirrorAuto.log
*/1 * * * * sleep 25 && /opt/nfschuangshu.sh >>/var/log/nfsMirrorAuto.log
*/1 * * * * sleep 45 && /opt/nfschuangshu.sh >>/var/log/nfsMirrorAuto.log
[root@NFS-30 rsync]# ps -efl | grep nfs
5 S root 781 1 0 80 0 - 1369 ep_pol 08:27 ? 00:00:00 /usr/sbin/nfsdcld
1 S root 1222 2 0 80 0 - 0 svc_ge 08:27 ? 00:00:00 [nfsd]
1 S root 1227 2 0 80 0 - 0 svc_ge 08:27 ? 00:00:00 [nfsd]
1 S root 1228 2 0 80 0 - 0 svc_ge 08:27 ? 00:00:00 [nfsd]
1 S root 1234 2 0 80 0 - 0 svc_ge 08:27 ? 00:00:00 [nfsd]
1 S root 1235 2 0 80 0 - 0 svc_ge 08:27 ? 00:00:00 [nfsd]
1 S root 1236 2 0 80 0 - 0 svc_ge 08:27 ? 00:00:00 [nfsd]
1 S root 1237 2 0 80 0 - 0 svc_ge 08:27 ? 00:00:00 [nfsd]
1 S root 1238 2 0 80 0 - 0 svc_ge 08:27 ? 00:00:00 [nfsd]
0 S root 2203 2184 0 80 0 - 55635 do_wai 08:28 ? 00:00:00 /bin/sh -c sleep 45 && /opt/nfschuangshu.sh >>/var/log/nfsMirrorAuto.log
0 S root 2205 2186 0 80 0 - 55635 do_wai 08:28 ? 00:00:00 /bin/sh -c sleep 5 && /opt/nfschuangshu.sh >>/var/log/nfsMirrorAuto.log
0 S root 2207 2185 0 80 0 - 55635 do_wai 08:28 ? 00:00:00 /bin/sh -c sleep 25 && /opt/nfschuangshu.sh >>/var/log/nfsMirrorAuto.log
[root@NFS-30 rsync]# ps -efl | grep nfs | wc -l
435
才知道导致原因时rsync+inotify定时任务导致过多的进程,400个保守估计,导致每个进程打开的文件描述符过多,知道了报错原因后将这些进程杀掉
杀死进程
[root@NFS-30 rsync]# for i in $(pgrep inotifywait);do kill $i;done
[root@NFS-30 rsync]# ps aux | grep inoti -c
1