场景一:
背景:
1、应用采用weblogic中间件,但是应用启动的时候是通过nohup启动的,nohup.out一直在写入
2、应用不能停,会影响业务连续性
3、nohup.out日志不能移除或者重命名,否则应用不重启的情况下,nohup.out无法继续写入
4、日志不断写入的情况下,nohup.out日志不断增大达到几十个G,阅读性较差
解决思路:
1、先将nohup.out 复制一份,复制完之后,通过cat /dev/null > nohup.out将nohup.out清空
2、通过split 将日志200M大小拆分
3、每小时执行一次,将拆分出来的日志放到对应数据日期的文件件下
4、每小时拆出来多个文件时,以MMSS00001,MMSS00002的形式保存
5、每天对数据日期文件夹进行压缩归档
环境:
Redhat7.9
代码:
splitLog.sh
#!/bin/bash
#curent date 当前日期 例如:20200101
curtdate=`date +"%Y%m%d"`
#curent time 当前时间 例如:0800
curttime=`date +"%H%M"`
#curttime=`date -d "1 hour ago" +"%H%M"`
echo "curent date: $curtdate"
echo "curent hours: $curttime"
#curtime = `date +"%Y%m%d%H%M%S"`
#source path 源文件路径(nohup.out所在文件夹)
sourcepath=/home/weblogic/Oracle/user_projects/domains/base_domain/bin/
#source file 源文件绝对路径
sourcefile="$sourcepath"nohup.out
echo "sourcefile: $sourcefile"
#tmp file 创建临时文件名称
sourcefile1="$sourcepath"nohup1.out
echo "sourcefile1: $sourcefile1"
#targetpath 目标文件路径(nohup.out拆分后存放路径)
targetpath=/home/weblogic/logs/"$curtdate"/
targetfile="$targetpath"nohup.out"$curttime"
echo "targetpath: $targetpath"
echo "targetfile: $targetfile"
#target path is exist 判断目标路径是否存在,如果不存在则创建
if [ -d $targetpath ] ;then
echo "$targetpath is existed"
else
echo "mkdir $targetpath "
mkdir -p $targetpath
fi
echo " target: $targetpath"
#size of sourcefile 判断源文件大小
fileSize=`ls -l $sourcefile | awk '{print $5}'`
#size to split 设置需要拆分的大小为:200M
splitSize=209715200
echo " splitSize: $splitSize"
echo " fileSize: $fileSize"
#当源文件大小大于200M时进行拆分,当源文件小于200M时直接拷贝
if [ "$fileSize" -gt "$splitSize" ];then
echo "split start big "
if [ ! -f $sourcefile1 ]; then
cp $sourcefile $sourcefile1
echo " copy tmp file done"
cat /dev/null > $sourcefile
echo "mkde null file done"
echo "splitSize: $splitSize"
split -b "$splitSize" -d -a 5 $sourcefile1 $targetfile
echo "split file done"
rm $sourcefile1
echo "remove tmp file done"
else
split -b "$splitSize" -d -a 5 $sourcefile1 $targetfile
rm $sourcefile1
echo "sourcefile1 is existed split done "
fi
else
echo "mv small nohup.out start"
cp $sourcefile $targetfile
echo "copy target file done"
cat /dev/null > $sourcefile
echo "make null sourcefile done"
fi
compressLogs.sh
#!/bin/bash
echo `date +"%Y%m%d%H%M%S"`
logpath="/home/weblogic/logs"
days=0
# tar directory days before
find $logpath -maxdepth 1 -mindepth 1 -type d -ctime +"$days" -exec tar -cvzf {}.tar.gz {} \;
# rm directory days before
if [ $? -ne 0 ];then
echo "tar logs failed"
else
find $logpath -maxdepth 1 -mindepth 1 -type d -ctime +"$days" -exec rm -rf {} \;
echo "rm logs done"
fi
crontab 设置
crontab -e
59 */1 * * * sh /weblogic/logs/splitLog.sh > splitLog.log
00 00 * * * sh /weblogic/logs/compressLogs.sh > compressLogs.log
场景二:
背景:
应用系统有多个路径下有日志需要清理,文件路径比较多,想提高代码复用
解决思路:
将路径写成数组参数,通过for循环对日志进行备份和清理,为保障安全性,先将10天前的原日志文件备份到backup路径下,然后在backup路径下对日志文件进行打包,然后对20天之前的打包文件进行删除
环境:
RedHat7.9
代码:
clearlog.sh
#!/bin/bash
curentdate=$(date +"%Y-%m-%d");
backdate=$(date -d "$curentdate -10 day" +"%Y-%m-%d");
deletedate=$(date -d "$curentdate -20 day" +"%Y-%m-%d");
echo $curentdate
echo $backdate
echo $deletedate
logsDir=("/data/logs/log1/" "/data/logs/log2/" "/data/logs/log3/" "/data/logs/log4/" "/data/logs/log5/" "/data/logs/log6/");
logsBackDir=("/data/backup/logs/log1/" "/data/backup/logs/log2/" "/data/backup/logs/log3/" "/data/backup/logs/log4/" "/data/backup/logs/log5/" "/data/backup/logs/log6/");
#判断是否有备份目录,如没有就创建
for ((i=0; i<${#logsBackDir[@]}; i++))
do
backdir=${logsBackDir[$i]};
echo "$backdir";
if [ ! -d "$backdir" ];then
mkdir -p $backdir;
echo "创建目录:$backdir";
fi
done
#将8天前的日期文件夹移到备份文件夹中
for ((i=0; i<${#logsDir[@]}; i++))
do
# 获取日志和备份路径
logdir=${logsDir[$i]}
backdir=${logsBackDir[$i]}
# 移动日志文件夹到备份路径下
if [ -d "$logdir$backdate" ];then
mv $logdir$backdate $backdir;
echo "移动 $logdir$backdate 到 $backdir";
else
echo "$logdir$backdate 不存在无需移动";
fi
done
#打包备份文件夹,并删除180天以前的备份日志:
for ((i=0; i<${#logsBackDir[@]}; i++))
do
backdir=${logsBackDir[$i]};
echo "$backdir";
cd $backdir;
if [ -d "$backdate" ];then
zip -rm $backdate.zip $backdate;
echo "$backdate.zip 打包成功";
else
echo "$backdir$backdate 不存在无需打包";
fi
if [ -f "$deletedate.zip" ];then
rm -f $deletedate.zip;
echo "删除$deletedate.zip成功";
else
echo "$backdir$deletedate.zip 不存在无需删除";
fi
done