墨墨导读:根据监控平台信息,发现数据库平台节点2内存使用率过高,达到98%。通过查询占用内存较高的进程、检查TFA状态、同步TFA配置等方式,使得系统恢复正常运作。
概述
根据监控平台信息,发现某数据库平台节点2内存使用率过高,内存使用率达到98%。
1. 查询占用内存较高的进程
grid 280483 183124 0 18:21 ? 00:00:00 [asmcmd daemon]
grid 280493 155171 0 18:21 ? 00:00:00 [asmcmd daemon]
grid 280497 104733 0 18:21 ? 00:00:00 [asmcmd daemon]
grid 280499 187375 0 18:21 ? 00:00:00 [asmcmd daemon]
grid 280533 239249 0 18:21 ? 00:00:00 [asmcmd daemon]
grid 280534 157752 0 18:21 ? 00:00:00 [asmcmd daemon]
grid 280536 281960 0 18:21 ? 00:00:00 [asmcmd daemon]
grid 280545 69656 0 18:21 ? 00:00:00 [asmcmd daemon]
grid 280552 128541 0 18:21 ? 00:00:00 [asmcmd daemon]
grid 280553 63409 0 18:21 ? 00:00:00 [asmcmd daemon]
grid 280558 108705 0 18:21 ? 00:00:00 [asmcmd daemon]
grid 280575 194378 0 18:21 ? 00:00:00 [asmcmd daemon]
内存使用率暂用最高的为asmcmd daemon,这个进程究竟在做什么导致消耗这么高的内存呢?
记下来跟踪一下该进程过程。
wait4(163639, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 163639
open("/tmp/clsecho_stderr_file.txt", O_RDONLY) = 4
ioctl(4, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7ffc8333d180) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(4, 0, SEEK_CUR) = 0
fstat(4, {st_mode=S_IFREG|0644, st_size=285, ...}) = 0
fcntl(4, F_SETFD, FD_CLOEXEC) = 0
read(4, "Can't open '/oracle/app/12.2.0/g"..., 8192) = 285
stat("/oracle/app/12.2.0/grid/bin/clsecho", {st_mode=S_IFREG|0755, st_size=11405, ...}) = 0
geteuid() = 1001
geteuid() = 1001
getegid() = 501
lseek(4, 99, SEEK_SET) = 99
lseek(4, 0, SEEK_CUR) = 99
pipe([6, 7]) = 0
pipe([8, 9]) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f059005ea10) = 164739
close(9) = 0
close(7) = 0
read(8, "", 4) = 0
close(8) = 0
ioctl(6, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7ffc8333d0f0) = -1 ENOTTY (Inappropriate ioctl for device)
lseek(6, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek)
fstat(6, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
brk(0) = 0x81493d000
brk(0x81495e000) = 0x81495e000
read(6, "20-Jul-20 18:14 ASMCMD Backgroun"..., 8192) = 102
read(6, "", 8192) = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=164739, si_status=0, si_utime=0, si_stime=195} ---
fstat(6, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
close(6) = 0
brk(0) = 0x81495e000
brk(0) = 0x81495e000
brk(0x81495c000) = 0x81495c000
brk(0) = 0x81495c000
wait4(164739, [{WIFEXITED