Mar 14 18:16:16 hmsdb1 kernel: Mem-info:
Mar 14 18:16:16 hmsdb1 kernel: Zone
MA freepages: 2696 min: 0 low: 0 high: 0
Mar 14 18:16:16 hmsdb1 kernel: Zone:Normal freepages: 1227 min: 1279 low: 4544 high: 6304
Mar 14 18:16:16 hmsdb1 kernel: Zone:HighMem freepages:139078 min: 255 low: 20992 high: 31488
Mar 14 18:16:16 hmsdb1 kernel: Free pages: 143001 (139078 HighMem)
Mar 14 18:16:16 hmsdb1 kernel: ( Active: 439970/11996, inactive_laundry: 1839, inactive_clean: 1871, free: 143002 )
Mar 14 18:16:16 hmsdb1 kernel: aa:0 ac:0 id:0 il:0 ic:0 fr:2696
Mar 14 18:16:16 hmsdb1 kernel: aa:1 ac:196 id:111 il:2 ic:0 fr:1227
Mar 14 18:16:16 hmsdb1 kernel: aa:419832 ac:19942 id:11885 il:1837 ic:1871 fr:139078
Mar 14 18:16:16 hmsdb1 kernel: 0*4kB 0*8kB 0*16kB 1*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 2*4096kB = 10784kB)
Mar 14 18:16:16 hmsdb1 kernel: 221*4kB 53*8kB 29*16kB 10*32kB 6*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 4908kB)
Mar 14 18:16:16 hmsdb1 kernel: 2112*4kB 3185*8kB 8005*16kB 3712*32kB 1295*64kB 549*128kB 202*256kB 76*512kB 29*1024kB 1*2048kB 0*409
6kB = 556312kB)
Mar 14 18:16:16 hmsdb1 kernel: Swap cache: add 147175, delete 143291, find 1330942/1338061, race 0+0
Mar 14 18:16:16 hmsdb1 kernel: 188659 pages of slabcache
Mar 14 18:16:16 hmsdb1 kernel: 1302 pages of kernel stacks
Mar 14 18:16:16 hmsdb1 kernel: 0 lowmem pagetables, 219433 highmem pagetables
Mar 14 18:16:16 hmsdb1 kernel: 32 bounce buffer pages, 32 are on the emergency list
Mar 14 18:16:16 hmsdb1 kernel: Free swap: 3905144kB
Mar 14 18:16:17 hmsdb1 kernel: 1572864 pages of RAM
Mar 14 18:16:17 hmsdb1 kernel: 817645 pages of HIGHMEM
Mar 14 18:16:17 hmsdb1 kernel: 556294 reserved pages
Mar 14 18:16:17 hmsdb1 kernel: 71343058 pages shared
Mar 14 18:16:17 hmsdb1 kernel: 3884 pages swap cached
Mar 14 18:16:17 hmsdb1 kernel: Out of Memory: Killed process 27406 (oracle).
Mar 14 18:16:17 hmsdb1 kernel: Fixed up OOM kill of mm-less task
Mar 14 19:01:43 hmsdb1 sshd(pam_unix)[833]: session opened for user root by (uid=0)
Mar 14 19:05:00 hmsdb1 sshd(pam_unix)[833]: session closed for user root
Mar 14 19:06:15 hmsdb1 kernel: Mem-info:
Mar 14 19:06:15 hmsdb1 kernel: Zone
MA freepages: 2696 min: 0 low: 0 high: 0
Mar 14 19:06:15 hmsdb1 kernel: Zone:Normal freepages: 842 min: 1279 low: 4544 high: 6304
Mar 14 19:06:15 hmsdb1 kernel: Zone:HighMem freepages:154781 min: 255 low: 20992 high: 31488
Mar 14 19:06:15 hmsdb1 kernel: Free pages: 158351 (154781 HighMem)
Mar 14 19:06:15 hmsdb1 kernel: ( Active: 439124/11917, inactive_laundry: 1915, inactive_clean: 1890, free: 158351 )
Mar 14 19:06:15 hmsdb1 kernel: aa:0 ac:0 id:0 il:0 ic:0 fr:2696
Mar 14 19:06:15 hmsdb1 kernel: aa:1 ac:459 id:109 il:33 ic:0 fr:874
Mar 14 19:06:15 hmsdb1 kernel: aa:418603 ac:20061 id:11917 il:1773 ic:1890 fr:154781
Mar 14 19:06:15 hmsdb1 kernel: 0*4kB 0*8kB 0*16kB 1*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 2*4096kB = 10784kB)
Mar 14 19:06:15 hmsdb1 kernel: 99*4kB 5*8kB 3*16kB 10*32kB 4*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 3492kB)
Mar 14 19:06:15 hmsdb1 kernel: 15659*4kB 3927*8kB 8115*16kB 3739*32kB 1294*64kB 550*128kB 202*256kB 76*512kB 29*1024kB 1*2048kB 0*40
96kB = 619124kB)
Mar 14 19:06:15 hmsdb1 kernel: Swap cache: add 147449, delete 143662, find 1332855/1340248, race 0+0
Mar 14 19:06:15 hmsdb1 kernel: 188711 pages of slabcache
Mar 14 19:06:15 hmsdb1 kernel: 1304 pages of kernel stacks
Mar 14 19:06:15 hmsdb1 kernel: 0 lowmem pagetables, 205380 highmem pagetables
Mar 14 19:06:15 hmsdb1 kernel: 32 bounce buffer pages, 32 are on the emergency list
Mar 14 19:06:15 hmsdb1 kernel: Free swap: 3914208kB
Mar 14 19:06:15 hmsdb1 kernel: 1572864 pages of RAM
Mar 14 19:06:15 hmsdb1 kernel: 817645 pages of HIGHMEM
Mar 14 19:06:15 hmsdb1 kernel: 556294 reserved pages
Mar 14 19:06:15 hmsdb1 kernel: 66330875 pages shared
Mar 14 19:06:15 hmsdb1 kernel: 3787 pages swap cached
Mar 14 19:06:15 hmsdb1 kernel: Out of Memory: Killed process 5841 (oracle).
Mar 14 19:06:15 hmsdb1 kernel: 1304 pages of kernel stacks
Mar 14 19:06:15 hmsdb1 kernel: 0 lowmem pagetables, 205380 highmem pagetables
Mar 14 19:06:15 hmsdb1 kernel: 32 bounce buffer pages, 32 are on the emergency list
Mar 14 19:06:15 hmsdb1 kernel: Free swap: 3914208kB
Mar 14 19:06:15 hmsdb1 kernel: 1572864 pages of RAM
Mar 14 19:06:15 hmsdb1 kernel: 817645 pages of HIGHMEM
Mar 14 19:06:15 hmsdb1 kernel: 556294 reserved pages
Mar 14 19:06:15 hmsdb1 kernel: 66330875 pages shared
Mar 14 19:06:15 hmsdb1 kernel: 3787 pages swap cached
Mar 14 19:06:15 hmsdb1 kernel: Out of Memory: Killed process 5841 (oracle).
Mar 14 19:06:15 hmsdb1 kernel: Fixed up OOM kill of mm-less task
Mar 14 19:06:51 hmsdb1 kernel: Mem-info:
Mar 14 19:06:51 hmsdb1 kernel: Zone
MA freepages: 2696 min: 0 low: 0 high: 0
Mar 14 19:06:51 hmsdb1 kernel: Zone:Normal freepages: 775 min: 1279 low: 4544 high: 6304
Mar 14 19:06:51 hmsdb1 kernel: Zone:HighMem freepages:150990 min: 255 low: 20992 high: 31488
Mar 14 19:06:51 hmsdb1 kernel: Free pages: 154470 (150990 HighMem)
Mar 14 19:06:51 hmsdb1 kernel: ( Active: 442870/11919, inactive_laundry: 1901, inactive_clean: 1890, free: 154470 )
Mar 14 19:06:51 hmsdb1 kernel: aa:0 ac:0 id:0 il:0 ic:0 fr:2696
Mar 14 19:06:51 hmsdb1 kernel: aa:1 ac:581 id:0 il:128 ic:0 fr:784
Mar 14 19:06:51 hmsdb1 kernel: aa:422219 ac:20069 id:11919 il:1773 ic:1890 fr:150990
Mar 14 19:06:51 hmsdb1 kernel: 0*4kB 0*8kB 0*16kB 1*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 2*4096kB = 10784kB)
Mar 14 19:06:51 hmsdb1 kernel: 27*4kB 6*8kB 12*16kB 5*32kB 3*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 3132kB)
Mar 14 19:06:51 hmsdb1 kernel: 11852*4kB 3935*8kB 8115*16kB 3739*32kB 1294*64kB 550*128kB 202*256kB 76*512kB 29*1024kB 1*2048kB 0*40
96kB = 603960kB)
Mar 14 19:06:51 hmsdb1 kernel: Swap cache: add 147449, delete 143662, find 1333002/1340395, race 0+0
Mar 14 19:06:51 hmsdb1 kernel: 188676 pages of slabcache
Mar 14 19:06:51 hmsdb1 kernel: 1314 pages of kernel stacks
Mar 14 19:06:51 hmsdb1 kernel: 0 lowmem pagetables, 206211 highmem pagetables
Mar 14 19:06:51 hmsdb1 kernel: 32 bounce buffer pages, 32 are on the emergency list
Mar 14 19:06:51 hmsdb1 kernel: Free swap: 3914208kB
Mar 14 19:06:51 hmsdb1 kernel: 1572864 pages of RAM
Mar 14 19:06:51 hmsdb1 kernel: 817645 pages of HIGHMEM
Mar 14 19:06:51 hmsdb1 kernel: 556294 reserved pages
Mar 14 19:06:51 hmsdb1 kernel: 66363658 pages shared
Mar 14 19:06:51 hmsdb1 kernel: 3787 pages swap cached
Mar 14 19:06:51 hmsdb1 kernel: Out of Memory: Killed process 22109 (oracle).
如上日志所示:
lowmem不断减少
221*4kB 53*8kB 29*16kB 10*32kB 6*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 4908kB
变成
99*4kB 5*8kB 3*16kB 10*32kB 4*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 3492kB
但highmem有在增加
2112*4kB 3185*8kB 8005*16kB 3712*32kB 1295*64kB 549*128kB 202*256kB 76*512kB 29*1024kB 1*2048kB 0*409
6kB = 556312kB
变成
15659*4kB 3927*8kB 8115*16kB 3739*32kB 1294*64kB 550*128kB 202*256kB 76*512kB 29*1024kB 1*2048kB 0*40
96kB = 619124kB
此处有点疑问的是2112变成15659究竟是不是碎片增加了。
并且此时通过日志看到,swap应该还有换入换出: Swap cache: add 147449, delete 143662, find 1332855/1340248, race 0+0
不过越到后面,SWAP就只有增加而没有删除了,呵呵,看来还真的是遇到lowmem问题了?如何解决