Database Versions:Oracle Database – Enterprise Edition – Version 11.2.0.4
OS versions:Linux OS – Version Oracle Linux 6.5
[oracle@trsen2:/home/oracle]$uname -a
Linux trsen2 3.8.13-16.2.1.el6uek.x86_64 #1 SMP Thu Nov 7 17:01:44 PST 2013 x86_64 x86_64 x86_64 GNU/Linux
alert log file informations at RAC's node2:
Fri Nov 11 10:11:40 2016
Archived Log entry 41145 added for thread 2 sequence 6213 ID 0x43f83f07 dest 1:
Fri Nov 11 10:17:57 2016
skgxpvfynet: mtype: 61 process 22473 failed because of a resource problem in the OS. The OS has most likely run out of buffers (rval: 4)
Errors in file /u01/app/oracle/diag/rdbms/trsenpr/trsenpr2/trace/trsenpr2_ora_22473.trc (incident=1120001):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27504: IPC error creating OSD context
ORA-27300: OS system dependent operation:sendmsg failed with status: 105
ORA-27301: OS failure message: No buffer space available
ORA-27302: failure occurred at: sskgxpsnd2
Incident details in: /u01/app/oracle/diag/rdbms/trsenpr/trsenpr2/incident/incdir_1120001/trsenpr2_ora_22473_i1120001.trc
Network communication issue and High Memory consumption would be observed during this time
Memory informations from em:
2016-11-11 10:04:28 99.38
2016-11-11 10:09:28 99.35
2016-11-11 10:14:28 99.36
2016-11-11 10:19:28 99.38
2016-11-11 10:24:28 99.55
2016-11-11 10:29:28 99.55
2016-11-11 10:34:28 99.48
2016-11-11 10:39:28 99.46
2016-11-11 10:44:28 65.23
2016-11-11 10:49:28 65.20
2016-11-11 10:54:28 65.13
CAUSE:
becuase having the database backup using RMAN, then high memory consumption .
High value of MTU for loop back adapter on UEK3 causes the issue and network adapter MTU is default value(1500)
SOLUTION:
1.We can modify the value of MTU.
In UEK3, the MTU value should be modified as below
#ifconfig lo mtu 16436
To make the change persistent over reboot add the following line in the file /etc/sysconfig/network-scripts/ifcfg-lo :
MTU=16436
Save the file and restart the network service to load the changes
#service network restart
2.reduce High physical Memory,for example:
tunning sql
increate physical memory
3.The case ,we will change time of rman backup database.