ORA-27301: OS failure message: No buffer space available

Fri Jun 09 00:18:04 2023
skgxpvfynet: mtype: 61 process 22514 failed because of a resource problem in the OS. The OS has most likely run out of buffers (rval: 4)
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_ora_22514.trc  (incident=644991):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27504: IPC error creating OSD context
ORA-27300: OS system dependent operation:sendmsg failed with status: 105
ORA-27301: OS failure message: No buffer space available
ORA-27302: failure occurred at: sskgxpsnd2
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl1/incident/incdir_644991/orcl1_ora_22514_i644991.trc
Fri Jun 09 00:18:05 2023
Dumping diagnostic data in directory=[cdmp_20230609001805], requested by (instance=1, osid=22514), summary=[incident=644991].
opiodr aborting process unknown ospid (22514) as a result of ORA-603
Fri Jun 09 00:18:06 2023
Sweep [inc][644991]: completed
Sweep [inc2][644991]: completed

Mon Jun 12 00:27:46 2023
skgxpvfynet: mtype: 61 process 8879 failed because of a resource problem in the OS. The OS has most likely run out of buffers (rval: 4)
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_m000_8879.trc  (incident=644992):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27504: IPC error creating OSD context
ORA-27300: OS system dependent operation:sendmsg failed with status: 105
ORA-27301: OS failure message: No buffer space available
ORA-27302: failure occurred at: sskgxpsnd2
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl1/incident/incdir_644992/orcl1_m000_8879_i644992.trc
Mon Jun 12 00:27:47 2023
Dumping diagnostic data in directory=[cdmp_20230612002747], requested by (instance=1, osid=8879 (M000)), summary=[incident=644992].

Mon Jun 12 01:44:39 2023
skgxpvfynet: mtype: 61 process 23858 failed because of a resource problem in the OS. The OS has most likely run out of buffers (rval: 4)
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_ora_23858.trc  (incident=644993):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27504: IPC error creating OSD context
ORA-27300: OS system dependent operation:sendmsg failed with status: 105
ORA-27301: OS failure message: No buffer space available
ORA-27302: failure occurred at: sskgxpsnd2
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl1/incident/incdir_644993/orcl1_ora_23858_i644993.trc
opiodr aborting process unknown ospid (23858) as a result of ORA-603
Mon Jun 12 01:44:40 2023
Dumping diagnostic data in directory=[cdmp_20230612014440], requested by (instance=1, osid=23858), summary=[incident=644993].
Mon Jun 12 01:44:42 2023
Sweep [inc][644993]: completed
Sweep [inc2][644993]: completed
Mon Jun 12 01:45:33 2023
skgxpvfynet: mtype: 61 process 26865 failed because of a resource problem in the OS. The OS has most likely run out of buffers (rval: 4)
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_w000_26865.trc  (incident=644994):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27504: IPC error creating OSD context
ORA-27300: OS system dependent operation:sendmsg failed with status: 105
ORA-27301: OS failure message: No buffer space available
ORA-27302: failure occurred at: sskgxpsnd2
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl1/incident/incdir_644994/orcl1_w000_26865_i644994.trc
Mon Jun 12 01:45:33 2023
Dumping diagnostic data in directory=[cdmp_20230612014533], requested by (instance=1, osid=26865 (W000)), summary=[incident=644994].
opidrv aborting process W000 ospid (26865) as a result of ORA-603
Mon Jun 12 01:45:34 2023
Process W000 died, see its trace file
Mon Jun 12 01:45:45 2023
Sweep [inc][644994]: completed
Sweep [inc2][644994]: completed
Mon Jun 12 01:46:33 2023
Mon Jun 12 01:48:13 2023
skgxpvfynet: mtype: 61 process 28160 failed because of a resource problem in the OS. The OS has most likely run out of buffers (rval: 4)
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_m000_28160.trc  (incident=644996):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27504: IPC error creating OSD context
ORA-27300: OS system dependent operation:sendmsg failed with status: 105
ORA-27301: OS failure message: No buffer space available
ORA-27302: failure occurred at: sskgxpsnd2
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl1/incident/incdir_644996/orcl1_m000_28160_i644996.trc
Mon Jun 12 01:48:13 2023
Dumping diagnostic data in directory=[cdmp_20230612014813], requested by (instance=1, osid=28160 (M000)), summary=[incident=644996].
opidrv aborting process M000 ospid (28160) as a result of ORA-603
Mon Jun 12 01:48:14 2023
Process m000 died, see its trace file
Mon Jun 12 01:48:54 2023
Sweep [inc][644996]: completed
Sweep [inc][644995]: completed
Sweep [inc2][644996]: completed
Sweep [inc2][644995]: completed
Mon Jun 12 02:45:06 2023
skgxpvfynet: mtype: 61 process 15584 failed because of a resource problem in the OS. The OS has most likely run out of buffers (rval: 4)
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_ora_15584.trc  (incident=644997):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27504: IPC error creating OSD context
ORA-27300: OS system dependent operation:sendmsg failed with status: 105
ORA-27301: OS failure message: No buffer space available
ORA-27302: failure occurred at: sskgxpsnd2
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl1/incident/incdir_644997/orcl1_ora_15584_i644997.trc
opiodr aborting process unknown ospid (15584) as a result of ORA-603
Mon Jun 12 02:45:07 2023
Dumping diagnostic data in directory=[cdmp_20230612024507], requested by (instance=1, osid=15584), summary=[incident=644997].
Mon Jun 12 02:45:13 2023
Sweep [inc][644997]: completed
Sweep [inc2][644997]: completed
Mon Jun 12 02:48:06 2023
skgxpvfynet: mtype: 61 process 19576 failed because of a resource problem in the OS. The OS has most likely run out of buffers (rval: 4)
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_pz99_19576.trc  (incident=644998):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27504: IPC error creating OSD context
ORA-27300: OS system dependent operation:sendmsg failed with status: 105
ORA-27301: OS failure message: No buffer space available
ORA-27302: failure occurred at: sskgxpsnd2
Incident details in: /u01/app/oracle/diag/rdbms/orcl/orcl1/incident/incdir_644998/orcl1_pz99_19576_i644998.trc
opidrv aborting process PZ99 ospid (19576) as a result of ORA-603
Mon Jun 12 02:48:07 2023
Dumping diagnostic data in directory=[cdmp_20230612024807], requested by (instance=1, osid=19576 (PZ99)), summary=[incident=644998].
Mon Jun 12 02:48:08 2023
Process PZ99 died, see its trace file
Mon Jun 12 02:48:18 2023
skgxpvfynet: mtype: 61 process 19851 failed because of a resource problem in the OS. The OS has most likely run out of buffers (rval: 4)
Errors in file /u01/app/oracle/diag/rdbms/orcl/orcl1/trace/orcl1_m000_19851.trc  (incident=644999):
ORA-00603: ORACLE server session terminated by fatal error

mos文档
Oracle Linux: ORA-27301:OS Failure Message: No Buffer Space Available (Doc ID 2041723.1)

处理办法

  # ifconfig lo mtu 16436
  # echo 'MTU=16436' >> /etc/sysconfig/network-scripts/ifcfg-lo

Note : While making the changes in CRS nodes, if network is restarted while CRS is up, it can hung CRS. So cluster services should be stopped prior to the network restart.

Changing MTU values when CRS/HAS up can create vulnerabilities;
1.  Stop CRS/HAS :  RAC: <GRID_HOME>/bin/crsctl stop crs
    For Oracle restart <GRID_HOME>/bin/crsctl stop has

2. Change MTU value

3. Start CRS/HAS:
    RAC: <GRID_HOME>/bin/crsctl start crs
    Oracle Restart: <GRID_HOME>/bin/crsctl start has