ORA-04030: out of process memory when trying to allocate 123416 bytes (QERHJ hash-joi,kllcqas:kllslt

突然接到应用说连不上数据库了，还发来了一个截图

还好不是什么核心库，喝口水，不慌。

监听程序无法分发客户机连接——难道数据库宕了，呸呸呸，往好的方面想想，可能只是监听掉了，启动一下监听就好了。

登上数据库，监听好好的，但是为啥没实例呢。奇怪了，ps -ef |grep pmon

空空如也。。。。乌鸦嘴了，宕库了

毕竟生产库，还是先恢复吧，启库。还好没报什么错，直接就起来了。打电话通知业务，完美恢复。

 

现在到了最难受的时候了，原因，为什么会宕库呢？

好，现在就看一下alert日志吧

Errors in file /oracle/app/diag/rdbms/tasdb/tasdb/trace/tasdb_ora_4449.trc  (incident=141047): ORA-04030: out of process memory when trying to allocate 123416 bytes (QERHJ hash-joi,kllcqas:kllsltba) Incident details in: /oracle/app/diag/rdbms/tasdb/tasdb/incident/incdir_141047/tasdb_ora_4449_i141047.trc Use ADRCI or Support Workbench to package the incident. See Note 411.1 at My Oracle Support for error and packaging details. Errors in file /oracle/app/diag/rdbms/tasdb/tasdb/trace/tasdb_ora_4449.trc  (incident=141048): ORA-04030: out of process memory when trying to allocate 1052696 bytes (pga heap,log read buffer) ORA-04030: out of process memory when trying to allocate 123416 bytes (QERHJ hash-joi,kllcqas:kllsltba)

ORA-04030报错，尝试分配内存给进程时报错了。再仔细瞅一瞅，你会发现会有以下内容

Errors in file /oracle/app/diag/rdbms/tasdb/tasdb/incident/incdir_141000/tasdb_ora_4417_i141000.trc: ORA-04030: out of process memory when trying to allocate 1052696 bytes (pga heap,log read buffer) ORA-07445: exception encountered: core dump [kgdsdst()+125] [SIGSEGV] [ADDR:0x7FFF223E3E50] [PC:0x832B5AD] [Address not mapped to object] [] ORA-04030: out of process memory when trying to allocate 123416 bytes (QERHJ hash-joi,kllcqas:kllsltba)

ORA-07445报错，转储时遇到地址映射错误。继续往下看

ORA-27300: OS system dependent operation:fork failed with status: 12 ORA-27301: OS failure message: Cannot allocate memory ORA-27302: failure occurred at: skgpspawn3 Process J000 died, see its trace file kkjcre1p: unable to spawn jobq slave process  Errors in file /oracle/app/diag/rdbms/tasdb/tasdb/trace/tasdb_cjq0_12776.trc: Process startup failed, error stack: Errors in file /oracle/app/diag/rdbms/tasdb/tasdb/trace/tasdb_psp0_12679.trc:

unable to spawn jobq slave process无法生成jobq slave进程

该消息意味着交换空间不足，或者文件系统/tmp是否已满

系统日志：

bdbill genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 29233 (tnslsnr)  bdbill genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 26830 (oracle)  bdbill genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 29189 (oracle)  bdbill genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 29188 (oracle)  bdbill genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 29190 (oracle)  bdbill genunix: [ID 470503 kern.warning] WARNING: Sorry, no swap space to grow stack for pid 29278 (oracle)

这就是swap空间不足的告警了

要解决此问题，请使用后续步骤：

1. 检查syslog是否有交换空间或/tmp完整错误。
    
2. 增加swap或/tmp空间

 

总结：

ORA-04030的原因；

1、确定是否还有足够可用内存top/topas
2、操作系统的限制limit
3、查某些进程使用pga太多，PGA_ALLOCATED字段
 

How to Configure Swap Space (Doc ID 286388.1)