redat 5.8由于文件系统100%，导致oracle数据库实例挂起处理实例

环境描述：
 操作系统 redhatlinux 5.8
 数据库版本 oracle 11.2.0.1 单实例

状况信息：
   由于本实例属于外围小系统并没有对其 文件系统，数据库实例等状态进行监控，故导致文件系统满了 导致实例挂起；

处理过程：
      1. 经过查看alter 日志：如下文标红部分 文件系统无空间了，导致redolog 无法进行切换，导致实例挂起；
     2. 由于出问题的时间为晚上10点左右，没有告警机制，导致到第二天早上过来后，清理文件系统，启动数据库实例后解决问题；
经验总结：
     增加相关监控项
     增加文件系统；

alter 告警日志如下：

VKRM started with pid=133, OS id=12948 
Mon Jul 18 22:00:01 2015
Begin automatic SQL Tuning Advisor run for special tuning task  "SYS_AUTO_SQL_TUNING_TASK"
Mon Jul 18 22:00:03 2015
DM00 started with pid=156, OS id=12966, job WAPAPP.SYS_EXPORT_SCHEMA_09
Mon Jul 18 22:00:04 2015
DW00 started with pid=158, OS id=12970, wid=1, job WAPAPP.SYS_EXPORT_SCHEMA_09
Mon Jul 18 22:00:27 2015
End automatic SQL Tuning Advisor run for special tuning task  "SYS_AUTO_SQL_TUNING_TASK"
Mon Jul 18 22:01:05 2015
Thread 1 cannot allocate new log, sequence 3948
Private strand flush not complete
  Current log# 3 seq# 3947 mem# 0: /u08/app/oracle/oradata/*/redo03.log
Mon Jul 18 22:01:21 2015
Thread 1 advanced to log sequence 3948 (LGWR switch)
  Current log# 4 seq# 3948 mem# 0: /u08/app/oracle/oradata/*/redo04.log
Mon Jul 18 22:02:08 2015
Non critical error ORA-00001 caught while writing to trace file "/u08/app/oracle/diag/rdbms/*/*/trace/*_dw00_12970.trc"
Error message: 
Writing to the above trace file is disabled for now on...
Mon Jul 18 22:02:09 2015
opidcl aborting process unknown ospid (12861) as a result of ORA-9945
Mon Jul 18 22:02:09 2015
opidcl aborting process unknown ospid (12792) as a result of ORA-9945
Mon Jul 18 22:02:09 2015
Non critical error ORA-00001 caught while writing to trace file "/u08/app/oracle/diag/rdbms/*/*/trace/*_dm00_12966.trc"
Error message: 
Writing to the above trace file is disabled for now on...
Mon Jul 18 22:02:10 2015
opidcl aborting process unknown ospid (12604) as a result of ORA-9945
Mon Jul 18 22:02:12 2015
opidcl aborting process unknown ospid (12639) as a result of ORA-9945
Mon Jul 18 22:02:13 2015
opidcl aborting process unknown ospid (12663) as a result of ORA-9945
Mon Jul 18 22:02:13 2015
opidcl aborting process unknown ospid (12691) as a result of ORA-9945
Mon Jul 18 22:02:13 2015
opidcl aborting process unknown ospid (12881) as a result of ORA-9945
Mon Jul 18 22:02:13 2015
Errors in file /u08/app/oracle/diag/rdbms/*/*/trace/*_pmon_11072.trc:
ORA-09817: Write to audit file failed.
Linux-x86_64 Error: 28: No space left on device
Additional information: 12
Errors in file /u08/app/oracle/diag/rdbms/*/*/trace/*_pmon_11072.trc:
ORA-09817: Write to audit file failed.
Linux-x86_64 Error: 28: No space left on device
Additional information: 12
Errors in file /u08/app/oracle/diag/rdbms/*/*/trace/*_pmon_11072.trc:
ORA-09817: Write to audit file failed.
Linux-x86_64 Error: 28: No space left on device
Additional information: 12
Errors in file /u08/app/oracle/diag/rdbms/*/*/trace/*_pmon_11072.trc:
ORA-09817: Write to audit file failed.Tue Jul 19 08:52:38 2015

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/25462274/viewspace-2122211/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/25462274/viewspace-2122211/