环境描述:
操作系统 redhatlinux 5.8
数据库版本 oracle 11.2.0.1 单实例
状况信息:
由于本实例属于外围小系统并没有对其 文件系统,数据库实例等状态进行监控,故导致文件系统满了 导致实例挂起;
处理过程:
1. 经过查看alter 日志:如下文标红部分 文件系统无空间了,导致redolog 无法进行切换,导致实例挂起;
2. 由于出问题的时间为晚上10点左右,没有告警机制,导致到第二天早上过来后,清理文件系统,启动数据库实例后解决问题;
经验总结:
增加相关监控项
增加文件系统;
alter 告警日志如下:
VKRM started with pid=133, OS id=12948
Mon Jul 18 22:00:01 2015
Begin automatic SQL Tuning Advisor run for special tuning task "SYS_AUTO_SQL_TUNING_TASK"
Mon Jul 18 22:00:03 2015
DM00 started with pid=156, OS id=12966, job WAPAPP.SYS_EXPORT_SCHEMA_09
Mon Jul 18 22:00:04 2015
DW00 started with pid=158, OS id=12970, wid=1, job WAPAPP.SYS_EXPORT_SCHEMA_09
Mon Jul 18 22:00:27 2015
End automatic SQL Tuning Advisor run for special tuning task "SYS_AUTO_SQL_TUNING_TASK"
Mon Jul 18 22:01:05 2015
Thread 1 cannot allocate new log, sequence 3948
Private strand flush not complete
Current log# 3 seq# 3947 mem# 0: /u08/app/oracle/oradata/*/redo03.log
Mon Jul 18 22:01:21 2015
Thread 1 advanced to log sequence 3948 (LGWR switch)
Current log# 4 seq# 3948 mem# 0: /u08/app/oracle/oradata/*/redo04.log
Mon Jul 18 22:02:08 2015
Non critical error ORA-00001 caught while writing to trace file "/u08/app/oracle/diag/rdbms/*/*/trace/*_dw00_12970.trc"
Error message:
Writing to the above trace file is disabled for now on...
Mon Jul 18 22:02:09 2015
opidcl aborting process unknown ospid (12861) as a result of ORA-9945
Mon Jul 18 22:02:09 2015
opidcl aborting process unknown ospid (12792) as a result of ORA-9945
Mon Jul 18 22:02:09 2015
Non critical error ORA-00001 caught while writing to trace file "/u08/app/oracle/diag/rdbms/*/*/trace/*_dm00_12966.trc"
Error message:
Writing to the above trace file is disabled for now on...
Mon Jul 18 22:02:10 2015
opidcl aborting process unknown ospid (12604) as a result of ORA-9945
Mon Jul 18 22:02:12 2015
opidcl aborting process unknown ospid (12639) as a result of ORA-9945
Mon Jul 18 22:02:13 2015
opidcl aborting process unknown ospid (12663) as a result of ORA-9945
Mon Jul 18 22:02:13 2015
opidcl aborting process unknown ospid (12691) as a result of ORA-9945
Mon Jul 18 22:02:13 2015
opidcl aborting process unknown ospid (12881) as a result of ORA-9945
Mon Jul 18 22:02:13 2015
Errors in file /u08/app/oracle/diag/rdbms/*/*/trace/*_pmon_11072.trc:
ORA-09817: Write to audit file failed.
Linux-x86_64 Error: 28: No space left on device
Additional information: 12
Errors in file /u08/app/oracle/diag/rdbms/*/*/trace/*_pmon_11072.trc:
ORA-09817: Write to audit file failed.
Linux-x86_64 Error: 28: No space left on device
Additional information: 12
Errors in file /u08/app/oracle/diag/rdbms/*/*/trace/*_pmon_11072.trc:
ORA-09817: Write to audit file failed.
Linux-x86_64 Error: 28: No space left on device
Additional information: 12
Errors in file /u08/app/oracle/diag/rdbms/*/*/trace/*_pmon_11072.trc:
ORA-09817: Write to audit file failed.Tue Jul 19 08:52:38 2015
操作系统 redhatlinux 5.8
数据库版本 oracle 11.2.0.1 单实例
状况信息:
由于本实例属于外围小系统并没有对其 文件系统,数据库实例等状态进行监控,故导致文件系统满了 导致实例挂起;
处理过程:
1. 经过查看alter 日志:如下文标红部分 文件系统无空间了,导致redolog 无法进行切换,导致实例挂起;
2. 由于出问题的时间为晚上10点左右,没有告警机制,导致到第二天早上过来后,清理文件系统,启动数据库实例后解决问题;
经验总结:
增加相关监控项
增加文件系统;
alter 告警日志如下:
VKRM started with pid=133, OS id=12948
Mon Jul 18 22:00:01 2015
Begin automatic SQL Tuning Advisor run for special tuning task "SYS_AUTO_SQL_TUNING_TASK"
Mon Jul 18 22:00:03 2015
DM00 started with pid=156, OS id=12966, job WAPAPP.SYS_EXPORT_SCHEMA_09
Mon Jul 18 22:00:04 2015
DW00 started with pid=158, OS id=12970, wid=1, job WAPAPP.SYS_EXPORT_SCHEMA_09
Mon Jul 18 22:00:27 2015
End automatic SQL Tuning Advisor run for special tuning task "SYS_AUTO_SQL_TUNING_TASK"
Mon Jul 18 22:01:05 2015
Thread 1 cannot allocate new log, sequence 3948
Private strand flush not complete
Current log# 3 seq# 3947 mem# 0: /u08/app/oracle/oradata/*/redo03.log
Mon Jul 18 22:01:21 2015
Thread 1 advanced to log sequence 3948 (LGWR switch)
Current log# 4 seq# 3948 mem# 0: /u08/app/oracle/oradata/*/redo04.log
Mon Jul 18 22:02:08 2015
Non critical error ORA-00001 caught while writing to trace file "/u08/app/oracle/diag/rdbms/*/*/trace/*_dw00_12970.trc"
Error message:
Writing to the above trace file is disabled for now on...
Mon Jul 18 22:02:09 2015
opidcl aborting process unknown ospid (12861) as a result of ORA-9945
Mon Jul 18 22:02:09 2015
opidcl aborting process unknown ospid (12792) as a result of ORA-9945
Mon Jul 18 22:02:09 2015
Non critical error ORA-00001 caught while writing to trace file "/u08/app/oracle/diag/rdbms/*/*/trace/*_dm00_12966.trc"
Error message:
Writing to the above trace file is disabled for now on...
Mon Jul 18 22:02:10 2015
opidcl aborting process unknown ospid (12604) as a result of ORA-9945
Mon Jul 18 22:02:12 2015
opidcl aborting process unknown ospid (12639) as a result of ORA-9945
Mon Jul 18 22:02:13 2015
opidcl aborting process unknown ospid (12663) as a result of ORA-9945
Mon Jul 18 22:02:13 2015
opidcl aborting process unknown ospid (12691) as a result of ORA-9945
Mon Jul 18 22:02:13 2015
opidcl aborting process unknown ospid (12881) as a result of ORA-9945
Mon Jul 18 22:02:13 2015
Errors in file /u08/app/oracle/diag/rdbms/*/*/trace/*_pmon_11072.trc:
ORA-09817: Write to audit file failed.
Linux-x86_64 Error: 28: No space left on device
Additional information: 12
Errors in file /u08/app/oracle/diag/rdbms/*/*/trace/*_pmon_11072.trc:
ORA-09817: Write to audit file failed.
Linux-x86_64 Error: 28: No space left on device
Additional information: 12
Errors in file /u08/app/oracle/diag/rdbms/*/*/trace/*_pmon_11072.trc:
ORA-09817: Write to audit file failed.
Linux-x86_64 Error: 28: No space left on device
Additional information: 12
Errors in file /u08/app/oracle/diag/rdbms/*/*/trace/*_pmon_11072.trc:
ORA-09817: Write to audit file failed.Tue Jul 19 08:52:38 2015
来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/25462274/viewspace-2122211/,如需转载,请注明出处,否则将追究法律责任。
转载于:http://blog.itpub.net/25462274/viewspace-2122211/