11gr2 突然挂掉了!
系统症状:
32G的虚拟内存使用96%, 系统盘硬盘读写100%, 数据文件所在磁盘0%, CPU使用率1%.
在服务器上登录sqlplus都成问题了, 等待好长时间也无法登录进去。 遂紧急强制杀掉oracle smon, pmon 等系统进程, 等待了片刻, sqlplus登录成功, 重启数据库。
类似情况也遇到过一次, 对一个频繁访问的表的一个字段执行了update操作之后, 没有及时提交, 过了没有几分钟, 前台应用基本上全都查询不到数据了, 即使那些根本不需要访问这个表的应用也无法使用了。
查看系统中的现象, 跟这次一样。 提交之后, 过了十来分钟, 系统才恢复正常。
系统down机时间应该在15点左右, 查看alert日志, 截取如下信息片段:
Fri Dec 14 10:56:00 2012
Completed checkpoint up to RBA [0x3917.2.10], SCN: 6924009416
Fri Dec 14 10:57:18 2012
opiodr aborting process unknown ospid (41484394) as a result of ORA-609
Fri Dec 14 10:57:18 2012
opiodr aborting process unknown ospid (50463196) as a result of ORA-609
Fri Dec 14 10:57:18 2012
opiodr aborting process unknown ospid (43384992) as a result of ORA-609
Fri Dec 14 14:00:40 2012
Beginning log switch checkpoint up to RBA [0x391a.2.10], SCN: 6925017630
Thread 1 advanced to log sequence 14618 (LGWR switch)
Current log# 1 seq# 14618 mem# 0: /data/oradata/o10g/redo01a.log
Current log# 1 seq# 14618 mem# 1: /data_duplex/oradata/o10g/redo01c.log
Fri Dec 14 14:00:47 2012
Archived Log entry 15137 added for thread 1 sequence 14617 ID 0x9edfe58 dest 1:
Archived Log entry 15138 added for thread 1 sequence 14617 ID 0x9edfe58 dest 2:
Fri Dec 14 14:05:45 2012
Completed checkpoint up to RBA [0x391a.2.10], SCN: 6925017630
Fri Dec 14 14:55:45 2012
Time drift detected. Please check VKTM trace file for more details.
Fri Dec 14 14:57:07 2012
opiodr aborting process unknown ospid (12976608) as a result of ORA-609
Fri Dec 14 14:57:21 2012
opiodr aborting process unknown ospid (18481186) as a result of ORA-609
Fri Dec 14 14:57:30 2012
opiodr aborting process unknown ospid (8847846) as a result of ORA-609
Fri Dec 14 14:57:30 2012
opiodr aborting process unknown ospid (65994930) as a result of ORA-609
Fri Dec 14 14:57:27 2012
opiodr aborting process unknown ospid (45089018) as a result of ORA-609
ORA-00020: maximum number of processes (800) exceeded
ORA-20 errors will not be written to the alert log for
Fri Dec 14 15:12:07 2012
Fri Dec 14 15:10:13 2012
ORA-00020: maximum number of processes (800) exceeded
opiodr aborting process unknown ospid (24707344) as a result of ORA-609 ORA-20 errors will not be written to the alert log for
Fri Dec 14 15:13:27 2012
Fri Dec 14 15:13:27 2012
opiodr aborting process unknown ospid (2162960) as a result of ORA-609
opiodr aborting process unknown ospid (15335484) as a result of ORA-609
the next minute. Please look at trace files to see all
Fri Dec 14 15:12:31 2012
ORA-20 errors will not be written to the alert log for
ORA-20 errors will not be written to the alert log forFri Dec 14 15:12:07 2012
Fri Dec 14 15:13:21 2012
ORA-00020: maximum number of processes (800) exceeded
opiodr aborting process unknown ospid (48890100) as a result of ORA-609
帮忙大概判断一下, 谢谢!