ORA-600 17182诊断一例

昨天客户的数据库突然宕了，根据提供的告警日志，发现日志存在着不少的ORA-600错误。

Fri Mar 26 09:16:46 2010
Errors in file /oracle/PRD/xxx/trace/usertrace/orcl_ora_12877.trc:
ORA-00600: internal error code, arguments: [17182], [32438472]

发生错误时产生了大量的trace跟踪文件，其中有一个trace文件高达50G，这也是我目前遇到的最大跟踪文件。由于客户的归档日志目录和trace文件所在位置放置于同一块磁盘，大量跟踪文件的产生使得磁盘空间耗用殆尽，最终导致数据库归档失败而宕掉。

Fri Mar 26 15:09:25 2010
ARC0: Beginning to archive log# 6 seq# 50246
ARC0: Error 19504 creating archivelog file '/oracle/PRD/messys/oraarch/arch_1_50246.arc'
Fri Mar 26 15:09:25 2010
Errors in file /oracle/PRD/messys/trace/background/orcl_arc0_2519.trc:
ORA-00600: internal error code, arguments: [kcrrmario.1], [], [], [], [], [], [], []
*** 2010-03-26 15:09:25.859All Archive destinations made inactive
Fri Mar 26 15:09:26 2010
Errors in file /oracle/PRD/messys/trace/background/orcl_arc0_2519.trc:
ORA-00600: internal error code, arguments: [kcrrmario.1], [], [], [], [], [], [], []
ARC0: terminating instance due to error 600
Fri Mar 26 15:09:26 2010
Errors in file /oracle/PRD/messys/trace/background/orcl_lgwr_2382.trc:
ORA-00600: internal error code, arguments: [], [], [], [], [], [], [], []
Fri Mar 26 15:09:26 2010
Errors in file /oracle/PRD/messys/trace/background/orcl_pmon_2378.trc:
ORA-00600: internal error code, arguments: [], [], [], [], [], [], [], []
Fri Mar 26 15:09:26 2010
Errors in file /oracle/PRD/messys/trace/background/orcl_dbw0_2380.trc:
ORA-00600: internal error code, arguments: [], [], [], [], [], [], [], []
Instance terminated by ARC0, pid = 2519

根据Oracle对ORA-00600 17182的官方解释：

Oracle has detected that the magic number in a memory chunk header has been overwritten.
This is a heap (in memory) corruption and there is no underlying data corruption. The error may occur in the one of the process specific heaps (the Call heap, PGA heap, or session heap) or in the shared heap (SGA).

由于内存块头被覆盖导致错误的发生，这个错误可能发生在进程堆或者SGA共享堆。开始怀疑硬件故障，但是根据客户提供的操作系统explorer日志，没有硬件故障。

在发生600错误的是还有一些ORA-7445错误的发生，同时还会有临时表空间不足的现象出现。

Fri Mar 26 09:49:03 2010
ORA-1652: unable to extend temp segment by 257 in tablespace   TEMP
Fri Mar 26 09:49:04 2010
Errors in file /oracle/PRD/messys/trace/usertrace/orcl_ora_13963.trc:
ORA-00600: internal error code, arguments: [17182], [32438472], [], [], [], [], [], []
Fri Mar 26 09:49:11 2010
Errors in file /oracle/PRD/messys/trace/usertrace/orcl_ora_13963.trc:
ORA-07445: exception encountered: core dump [00000001010D6E2C] [SIGBUS] [Invalid address alignment] [1952805703] [] []
Fri Mar 26 09:49:12 2010
Errors in file /oracle/PRD/messys/trace/usertrace/orcl_ora_13963.trc:
ORA-07445: exception encountered: core dump [0000000100FB6898] [SIGSEGV] [Address not mapped to object] [36691968] [] []
ORA-07445: exception encountered: core dump [00000001010D6E2C] [SIGBUS] [Invalid address alignment] [1952805703] [] []
Fri Mar 26 09:49:14 2010
Errors in file /oracle/PRD/messys/trace/usertrace/orcl_ora_13963.trc:
ORA-07445: exception encountered: core dump [0000000100FB6898] [SIGSEGV] [Address not mapped to object] [36691968] [] []
ORA-07445: exception encountered: core dump [0000000100FB6898] [SIGSEGV] [Address not mapped to object] [36691968] [] []
ORA-07445: exception encountered: core dump [00000001010D6E2C] [SIGBUS] [Invalid address alignment] [1952805703] [] []

根据Oracle Bug 2698127的描述，主要表现为ORA-600: internal error code, arguments: [17182]，偶尔还会有ORA-7445现象的发生，同时还会导致临时表空间使用率暴增。根据用户提供的日志，以上的几点都比较吻合，因此将该次故障诊断为Bug 2698127，建议客户打上P2281320的补丁，再做后续的观察。