ORA-600(1881)错误-CSDN博客

客户的数据库出现了这个错误，错误是由于硬件问题所致。

详细的错误信息为：

Fri Jul 15 02:31:52 CST 2011
WARNING: failed to read mirror side 1 of virtual extent 20 logical extent 0 of file 659 in group 2 from disk 0 allocation unit 674791264; if possible, will try another mirror side
WARNING: failed to write mirror side 1 of virtual extent 20 of file 659 in group 2
WARNING: IO Failed. au:25396 diskname:/dev/rac/fradg01
rq:ffffffff792d7c48 buffer:ffffffff7a07f000 au_offset(bytes):512 iosz:1048064 operation:0
status:2
Fri Jul 15 02:31:52 CST 2011
Errors in file /oracle/admin/xjboss/bdump/xjboss2_arc0_14247.trc:
ORA-19501: read error on file "+FRADG/xjboss/archivelog/2011_07_15/thread_2_seq_1708.659.756527491", blockno 40961 (blocksize=512)
ORA-27063: number of bytes read/written is incorrect
Fri Jul 15 02:31:52 CST 2011
Errors in file /oracle/admin/xjboss/bdump/xjboss2_arc0_14247.trc:
ORA-19501: read error on file "+FRADG/xjboss/archivelog/2011_07_15/thread_2_seq_1708.659.756527491", blockno 40961 (blocksize=512)
ORA-27063: number of bytes read/written is incorrect
ARCH: FAL archive failed. Archiver continuing
Fri Jul 15 02:31:52 CST 2011
ORACLE Instance xjboss2 - Archival Error. Archiver continuing.
FAL[server, ARC0]: FAL archive failed, see trace file.
Fri Jul 15 02:33:36 CST 2011
Thread 2 advanced to log sequence 1710 (LGWR switch)
Current log# 6 seq# 1710 mem# 0: +DATADG/xjboss/onlinelog/group_6.351.748589869
Current log# 6 seq# 1710 mem# 1: +FRADG/xjboss/onlinelog/group_6.262.748589871
Fri Jul 15 02:33:37 CST 2011
Errors in file /oracle/admin/xjboss/bdump/xjboss2_arc0_14247.trc:
ORA-00600: internal error code, arguments: [1881], [0xFFFFFFFF7B42DEE0], [0x106A96878], [], [], [], [], []
Fri Jul 15 02:33:38 CST 2011
Trace dumping is performing id=[cdmp_20110715023338]
Fri Jul 15 02:33:38 CST 2011
Errors in file /oracle/admin/xjboss/bdump/xjboss2_arc0_14247.trc:
ORA-00600: internal error code, arguments: [1881], [0xFFFFFFFF7B42DEE0], [0x106A96878], [], [], [], [], []
Fri Jul 15 02:33:38 CST 2011
Errors in file /oracle/admin/xjboss/bdump/xjboss2_arc0_14247.trc:
ORA-00600: internal error code, arguments: [1881], [0xFFFFFFFF7B42DEE0], [0x106A96878], [], [], [], [], []
Fri Jul 15 02:33:38 CST 2011
Errors in file /oracle/admin/xjboss/bdump/xjboss2_arc0_14247.trc:
ORA-00600: internal error code, arguments: [1881], [0xFFFFFFFF7B42DEE0], [0x106A96878], [], [], [], [], []
Fri Jul 15 02:33:38 CST 2011
Errors in file /oracle/admin/xjboss/bdump/xjboss2_arc0_14247.trc:
ORA-00600: internal error code, arguments: [1881], [0xFFFFFFFF7B42DEE0], [0x106A96878], [], [], [], [], []
Fri Jul 15 02:33:38 CST 2011
Errors in file /oracle/admin/xjboss/bdump/xjboss2_arc0_14247.trc:
ORA-00600: internal error code, arguments: [1881], [0xFFFFFFFF7B42DEE0], [0x106A96878], [], [], [], [], []

通过这个ORA-600错误在metalink上找不到有价值的信息。虽然这里最明显的错误信息是ORA-600(1881)错误，但是这个错误只是其他错误所引发的现象而已，重要的错误信息实际上是ORA-19501和ORA-27063。

根据ORA-19501错误，Oracle在读取ASM上的文件时出现了IO相关的错误，一般来说在读取文件时产生的错误比较少见。

检查ASM对应的日志，并未发现任何错误信息。

再次检查告警日志，发现了更为关键的信息，不过这些信息是以WARNING方式出现在告警日志文件中。这些信息明确指出，在读和写磁盘镜像时出现了错误，而且裸设备/dev/rac/fradg01也出现了IO错误。很显然出现错误的这个设备，就是后面出现读错误的ASM磁盘对应的裸设备。

检查这个时间段对应的操作系统日志，果然发现了对应的硬件错误信息：

Jul 15 03:12:00 xjboss-db2 scsi: [ID 243001 kern.warning] WARNING: /scsi_vhci (scsi_vhci0):
Jul 15 03:12:00 xjboss-db2 /scsi_vhci/ssd@g6000b5d0006a0000006a10c100040000 (ssd10): Command Timeout on path /pci@1,700000/QLGC,qlc@0/fp@0,0 (fp2)
Jul 15 03:12:00 xjboss-db2 scsi: [ID 107833 kern.warning] WARNING: /scsi_vhci/ssd@g6000b5d0006a0000006a10c100040000 (ssd10):
Jul 15 03:12:00 xjboss-db2 SCSI transport failed: reason 'timeout': retrying command
Jul 15 03:12:01 xjboss-db2 scsi: [ID 107833 kern.warning] WARNING: /scsi_vhci/ssd@g6000b5d0006a0000006a10c100040000 (ssd10):
Jul 15 03:12:01 xjboss-db2 SCSI transport failed: reason 'tran_err': retrying command
Jul 15 03:13:32 xjboss-db2 scsi: [ID 107833 kern.warning] WARNING: /scsi_vhci/ssd@g6000b5d0006a0000006a10c100040000 (ssd10):
Jul 15 03:13:32 xjboss-db2 Error for Command: write(10) Error Level: Retryable
Jul 15 03:13:32 xjboss-db2 scsi: [ID 107833 kern.notice] Requested Block: 376276993 Error Block: 376276993
Jul 15 03:13:32 xjboss-db2 scsi: [ID 107833 kern.notice] Vendor: FUJITSU Serial Number: 6A10C10004
Jul 15 03:13:32 xjboss-db2 scsi: [ID 107833 kern.notice] Sense Key: Aborted Command
Jul 15 03:13:32 xjboss-db2 scsi: [ID 107833 kern.notice] ASC: 0xc0 (), ASCQ: 0x0, FRU: 0x10
Jul 15 03:13:52 xjboss-db2 scsi: [ID 107833 kern.warning] WARNING: /scsi_vhci/ssd@g6000b5d0006a0000006a10c100040000 (ssd10):
Jul 15 03:13:52 xjboss-db2 Error for Command: write(10) Error Level: Retryable
Jul 15 03:13:52 xjboss-db2 scsi: [ID 107833 kern.notice] Requested Block: 378120193 Error Block: 378120193
Jul 15 03:13:52 xjboss-db2 scsi: [ID 107833 kern.notice] Vendor: FUJITSU Serial Number: 6A10C10004
Jul 15 03:13:52 xjboss-db2 scsi: [ID 107833 kern.notice] Sense Key: Aborted Command
Jul 15 03:13:52 xjboss-db2 scsi: [ID 107833 kern.notice] ASC: 0xc0 (), ASCQ: 0x0, FRU: 0x10
Jul 15 03:13:52 xjboss-db2 scsi: [ID 107833 kern.warning] WARNING: /scsi_vhci/ssd@g6000b5d0006a0000006a10c100040000 (ssd10):
Jul 15 03:13:52 xjboss-db2 Error for Command: write(10) Error Level: Retryable
Jul 15 03:13:52 xjboss-db2 scsi: [ID 107833 kern.notice] Requested Block: 378128385 Error Block: 378128385
Jul 15 03:13:52 xjboss-db2 scsi: [ID 107833 kern.notice] Vendor: FUJITSU Serial Number: 6A10C10004
Jul 15 03:13:52 xjboss-db2 scsi: [ID 107833 kern.notice] Sense Key: Aborted Command
Jul 15 03:13:52 xjboss-db2 scsi: [ID 107833 kern.notice] ASC: 0xc0 (), ASCQ: 0x0, FRU: 0x10
Jul 15 03:16:26 xjboss-db2 sendmail[27712]: [ID 702911 mail.crit] My unqualified host name (xjboss-db2) unknown; sleeping for retry
Jul 15 03:17:26 xjboss-db2 sendmail[27712]: [ID 702911 mail.alert] unable to qualify my own domain name (xjboss-db2) -- using short name
Jul 15 09:20:33 xjboss-db2 scsi: [ID 243001 kern.warning] WARNING: /scsi_vhci (scsi_vhci0):
Jul 15 09:20:33 xjboss-db2 /scsi_vhci/ssd@g6000b5d0006a0000006a10c100040000 (ssd10): Command Timeout on path /pci@1,700000/QLGC,qlc@0/fp@0,0 (fp2)
Jul 15 09:20:33 xjboss-db2 scsi: [ID 107833 kern.warning] WARNING: /scsi_vhci/ssd@g6000b5d0006a0000006a10c100040000 (ssd10):