oracle support一次危机处理（之一）

天有不测风云，

IBM 的中端磁盘阵列这次又惹祸了。在微码升级的时候，

DS4800 发生故障。IBM工程师把阵列故障恢复后，拍拍屁股走人，留下了阵列上不一致的数据。

没办法，阵列上的数据又得我们来想办法抢救了。。。。

还好有oracle support的支持我们最终化解了这次危机。下文详细介绍了整个恢复的过程。

 

 

oracle工程师到达现场后，根据错误信息，

 

问题情况如下SR信息所示：（HP-UX PA-RISC (64-bit)+ORACLE 9.2.0.8.0.+filesystem+no archivelog mode+no backup)

** Customer's Management 24x7 contact name:***************

** Customer's Management 24x7 contact number: ****************

** Primary Customer contact name: *******************

** Current Customer 24x7 voice phone number: ********************

** Current Customer E-mail address: ********************

** Current Customer Pager/Fax number: *****************

### Characteristics of Problem ###

This is a critical production system

Database is down and will not start

Data corruption issue and media recovery failed to fix the problem

 

### Note:28814.1 did not help because: ###

It did not have the needed information.

### Other: Note:28814.1 did not help because: ###

Sys owned objects corrupted

### The corruption was detected on: ###

Data file.

### Other: The corruption was detected on: ###

system tablespace

### The corruption affected a segment of type: ###

Table.

### Other: The corruption affected a segment of type: ###

TRANSACTION MANAGED DATA BLOCK

### The affected segment belongs to: ###

Database Dictionary.

### Other: The affected segment belongs to: ###

Database dictionary

### What error(s) and message(s) are Oracle reporting? ###

09:52:24 SQL> recover database allow 1 corruption;

ORA-00283: recovery session canceled due to errors

ORA-10562: Error occurred while applying redo to data block (file# 1, block# 1490)

ORA-10564: tablespace SYSTEM

ORA-01110: data file 1: '/oradata/yincang/rlv_system'

ORA-10561: block type 'TRANSACTION MANAGED DATA BLOCK', data object# 201

ORA-00600: internal error code, arguments: [kcoapl_blkchk], [1], [1490], [6101], [], [], [], []

### If Oracle is reporting a datablock corruption: ###

none

### When was the last known good backup taken? ###

No backup avaiable

### What type of backup is it? ###

Export.

### Which tool, if any, was used to perform the backup? ###

Operating System.

### Is this database running on Archivelog mode? ###

No.

### If yes, are all archivelogs available since last known good backup? ###

n/a

### Is this database using Raw device or File System? ###

File system

### Can we use Oracle Collaborative Suite (OCS) to diagnose this issue? ###

 

 

因为存储的突然闪断，导致数据库无法启动：

Oracle 专家到达现场后，尝试启动数据库，看到了如下的错误：

SQL> alter database open;

alter database open

*

ERROR at line 1:

ORA-01122: database file 41 failed verification check

ORA-01110: data file 41: '/oradata/yincang/nircnircdata11'

ORA-01207: file is more recent than controlfile - old controlfile

于是重建控制文件，并尝试用控制文件来恢复并打开数据库，报告如下错误：

SQL> recover database allow 1 corruption;

ORA-00283: recovery session canceled due to errors

ORA-10562: Error occurred while applying redo to data block (file# 1, block# 1490)

ORA-10564: tablespace SYSTEM

ORA-01110: data file 1: '/oradata/yincang/rlv_system'

ORA-10561: block type 'TRANSACTION MANAGED DATA BLOCK', data object# 201

ORA-00600: internal error code, arguments: [kcoapl_blkchk], [1], [1490], [6101], [], [], [], []