墨墨导读:一套19C CDB数据库,存储更换HBA卡宕,本文详述这起begin backup导致的故障恢复全过程。
半夜接到客户反馈,一套19C CDB数据库,存储更换HBA卡宕,起不来了,OPEN时提示需要介质恢复,这里截了一段ALERT LOG。
2020-07-28T23:40:53.328908+08:00
Errors in file /u02/app/oracle/diag/rdbms/racdb3/racdb32/trace/racdb32_ora_306493.trc:
ORA-10873: file 1 needs to be either taken out of backup mode or media recovered
ORA-01110: data file 1: '+DATA/RACDB3/DATAFILE/system.278.1037610503'
2020-07-28T23:40:53.387627+08:00
Errors in file /u02/app/oracle/diag/rdbms/racdb3/racdb32/trace/racdb32_ora_306493.trc:
ORA-10873: file 1 needs to be either taken out of backup mode or media recovered
ORA-01110: data file 1: '+DATA/RACDB3/DATAFILE/system.278.1037610503'
ORA-10873 signalled during: ALTER DATABASE OPEN /* db agent *//* {0:17:3557} */...
2020-07-28T23:40:55.357177+08:00
License high water mark = 2
2020-07-28T23:40:55.357492+08:00
USER(prelim) (ospid: 310054): terminating the instance
2020-07-28T23:40:56.369307+08:00
Instance terminated by USER(prelim), pid = 310054
ORA-10873: file 1 needs to be either taken out of backup mode or media recovered
这里报ORA-10873是由于数据库或表空间BEGIN BACKUP导致,正确的处理方法只需要end backup即可。
alter database end backup;
alter tablespace [tablespace_name] end backup;
alter database open;
[oracle@test ~]$ oerr ora 10873
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = “en_us.utf8”
are supported and installed on your system.
perl: warning: Falling back to the standard locale (“C”).
10873, 00000, “file %s needs to be either taken out of backup mode or media recovered”
// *Cause: An attempt was made to open a database after an instance failure or
// SHUTDOWN ABORT interrupted an online backup.
// *Action: If the indicated file is not a restored backup, then issue the
// ALTER DATABASE END BACKUP command and open the database. If the
// file is a restored online backup, then apply media recovery to
// it and open the database.
当时RECOVER DATABASE 提示找不到归档(需要6-18号的归档)
由于有存储相关操作,误以为其它原因导致的问题,没有关注该报错,查询vdatafile,vdatafile,vdatafile_header发现检查点为上个月的6-18号。
---当前日期为2020-07-28T23:40:56
SQL> col name for a10
SQL> select a.con_id,a.name,b.file#,b.rfile#,b.checkpoint_change#,b.checkpoint_time,b.status from v$containers a,v$datafile b where a.con_id=b.con_id order by checkpoint_change#;
CON_ID NAME FILE# RFILE# CHECKPOINT_CHANGE# CHECKPOINT_TIME STATUS
-------------------- ---------- -------------------- -------------------- -------------------- ------------------- -------
2 PDB$SEED 6 4 2336937 2020-04-13 09:26:28 ONLINE
2 PDB$SEED 5 1 2336937 2020-04-13 09:26:28 SYSTEM
2 PDB$SEED 8 9 2336937 2020-04-13 09:26:28 ONLINE
1 CDB$ROOT 178 178 8041849750453 2020-06-18 19:28:19 ONLINE
1 CDB$ROOT 7 7 8041849750453 2020-06-18 19:28:19 ONLINE
1 CDB$ROOT 9 9 8041849750453 2020-06-18 19:28:19 ONLINE
1 CDB$ROOT 4 4 8041849750453 2020-06-18 19:28:19 ONLINE
1 CDB$ROOT 3 3 8041849750453 2020-06-18 19:28:19 ONLINE
1 CDB$ROOT 1 1 8041849750453 2020-06-18 19:28:19 SYSTEM
接着检查日志及询问客户也没有做restore的操作,误判断为出现了异常,未找到解决办法,因为有最近的全备,