rman FUZZINESS SCN X$KCVFH

In Multi-tenant Database (Version 12.1.0.1.2)


(1)  Restore Database Preview reports  'RMAN-05119: recovery can not be done to a consistent state' although we have level 0/Full Database backup along with archivelogs backup to clear the Fuzziness

{ Output Truncated }
RMAN>
run {
ALLOCATE CHANNEL CH1 DEVICE TYPE sbt_tape parms 'SBT_LIBRARY=/u01/app/oracle/product/12.1.0/dbhome_1/lib/libopc.so,
ENV=(OPC_PFILE=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/opcConfig.ora)';
restore database preview;  }

allocated channel: CH1
channel CH1: SID=249 device type=SBT_TAPE
channel CH1: Oracle Database Backup Service Library VER=3.16.9.21

Starting restore at 14-FEB-17
Starting restore at 14-FEB-17

List of Backup Sets
==================

BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
20 Full 549.75M SBT_TAPE 06:37:35 13-FEB-17
BP Key: 20 Status: AVAILABLE Compressed: NO Tag: TAG20170212T201557
..
..

new media label is "<label>" for piece "<piece name>"
validation succeeded for backup piece
RMAN-05119: recovery can not be done to a consistent state.                                                                      <<<<<<<<<<<<<<<<<<<
Media recovery start SCN is 2819679
Recovery must be done beyond SCN 2836475 to clear datafile fuzziness
new media label is "<label>" for piece "<piece name>"
..
new media label is "<label>" for piece "<piece name>"
validation succeeded for backup piece
Finished restore at 14-FEB-17
released channel: CH1

(2)  Recover Database also errors out 'RMAN-06054: media recovery requesting unknown archived log for... starting SCN of xxxxx" during recovery

RMAN> recover database ;
allocated channel: CH1
channel CH1: SID=249 device type=SBT_TAPE
channel CH1: Oracle Database Backup Service Library VER=3.16.9.21

Starting recover at 14-FEB-17

starting media recovery
..
..

channel CH1: restored backup piece 1
channel CH1: restore complete, elapsed time: 00:00:01
archived log file name=<path>/arch/1_51_931000987.arc thread=1 sequence=51
unable to find archived log
archived log thread=1 sequence=52
released channel: CH1
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 02/14/2017 09:49:47
RMAN-06054: media recovery requesting unknown archived log for thread 1 with sequence n and starting SCN of xxxxx               <<<<<<<<<<<<<

RMAN>

CHANGES

Database upgraded to 12.1.0.1.2 and valid full database with archivelogs backup exists.

CAUSE

BUG 20315311:  RMAN-5119: RECOVERY CAN NOT BE DONE TO A CONSISTENT STATE  (Refer Note: 20315311.8)

SOLUTION

Error seems to be a wrong notification which misleads the recovery process.  Ignore the error (or apply the patch 20315311) and proceed with the 'alter database open' after the recover database.   

Sample output of the errors 

Assuming you have valid backup of database and archivelogs, Oracle recommends validating the integrity of the backup by running 'restore database preview'.

Due to a code defect, 'restore database preview' command returns the error RMAN-05119 although valid backups exist. 
 

For example:

RMAN>
run {
ALLOCATE CHANNEL CH1 DEVICE TYPE sbt_tape parms 'SBT_LIBRARY=/u01/app/oracle/product/12.1.0/dbhome_1/lib/libopc.so,
ENV=(OPC_PFILE=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/opcConfig.ora)';
restore database preview;}

allocated channel: CH1
channel CH1: SID=249 device type=SBT_TAPE
channel CH1: Oracle Database Backup Service Library VER=3.16.9.21

Starting restore at 14-FEB-17
Starting restore at 14-FEB-17

List of Backup Sets
==================

BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
20 Full 549.75M SBT_TAPE 06:37:35 13-FEB-17
BP Key: 20 Status: AVAILABLE Compressed: NO Tag: TAG20170212T201557
Handle: <piece name> Media: <name>
List of Datafiles in backup set 20
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
1 Full 2819679 12-FEB-17 <path>/data/cdb/system01.dbf

BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
21 Full 168.25M SBT_TAPE 01:25:44 13-FEB-17
BP Key: 21 Status: AVAILABLE Compressed: NO Tag: TAG20170212T201557
Handle: <piece name> Media: <name>
List of Datafiles in backup set 21
Container ID: 2, PDB Name: PDB$SEED
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
2 Full 1963812 24-OCT-14 <path>/data/pdb/SYSTEM01.DBF

BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
9 Full 340.25M SBT_TAPE 02:23:30 12-FEB-17
BP Key: 9 Status: AVAILABLE Compressed: NO Tag: TAG20170212T201557
Handle: <piece name> Media: <name>
List of Datafiles in backup set 9
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
3 Full 2819680 12-FEB-17 <path>/data/cdb/sysaux01.dbf
...
...
....

{ Output Truncated }

...
...

BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
17 Full 1.50M SBT_TAPE 00:00:36 13-FEB-17
BP Key: 17 Status: AVAILABLE Compressed: NO Tag: TAG20170212T201557
Handle: <piece name> Media: <name>
List of Datafiles in backup set 17
Container ID: 4, PDB Name: PDB2
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
15 Full 2814042 11-FEB-17 <path>/data/pdb2/DEFTBS01.DBF

BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
18 Full 1.25M SBT_TAPE 00:00:40 13-FEB-17
BP Key: 18 Status: AVAILABLE Compressed: NO Tag: TAG20170212T201557
Handle: <piece name> Media: <name>
List of Datafiles in backup set 18
Container ID: 4, PDB Name: PDB2
File LV Type Ckp SCN Ckp Time Name
---- -- ---- ---------- --------- ----
16 Full 2814042 11-FEB-17 <path>/data/pdb2/USERTBS01.DBF

List of Backup Sets
===================

BS Key Size Device Type Elapsed Time Completion Time
------- ---------- ----------- ------------ ---------------
24 10.00M SBT_TAPE 00:00:19 13-FEB-17
BP Key: 24 Status: AVAILABLE Compressed: NO Tag: TAG20170213T033311
Handle: <piece name> Media: <name>

   List of Archived Logs in backup set 24
   Thrd Seq Low SCN Low Time Next SCN Next Time
   ---- ------- ---------- --------- ---------- ---------
   1 45 2819303 12-FEB-17 2822492 12-FEB-17
   1 46 2822492 12-FEB-17 2825162 12-FEB-17

BS Key Size Device Type Elapsed Time Completion Time
------- ---------- ----------- ------------ ---------------
25 10.00M SBT_TAPE 00:00:21 13-FEB-17
BP Key: 25 Status: AVAILABLE Compressed: NO Tag: TAG20170213T033311
Handle: <piece name> Media: <name>
   List of Archived Logs in backup set 25
   Thrd Seq Low SCN Low Time Next SCN Next Time
   ---- ------- ---------- --------- ---------- ---------
   1 47 2825162 12-FEB-17 2828495 12-FEB-17
   1 48 2828495 12-FEB-17 2832292 13-FEB-17
BS Key Size Device Type Elapsed Time Completion Time
------- ---------- ----------- ------------ ---------------
23 10.00M SBT_TAPE 00:03:30 13-FEB-17
BP Key: 23 Status: AVAILABLE Compressed: NO Tag: TAG20170213T033311
Handle: <piece name> Media: <name>

   List of Archived Logs in backup set 23
   Thrd Seq Low SCN Low Time Next SCN Next Time
   ---- ------- ---------- --------- ---------- ---------
   1 49 2832292 13-FEB-17 2836038 13-FEB-17
   1 50 2836038 13-FEB-17 2838489 13-FEB-17                       <<<<<<<<<<<<<<<<< Archivelog 50 Available
 

BS Key Size Device Type Elapsed Time Completion Time
------- ---------- ----------- ------------ ---------------
22 256.00K SBT_TAPE 00:03:25 13-FEB-17

BP Key: 22 Status: AVAILABLE Compressed: NO Tag: TAG20170213T033311
Handle: <piece name> Media: <name>

   List of Archived Logs in backup set 22
   Thrd Seq Low SCN Low Time Next SCN Next Time
   ---- ------- ---------- --------- ---------- ---------
   1 51 2838489 13-FEB-17 2838634 13-FEB-17                                              
   new media label is "<label>" for piece "<piecename>"
...
...
new media label is "<label>" for piece "<piecename>"
validation succeeded for backup piece
RMAN-05119: recovery can not be done to a consistent state.                                                  <<<<<<<<<<<<<<<<< RMAN-05119 reported
Media recovery start SCN is 2819679
Recovery must be done beyond SCN 2836475 to clear datafile fuzziness
new media label is "<label>" for piece "<piecename>"
new media label is "<label>" for piece "<piecename>"


As one more level of check, check the FUZZINESS and FUZZINESS SCN  to clear the fuzziness by querying v$datafile_header and  x$kcvfh view.

SQL> select file#,substr(name,1,70) name, error, status,recover, fuzzy, resetlogs_change#, resetlogs_time,checkpoint_change#,checkpoint_time from v$datafile_header;

FILE# NAME                                      ERROR STATUS REC FUZ RESETLOGS_CHANGE# RESETLOGS_TIME CHECKPOINT_CHANGE# CHECKPOINT_TIME
---------- ----------------------------------------------- ----- ------- --- ---------- ----------------- ------------------- -----------------
1 <path name>/data/cdb/system01.dbf         ONLINE       YES    2746787        19-12-2016 11:23:07 2819679 12-02-2017 20:15:57                 <<<<<<<< Datafile 1 Fuzzy  
2 <path name>/data/pdb/SYSTEM01.DBF         ONLINE       NO           1         24-10-2014 15:21:02 1963812 24-10-2014 23:00:20
3 <path name>/data/cdb/sysaux01.dbf         ONLINE       YES    2746787        19-12-2016 11:23:07 2819680 12-02-2017 20:15:57                 <<<<<<<< Datafile 3 Fuzzy
4 <path name>/data/pdb/SYSAUX01.DBF         ONLINE       NO          1         24-10-2014 15:21:02 1963812 24-10-2014 23:00:20
5 <path name>/data/cdb/undotbs01.dbf        ONLINE       NO     2746787        19-12-2016 11:23:07 2819681 12-02-2017 20:15:57
6 <path name>/data/cdb/<name1>..dbf         ONLINE       NO     2746787        19-12-2016 11:23:07 2834618 13-02-2017 01:45:39
7 <path name>/data/pdb/<name1>..DBF         ONLINE       NO           1        24-10-2014 15:21:02 1963812 24-10-2014 23:00:20
8 <path name>/data/pdb/USERTBS01.DBF        ONLINE       NO          1         24-10-2014 15:21:02 1963812 24-10-2014 23:00:20
9 <path name>/data/pdb1/SYSTEM01.DBF        ONLINE       NO     2746787        19-12-2016 11:23:07 2813870 11-02-2017 14:09:09
10 <path name>/data/pdb1/SYSAUX01.DBF       ONLINE       NO     2746787        19-12-2016 11:23:07 2813870 11-02-2017 14:09:09
11 <path name>/data/pdb1/<name1>.DBF       ONLINE       NO     2746787        19-12-2016 11:23:07 2813870 11-02-2017 14:09:09
12 <path name>/data/pdb1/USERTBS01.DBF      ONLINE       NO     2746787        19-12-2016 11:23:07 2813870 11-02-2017 14:09:09
13 <path name>/data/pdb2/SYSTEM01.DBF       ONLINE       NO     2746787        19-12-2016 11:23:07 2814042 11-02-2017 14:09:10
14 <path name>/data/pdb2/SYSAUX01.DBF       ONLINE       NO     2746787        19-12-2016 11:23:07 2814042 11-02-2017 14:09:10
15 <path name>/data/pdb2/<name1>..DBF       ONLINE       NO     2746787        19-12-2016 11:23:07 2814042 11-02-2017 14:09:10
16 <path name>/data/pdb2/USERTBS01.DBF      ONLINE       NO     2746787        19-12-2016 11:23:07 2814042 11-02-2017 14:09:10

--------------------X$KCVFH;

At least one datafile has scn much lower than the rest of the database:

SQL>select distinct to_char(fhscn, '99999999999999999999') as "Stby_datafile_hdr_SCN" from X$KCVFH;

Stby_datafile_hdr_SCN
---------------------
  10571110002345
  10591313115628
  10591313133587
  10591313148383
  10591313156525
  10591313172772
  10591313186729

SQL>select min(to_char(fhscn, '99999999999999999999')) as Standby_Datafile_Header_SCN from X$KCVFH;

STANDBY_DATAFILE_HEAD
---------------------
  10571110002345
 
x$kcbfh.fhscn gives checkpoint scn for a particular datafile. Datafile recovery should start with this checkpoint scn.
If scn is too low, the redo containing this could be found in very old archive redo log which may already been deleted.
 

SOLUTION

1. Identify the datafile with lowest checkpoint scn:
  a. at standby:
  SQL>col name format a55
  SQL>select file#, name, to_char(checkpoint_change#, '99999999999999999999') from v$datafile_header where checkpoint_change# < 10571110002350;

  FILE# NAME TO_CHAR(CHECKPOINT_CH
  ---------- ------------------------------------------------------- ---------------------
  299 +<path>/datafile/sysaux.622.811623929 10571110002345

  b. at primary:
  SQL > select file#, name from v$datafile where file# = 299

2. Copy datafile identified in step 1 from primary to standby database as per instruction in:
  Note 605234.1 == > How to Copy ASM datafile From Primary Database to Standby Database on ASM using RMAN

3. Recreate standby control file from current control file at primary.
  Note 734862.1 == > Step by step guide on how to recreate standby control file when datafiles are on ASM and using Oracle Managed Files

4. Restart standby recovery.
  sql> alter database recover managed standby database disconnect;

---------------------------

SQL> select FHDBI,FHDBN,HXFIL File_num,substr(HXFNM,1,70) File_name,FHTYP Type,FHSCN SCN, FHTNM TBS_NAME,FHSTA status,FHRBA_SEQ Sequence,fhafs fuzzy_scn from X$KCVFH;
 

FHDBI      FHDBN FILE_NUM FILE_NAME                              TYPE SCN     TBS_NAME      STATUS     SEQUENCE          FUZZY_SCN
---------- --------- ---------- --------------------------------------------- ---------- ---------- ---------------- ------------------- ---------- ----------------  ---------------
3004172638 DBCDB 1  <path name>/data/cdb/system01.dbf    3   2819679 SYSTEM          8256          45            2836475     <<<<<<<<<<<<<<<  Fuzziness SCN to be cleared----
3004172638 DBCDB 2  <path name>/data/pdb/SYSTEM01.DBF    3   1963812 SYSTEM              0      1868                 0
3004172638 DBCDB 3  <path name>/data/cdb/sysaux01.dbf    3   2819680 SYSAUX          64            45           2824670
3004172638 DBCDB 4  <path name>/data/pdb/SYSAUX01.DBF    3   1963812 SYSAUX            0        1868                 0
3004172638 DBCDB 5  <path name>/data/cdb/undotbs01.dbf   3   2819681 UNDOTBS1          0           45                0
3004172638 DBCDB 6  <path name>/data/cdb/<name1>.dbf    3   2834618  <ts name>            0           49                0
3004172638 DBCDB 7  <path name>/data/pdb/<name1>.DBF    3   1963812 <ts name>            0         1868                0
3004172638 DBCDB 8  <path name>/data/pdb/USERTBS01.DBF   3   1963812 USERTBS           0         1868                0
3004172638 DBCDB 9  <path name>/data/pdb1/SYSTEM01.DBF   3   2813870 SYSTEM            0           43                0
3004172638 DBCDB 10 <path name>/data/pdb1/SYSAUX01.DBF   3   2813870 SYSAUX            0           43                0
3004172638 DBCDB 11 <path name>/data/pdb1/<name1>.DBF   3   2813870 <ts name>             0           43                0
3004172638 DBCDB 12 <path name>/data/pdb1/USERTBS01.DBF  3   2813870 USERTBS           0           43                0
3004172638 DBCDB 13 <path name>/data/pdb2/SYSTEM01.DBF   3   2814042 SYSTEM            0           43                0
3004172638 DBCDB 14 <path name>/data/pdb2/SYSAUX01.DBF   3   2814042 SYSAUX            0           43                0
3004172638 DBCDB 15 <path name>/data/pdb2/<name1>.DBF   3   2814042 <ts name>             0           43                0
3004172638 DBCDB 16 <path name>/data/pdb2/USERTBS01.DBF  3   2814042 USERTBS           0           43                0




As per x$kcvfh to clear the fuzziness , recovery should go beyond "2836475".  The same is suggested in RMAN's 'restore database preview'.  I.e:
"Recovery must be done beyond SCN 2836475 to clear datafile fuzziness"

To find which archivelog sequence has scn "2836475" information, run the sql below: 
 

SQL> select thread#,dest_id,sequence#, name,applied ,first_time, first_change#,status ,deleted
from gv$archived_log where 2836475 between first_change# and next_change#;

THREAD# DEST_ID SEQUENCE# NAME                                          APPLIED    FIRST_TIM FIRST_CHANGE# S DEL 
-------- ----- ------- ------------------------------------------- ---------    --------- ------------- - --- 
1           1   50       <path>/arch/1_50_ NO931000987.arc 13-FEB-17  2836038                 A NO         


From the above output, we understand, to clear the fuzziness and open the database, we need to apply the archivelog sequence 50 which has scn 2836475. 

Now, we proceed with RECOVER DATABASE. 

In this example: 
As per expectation, RMAN should recover the database until the archivelogs which known to controlfile . i.e untill archivelog sequence 50 (51-1) RMAN should apply to clear the fuzziness and RMAN should end its recovery. 

But RMAN expecting archivelog sequence 52 which does not exist in the backup and actually NOT needed for recovery.

RMAN> run {
ALLOCATE CHANNEL CH1 DEVICE TYPE sbt_tape parms 'SBT_LIBRARY=/u01/app/oracle/product/12.1.0/dbhome_1/lib/libopc.so,
ENV=(OPC_PFILE=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/opcConfig.ora)';
recover database;
}
allocated channel: CH1
channel CH1: SID=249 device type=SBT_TAPE
channel CH1: Oracle Database Backup Service Library VER=3.16.9.21

Starting recover at 14-FEB-17

starting media recovery

new media label is "LABEL" for piece "<piece name >"
new media label is "LABEL" for piece "<piece name >"
..
..
channel CH1: restoring archived log
archived log thread=1 sequence=45
channel CH1: restoring archived log
archived log thread=1 sequence=46
channel CH1: reading from backup piece <piece name >
channel CH1: piece handle=<piece name > tag=TAG20170213T033311
channel CH1: restored backup piece 1
channel CH1: restore complete, elapsed time: 00:00:01
archived log file name=<path>/arch/1_45_931000987.arc thread=1 sequence=45
archived log file name=<path>/arch/1_46_931000987.arc thread=1 sequence=46
channel CH1: starting archived log restore to default destination
channel CH1: restoring archived log
archived log thread=1 sequence=47
channel CH1: restoring archived log
archived log thread=1 sequence=48
channel CH1: reading from backup piece <piece name >
channel CH1: piece handle=<piece name > tag=TAG20170213T033311
channel CH1: restored backup piece 1
channel CH1: restore complete, elapsed time: 00:00:01
archived log file name=<path>/arch/1_47_931000987.arc thread=1 sequence=47
archived log file name=<path>/arch/1_48_931000987.arc thread=1 sequence=48
channel CH1: starting archived log restore to default destination
channel CH1: restoring archived log
archived log thread=1 sequence=49
channel CH1: restoring archived log
archived log thread=1 sequence=50
channel CH1: reading from backup piece 0nrsem3o_1_1
channel CH1: piece handle=0nrsem3o_1_1 tag=TAG20170213T033311
channel CH1: restored backup piece 1
channel CH1: restore complete, elapsed time: 00:00:03
archived log file name=<path>/arch/1_49_931000987.arc thread=1 sequence=49
archived log file name=<path>/arch/1_50_931000987.arc thread=1 sequence=50
channel CH1: starting archived log restore to default destination
channel CH1: restoring archived log
archived log thread=1 sequence=51
channel CH1: reading from backup piece 0prsem3o_1_1
channel CH1: piece handle=0prsem3o_1_1 tag=TAG20170213T033311
channel CH1: restored backup piece 1
channel CH1: restore complete, elapsed time: 00:00:01
archived log file name=<path>/arch/1_51_931000987.arc thread=1 sequence=51
unable to find archived log
archived log thread=1 sequence=52
released channel: CH1
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 02/14/2017 09:49:47
RMAN-06054: media recovery requesting unknown archived log for thread 1 with sequence 52 and starting SCN of 2838634                <<<<<<<<<<<<<

RMAN>


Ignore the above RMAN-06054 error and proceed to open the database with resetlogs option.   If the source and target databases' online redo file location is different, rename all the redo log files using 'alter database rename...' and proceed with the open as below:

     SQL> alter database rename file '<old path/redo01.log'> to '<new path/redo01.log'>;

     SQL> alter database open resetlogs;

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值