DG同步异常问题分析处理(归档日志文件损坏)

一.问题现象
主库归档日志齐全，但是备库无法接收并应用主库归档日志。
主库:
THREAD# Last Primary Seq Generated 
 ---------- -------------------------- 
 1 1884 
 2 1874 
 
备库应用： 
THREAD# Last Standby Seq Applied 
 ---------- ------------------------ 
 1 1503 
 2 1489 
 
SQL> select * from V$ARCHIVE_GAP; 
 THREAD# LOW_SEQUENCE# HIGH_SEQUENCE# CON_ID 
 ---------- ------------- -------------- ---------- 
 2 1490 1618 1 


二.计划将缺失归档手动发送到备库进行应用
1.拷贝主库归档至备库
asmcmd -p 
ASMCMD [+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24] > cp thread_2_seq_1490.318.976964987 /tmp/thread_2_seq_1490.318.976964987 
scp thread_2_seq_1490.318.976964987 oracle@racdg:/tmp/thread_2_seq_1490.318.976964987 

2.备库注册归档
SQL> alter database register logfile '/tmp/thread_2_seq_1490.318.976964987'; 
 alter database register logfile '/tmp/thread_2_seq_1490.318.976964987' 
 * 
 ERROR at line 1: 
 ORA-00317: file type 49049 in header is not log file 
 ORA-00334: archived log: '/tmp/thread_2_seq_1490.318.976964987' 

*初步判断为归档日志损坏


三.确认主库归档日志是否损坏
SQL> oradebug setmypid
Statement processed.
SQL> alter system dump logfile '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1490.318.976964987';
alter system dump logfile '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1490.318.976964987'
*
ERROR at line 1:
ORA-00317: file type 49049 in header is not log file
ORA-00334: archived log:
'+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1490.318.976964987'

SQL> oradebug tracefile_name
/oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_ora_118586.trc

*由此判断确实为归档日志发生损坏


四.进一步分析
查看主库alert日志，发现主库归档目录爆满，从而可能是导致归档日志损坏的原因。

点击(此处)折叠或打开

1. 2018-05-24T13:22:03.656143+08:00
2. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
3. ORA-00317: file type 49049 in header is not log file《==============================
4. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1490.318.976964987'
5. ORA-17502: ksfdcre:4 Failed to create file +DGRECOVER
6. ORA-15041: diskgroup "DGRECOVER" space exhausted<=============磁盘组空间已满，看起来这个损坏的归档日志应该与ASM磁盘组空间满了有关
7. ORA-17502: ksfdcre:4 Failed to create file +DGRECOVER
8. ORA-15041: diskgroup "DGRECOVER" space exhausted
9. 2018-05-24T13:22:25.524719+08:00
10. Error ORA-235 occurred during an un-locked control file transaction. This
11. error can be ignored. The control file transaction will be retried.
12. 2018-05-24T13:22:27.296438+08:00
13. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
14. ORA-00317: file type 49049 in header is not log file
15. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1490.318.976964987'
16. 2018-05-24T13:22:54.905226+08:00
17. Error ORA-235 occurred during an un-locked control file transaction. This
18. error can be ignored. The control file transaction will be retried.
19. 2018-05-24T13:22:56.888381+08:00
20. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
21. ORA-00317: file type 49049 in header is not log file
22. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1490.318.976964987'
23. 2018-05-24T13:23:19.777477+08:00
24. Error ORA-235 occurred during an un-locked control file transaction. This
25. error can be ignored. The control file transaction will be retried.
26. 2018-05-24T13:23:24.044757+08:00
27. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
28. ORA-00317: file type 49049 in header is not log file
29. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1490.318.976964987'
30. 2018-05-24T13:24:47.608317+08:00
31. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
32. ORA-00317: file type 8 in header is not log file
33. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1512.285.976966639'
34. 2018-05-24T13:24:48.364578+08:00
35. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
36. ORA-00317: file type 0 in header is not log file
37. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1502.300.976966159'
38. 2018-05-24T13:24:48.658017+08:00
39. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
40. ORA-00317: file type 15 in header is not log file
41. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1499.303.976965985'
42. 2018-05-24T13:24:53.282869+08:00
43. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
44. ORA-00317: file type 192 in header is not log file
45. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1495.308.976965677'
46. 2018-05-24T13:24:53.492761+08:00
47. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
48. ORA-00317: file type 49049 in header is not log file
49. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1490.318.976964987'
50. 2018-05-24T13:25:05.524094+08:00
51. Archived Log entry 5630 added for T-2.S-1624 ID 0xe5ac04db LAD:1
52. krse_arc_driver_core: Successful archiving of previously failed ORL
53. Unable to create archive log file '+DGRECOVER'
54. 2018-05-24T13:25:06.255375+08:00
55. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_arc0_41291.trc:
56. ORA-19504: failed to create file "+DGRECOVER"
57. ORA-17502: ksfdcre:4 Failed to create file +DGRECOVER
58. ORA-15041: diskgroup "DGRECOVER" space exhausted
59. ARC0: Error 19504 Creating archive log file to '+DGRECOVER'
60. 2018-05-24T13:25:10.697858+08:00
61. Error ORA-235 occurred during an un-locked control file transaction. This
62. error can be ignored. The control file transaction will be retried.
63. 2018-05-24T13:25:11.665704+08:00
64. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
65. ORA-00317: file type 8 in header is not log file
66. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1512.285.976966639'
67. 2018-05-24T13:25:13.282221+08:00
68. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
69. ORA-00317: file type 0 in header is not log file
70. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1502.300.976966159'
71. 2018-05-24T13:25:13.426907+08:00
72. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
73. ORA-00317: file type 15 in header is not log file
74. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1499.303.976965985'
75. 2018-05-24T13:25:13.563661+08:00
76. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
77. ORA-00317: file type 192 in header is not log file
78. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1495.308.976965677'
79. 2018-05-24T13:25:14.596499+08:00
80. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
81. ORA-00317: file type 49049 in header is not log file
82. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1490.318.976964987'
83. 2018-05-24T13:25:23.604762+08:00
84. Error ORA-235 occurred during an un-locked control file transaction. This
85. error can be ignored. The control file transaction will be retried.
86. 2018-05-24T13:25:24.220078+08:00
87. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
88. ORA-00317: file type 8 in header is not log file
89. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1512.285.976966639'
90. 2018-05-24T13:25:24.586326+08:00
91. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
92. ORA-00317: file type 0 in header is not log file
93. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1502.300.976966159'
94. 2018-05-24T13:25:24.996687+08:00
95. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
96. ORA-00317: file type 15 in header is not log file
97. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1499.303.976965985'
98. 2018-05-24T13:25:25.383332+08:00
99. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
100. ORA-00317: file type 192 in header is not log file
101. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1495.308.976965677'
102. 2018-05-24T13:25:26.017913+08:00
103. Errors in file /oracle/app/oracle/diag/rdbms/racdb12c/racdb12c2/trace/racdb12c2_tt00_41289.trc:
104. ORA-00317: file type 49049 in header is not log file
105. ORA-00334: archived log: '+DGRECOVER/RACDB12C/ARCHIVELOG/2018_05_24/thread_2_seq_1490.318.976964987'

五.问题解决
根据当前备库的SCN，在主库上进行增量备份，然后应用到备库上。
操作步骤，参考博客:
DG归档丢失场景下重新同步physical standby
http://blog.itpub.net/15412087/viewspace-2152589/

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/15412087/viewspace-2155141/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/15412087/viewspace-2155141/