今天有个环境报rman备份失败,处理完紧急事情后,登录查看,发现报错如下
input archived log thread=1 sequence=811710 RECID=796334 STAMP=1137265897
channel d1: starting piece 1 at 02-JUN-23
released channel: d1
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on d1 channel at 06/02/2023 11:26:37
ORA-19599: block number 75960 is corrupt in archived log /fra/archivelog/1_811705_990835557.dbf
Recovery Manager complete.
orcl:/backup/orcl/log@db>
查看alert日志,发现报错
Fri Jun 02 15:39:01 2023
***
Corrupt block seq: 811705 blocknum=75960.
Bad header found during backing up archived log
Data in bad block - seq:0. bno:0. time:0
beg:0 cks:0
calculated check value: 0
Reread of seq=811705, blocknum=75960, file=/home/oracle/archivelog/1_811705_990835557.dbf, found same corrupt data
Reread of seq=811705, blocknum=75960, file=/home/oracle/archivelog/1_811705_990835557.dbf, found same corrupt data
Reread of seq=811705, blocknum=75960, file=/home/oracle/archivelog/1_811705_990835557.dbf, found same corrupt data
Reread of seq=811705, blocknum=75960, file=/home/oracle/archivelog/1_811705_990835557.dbf, found same corrupt data
Reread of seq=811705, blocknum=75960, file=/home/oracle/archivelog/1_811705_990835557.dbf, found same corrupt data
Fri Jun 02 15:40:38 2023
应该是归档日志文件坏块了,数据库每天有expdp全库导出的备份,这里就删除归档后重新做数据库完全备份
rman > delete force noprompt archivelog until time 'sysdate - 1/6';
20230805日更新
同样的问题又一次出现了,同样报错
Fri Aug 04 01:58:45 2023
***
Corrupt block seq: 890857 blocknum=312.
Bad header found during backing up archived log
Data in bad block - seq:0. bno:0. time:0
beg:0 cks:0
calculated check value: 0
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Fri Aug 04 11:28:15 2023
***
Corrupt block seq: 890857 blocknum=312.
Bad header found during backing up archived log
Data in bad block - seq:0. bno:0. time:0
beg:0 cks:0
calculated check value: 0
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Sat Aug 05 00:12:38 2023
***
Corrupt block seq: 890857 blocknum=312.
Bad header found during backing up archived log
Data in bad block - seq:0. bno:0. time:0
beg:0 cks:0
calculated check value: 0
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Sat Aug 05 00:32:21 2023
[root@newdb trace]# strings orcl_ora_2667.trc
Trace file /home/oracle/app/diag/rdbms/orcl/orcl/trace/orcl_ora_2667.trc
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
ORACLE_HOME = /home/oracle/app/oracle/product/11.2.0/dbhome_1
System name: Linux
Node name: newdb
Release: 4.1.12-61.1.28.el6uek.x86_64
Version: #2 SMP Thu Feb 23 20:03:53 PST 2017
Machine: x86_64
VM name: VMWare Version: 6
Instance name: orcl
Redo thread mounted by this instance: 1
Oracle process number: 58
Unix process pid: 2667, image: oracle@newdb (TNS V1-V3)
*** 2023-08-05 00:12:38.192
*** SESSION ID:(6430.22735) 2023-08-05 00:12:38.192
*** CLIENT ID:() 2023-08-05 00:12:38.192
*** SERVICE NAME:(SYS$USERS) 2023-08-05 00:12:38.192
*** MODULE NAME:(backup archivelog) 2023-08-05 00:12:38.192
*** ACTION NAME:(0000187 STARTED16) 2023-08-05 00:12:38.192
Corrupt block seq: 890857 blocknum=312.
Bad header found during backing up archived log
Data in bad block - seq:0. bno:0. time:0
beg:0 cks:0
calculated check value: 0
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
Reread of seq=890857, blocknum=312, file=/home/oracle/archivelog/1_890857_990835557.dbf, found same corrupt data
[root@newdb trace]#
mos文档:ORA-1578 ORA-353 ORA-19599 Corrupt blocks with zeros when filesystemio_options=SETALL on ext4 file system using Linux (Doc ID 1487957.1)
处理办法
1、查询系统,发现文件系统确实ext4格式
2、查看参数,filesystemio_options配置成SETALL
3、修改filesystemio_options,重启数据库
sql > ALTER SYSTEM SET filesystemio_options='NONE' SCOPE=SPFILE;