oracle集群一台宕机,请教一下,RAC的同一个节点频繁因ORA-00227宕机

环境:CentOS6.5 + Oracle RAC 11.2.0.4.0(xxxdb1,xxxdb2两个实例)

问题介绍:实例xxxdb1节点多次因ORA-000227和ORA-00202导致宕机,宕机后通过srvctl命令可以重新启动实例。

RAC集群大部分业务只访问这个节点,控制文件current.257.1024919441是近期通过rman新生成的。

操作系统级别未发现有硬件的故障日志,另外一个实例xxxdb2运行正常。

请教一下,因为这个控制文件是刚刚备份后添加以及故障后实例可以正常启动,感觉集群的控制文件本身是没有问题的,应该怎么解决此问题?

Alter Log:

---------------------------------------------------------

Fri Nov 22 19:58:33 2019

Hex dump of (file 0, block 4) in trace file /u01/app/oracle/diag/rdbms/xxxdb/xxxdb1/trace/xxxdb1_lmon_79295.trc

Corrupt block relative dba: 0x00000004 (file 0, block 4)

Bad check value found during control file block read

Data in bad block:

type: 21 format: 2 rdba: 0x00000004

last change scn: 0x0000.00000000 seq: 0x1 flg: 0x04

spare1: 0x0 spare2: 0x0 spare3: 0x0

consistency value in tail: 0x00001501

check value in block header: 0x3b08

computed block checksum: 0x1

Control file block read for fixed table access has failed due to concurrent controlfile updates.

This is not a real data corruption. Please retry the query.

Errors in file /u01/app/oracle/diag/rdbms/xxxdb/xxxdb1/trace/xxxdb1_lmon_79295.trc:

ORA-00202: control file: '+CTL/xxxdb/controlfile/current.257.1024919441'

Errors in file /u01/app/oracle/diag/rdbms/xxxdb/xxxdb1/trace/xxxdb1_lmon_79295.trc  (incident=69689):

ORA-00227: corrupt block detected in control file: (block 4, # blocks 1)

ORA-00202: control file: '+CTL/xxxdb/controlfile/current.257.1024919441'

Incident details in: /u01/app/oracle/diag/rdbms/xxxdb/xxxdb1/incident/incdir_69689/xxxdb1_lmon_79295_i69689.trc

Errors in file /u01/app/oracle/diag/rdbms/xxxdb/xxxdb1/trace/xxxdb1_lmon_79295.trc:

ORA-00227: corrupt block detected in control file: (block 4, # blocks 1)

ORA-00202: control file: '+CTL/xxxdb/controlfile/current.257.1024919441'

LMON (ospid: 79295): terminating the instance due to error 227

xxxdb1_lmon_79295.trc:

-----------------------------------

***************** End of Resource Limits Dump ******************

----- Java Stack -----

----- VKTM Time Drifts Circular Buffer -----

Incident 69689 created, dump file: /u01/app/oracle/diag/rdbms/xxxdb/xxxdb1/incident/incdir_69689/xxxdb1_lmon_79295_i69689.trc

ORA-00227: corrupt block detected in control file: (block 4, # blocks 1)

ORA-00202: control file: '+CTL/xxxdb/controlfile/current.257.1024919441'

LMON caught an error 227 in the main loop

error 227 detected in background process

ORA-00227: corrupt block detected in control file: (block 4, # blocks 1)

ORA-00202: control file: '+CTL/xxxdb/controlfile/current.257.1024919441'

kjzduptcctx: Notifying DIAG for crash event

----- Abridged Call Stack Trace -----

ksedsts()+465

----- End of Abridged Call Stack Trace -----

*** 2019-11-22 19:58:35.165

LMON (ospid: 79295): terminating the instance due to error 227

ksuitm: waiting up to [5] seconds before killing DIAG(79285)

srvctl命令成功恢复实例:

----------------------------------------------------

srvctl start instance -d xxxdb -i xxxdb1

备份ASM上控制文件到本地磁盘后,DBV检测备份文件无坏块:

-----------------------------------------------------------------------------

ASMCMD> cp current.257.1024919441 /home/grid/controlfile_20191123_bak.ctl

copying +CTL/xxxdb/controlfile/current.257.1024919441 -> /home/grid/controlfile_20191123_bak.ctl

ASMCMD> exit

[grid@rac2 ~]$ dbfsize controlfile_20191123_bak.ctl

Database file: controlfile_20191123_bak.ctl

Database file type: file system

Database file size: 2384 16384 byte blocks

[grid@rac2 ~]$ dbv file=/home/grid/controlfile_20191123_bak.ctl blocksize=16384

DBVERIFY: Release 11.2.0.4.0 - Production on Sat Nov 23 16:19:12 2019

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

DBVERIFY - Verification starting : FILE = /home/grid/controlfile_20191123_bak.ctl

DBVERIFY - Verification complete

Total Pages Examined         : 2384

Total Pages Processed (Data) : 0

Total Pages Failing   (Data) : 0

Total Pages Processed (Index): 0

Total Pages Failing   (Index): 0

Total Pages Processed (Other): 1365

Total Pages Processed (Seg)  : 0

Total Pages Failing   (Seg)  : 0

Total Pages Empty            : 1019

Total Pages Marked Corrupt   : 0

Total Pages Influx           : 0

Total Pages Encrypted        : 0

Highest block SCN            : 1154908 (65535.1154908)

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值