oracle 关闭数据库实列,10.2.0.4+ASM单实列驱除问题 - Oracle数据库管理 - Oracle数据库数据恢复、性能优化来问问AskMaclean - ParnassusData诗...

最新推荐文章于 2021-05-11 03:02:30 发布

杨良昆

最新推荐文章于 2021-05-11 03:02:30 发布

阅读量514

点赞数

文章标签： oracle 关闭数据库实列

HI：刘大各位好！

今天遇到一个问题ASM单实例被CSS驱除了，导致数据库宕机！

操作系统SESE10SP2，确定是单机CRSCTL CHECK CRS只有CSSD进程

PS -EF|GREP D.BIN 只有CSSD进程，数据库无RAC进程。OCRCHECK 是LOCAL，/ETC/ORACLE/OCR.LOC也是LOCAL=YES

1.数据库关键日志

Fri May 24 09:02:01 2013

Errors in file /home/oracle/product/admin/cqzjoa/bdump/cqzjoa_rbal_9310.trc:

ORA-15064: communication failure with ASM instance

Fri May 24 09:02:01 2013

Errors in file /home/oracle/product/admin/cqzjoa/bdump/cqzjoa_lgwr_9287.trc:

ORA-15064: communication failure with ASM instance

2.ASM关键日志

Fri May 24 09:02:00 2013

Errors in file /home/oracle/product/admin/+ASM/bdump/+asm_gmon_8999.trc:

ORA-29702: error occurred in Cluster Group Service operation

GMON: terminating instance due to error 29702

Instance terminated by GMON, pid = 8999

Fri May 24 09:34:34 2013

Starting ORACLE instance (normal)

3.cssd关键日志

[ CSSD]2012-11-19 09:04:23.559 [158511696] >TRACE: clssgmReconfigThread: completed for reconfig(1), with status(1)

[ CSSD]2013-05-24 08:56:39.790 [133345872] >WARNING: clssnmPollingThread: node cqzj-nw-db-01 (0) at 50 2.000000e+00artbeat fatal, eviction in 14.184 seconds

[ CSSD]2013-05-24 08:56:39.790 [133345872] >TRACE: clssnmPollingThread: node cqzj-nw-db-01 (0) is impending reconfig, flag 5, misstime 15816

[ CSSD]2013-05-24 08:56:39.790 [133345872] >TRACE: clssnmPollingThread: diskTimeout set to (27000)ms impending reconfig status(1)

[ CSSD]2013-05-24 08:56:46.717 [133345872] >WARNING: clssnmPollingThread: node cqzj-nw-db-01 (0) at 75 2.000000e+00artbeat fatal, eviction in 7.256 seconds

[ CSSD]2013-05-24 08:56:47.771 [133345872] >WARNING: clssnmPollingThread: node cqzj-nw-db-01 (0) at 75 2.000000e+00artbeat fatal, eviction in 6.204 seconds

[ CSSD]2013-05-24 08:56:51.125 [133345872] >WARNING: clssnmPollingThread: node cqzj-nw-db-01 (0) at 90 2.000000e+00artbeat fatal, eviction in 2.848 seconds

[ CSSD]2013-05-24 08:56:52.133 [133345872] >WARNING: clssnmPollingThread: node cqzj-nw-db-01 (0) at 90 2.000000e+00artbeat fatal, eviction in 1.840 seconds

[ CSSD]2013-05-24 08:56:53.140 [133345872] >WARNING: clssnmPollingThread: node cqzj-nw-db-01 (0) at 90 2.000000e+00artbeat fatal, eviction in 0.832 seconds

[ CSSD]2013-05-24 08:56:54.008 [133345872] >TRACE: clssnmPollingThread: Eviction started for node cqzj-nw-db-01 (0), flags 0x0005, state 3, wt4c 0

[ CSSD]2013-05-24 08:56:54.008 [133345872] >WARNING: clssnmPollingThread: state(5) clusterState(1) exit

[ CSSD]2013-05-24 08:56:54.008 [133345872] >ERROR: ###################################

[ CSSD]2013-05-24 08:56:54.008 [133345872] >ERROR: clssscExit: CSSD aborting

[ CSSD]2013-05-24 08:56:54.008 [133345872] >ERROR: ###################################

4.OS关键日志

ay 24 08:55:34 CQZJ-NW-DB-01 kernel: ping timeout of 5 secs expired, last rx 8312435384, last ping 8312436634, now 8312437884

May 24 08:55:34 CQZJ-NW-DB-01 kernel: connection2:0: iscsi: detected conn error (1011)

May 24 08:56:16 CQZJ-NW-DB-01 multipathd: sdm: directio checker reports path is down

May 24 08:56:16 CQZJ-NW-DB-01 multipathd: checker failed path 8:192 in map 3600140571ffa2ded4915d4063dbd55d1

May 24 08:56:18 CQZJ-NW-DB-01 kernel: device-mapper: multipath: Failing path 8:192.

May 24 09:00:07 CQZJ-NW-DB-01 multipathd: sdm: directio checker reports path is down

May 24 09:01:38 CQZJ-NW-DB-01 multipathd: sdm: directio checker reports path is down

May 24 09:01:51 CQZJ-NW-DB-01 iscsid: semop up failed 22

May 24 09:01:38 CQZJ-NW-DB-01 multipathd: sdm: directio checker reports path is down

May 24 09:01:55 CQZJ-NW-DB-01 kernel: oracle(43763): floating-point assist fault at ip 4000000007923d62, isr 0000020000007001