cssd拉起oracle,升级补丁后cssd无法启动 - Oracle数据库管理 - Oracle数据库数据恢复、性能优化来问问AskMaclean - ParnassusData诗檀软件旗下网站 ...

环境为2节点rac  版本11.2.0.3.0   操作系统 aix 6100-08-00

升级第一节点至11.2.0.3.6升级完成后，在最后执行

/u01/app/11.2.0/grid/crs/install/rootcrs.pl -patch : run as root

时报错

crsctl stat res -t -init

NAME           TARGET  STATE        SERVER                   STATE_DETAILS

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.asm

1        ONLINE  OFFLINE                               Instance Shutdown

ora.cluster_interconnect.haip

1        ONLINE  OFFLINE

ora.crf

1        ONLINE  ONLINE       rac1

ora.crsd

1        ONLINE  OFFLINE

ora.cssd

1        ONLINE  OFFLINE                               STARTING

ora.cssdmonitor

1        ONLINE  ONLINE       rac1

ora.ctssd

1        ONLINE  OFFLINE

ora.diskmon

1        OFFLINE OFFLINE

ora.drivers.acfs

1        ONLINE  OFFLINE

ora.evmd

1        ONLINE  OFFLINE

ora.gipcd

1        ONLINE  ONLINE       rac1

ora.gpnpd

1        ONLINE  ONLINE       rac1

ora.mdnsd

1        ONLINE  ONLINE       rac1

发现cssd始终显示starting

查看cssd日志

2013-05-07 14:57:16.695: [    CSSD][2577]clssnmvDHBValidateNcopy: node 2, rac2, has a disk HB, but no network HB, DHB has rcfg 262286478, wrtcnt, 120834, LATS 18014328, lastSeqNo 120833, uniqueness 1367900300, timestamp 1367909836/18014113

2013-05-07 14:57:17.323: [    CSSD][3348]clssgmWaitOnEventValue: after CmInfo State  val 3, eval 1 waited 0

2013-05-07 14:57:17.708: [    CSSD][2577]clssnmvDHBValidateNcopy: node 2, rac2, has a disk HB, but no network HB, DHB has rcfg 262286478, wrtcnt, 120835, LATS 18015340, lastSeqNo 120834, uniqueness 1367900300, timestamp 1367909837/18015117

2013-05-07 14:57:18.034: [    CSSD][3862]clssnmSendingThread: sending join msg to all nodes

2013-05-07 14:57:18.034: [    CSSD][3862]clssnmSendingThread: sent 4 join msgs to all nodes

2013-05-07 14:57:18.323: [    CSSD][3348]clssgmWaitOnEventValue: after CmInfo State  val 3, eval 1 waited 0

2013-05-07 14:57:18.728: [    CSSD][2577]clssnmvDHBValidateNcopy: node 2, rac2, has a disk HB, but no network HB, DHB has rcfg 262286478, wrtcnt, 120836, LATS 18016360, lastSeqNo 120835, uniqueness 1367900300, timestamp 1367909838/18016121

2013-05-07 14:57:19.331: [    CSSD][3348]clssgmWaitOnEventValue: after CmInfo State  val 3, eval 1 waited 0

2013-05-07 14:57:19.737: [    CSSD][2577]clssnmvDHBValidateNcopy: node 2, rac2, has a disk HB, but no network HB, DHB has rcfg 262286478, wrtcnt, 120837, LATS 18017369, lastSeqNo 120836, uniqueness 1367900300, timestamp 1367909839/18017126

2013-05-07 14:57:20.332: [    CSSD][3348]clssgmWaitOnEventValue: after CmInfo State  val 3, eval 1 waited 0

2013-05-07 14:57:20.644: [    CSSD][1029]clssscSelect: cookie accept request 110977d00

2013-05-07 14:57:20.644: [    CSSD][1029]clssgmAllocProc: (111b16e30) allocated

2013-05-07 14:57:20.644: [    CSSD][1029]clssgmClientConnectMsg: properties of cmProc 111b16e30 - 0,1,2,3,4

2013-05-07 14:57:20.644: [    CSSD][1029]clssgmClientConnectMsg: Connect from con(30b1) proc(111b16e30) pid(8192022) version 11:2:1:4, properties: 0,1,2,3,4

2013-05-07 14:57:20.644: [    CSSD][1029]clssgmClientConnectMsg: msg flags 0x0000

2013-05-07 14:57:20.645: [    CSSD][1029]clssscSelect: cookie accept request 111b16e30

2013-05-07 14:57:20.645: [    CSSD][1029]clssscevtypSHRCON: getting client with cmproc 111b16e30

2013-05-07 14:57:20.645: [    CSSD][1029]clssgmRegisterClient: proc(4/111b16e30), client(1/111252f50)

2013-05-07 14:57:20.645: [    CSSD][1029]clssgmJoinGrock: global grock CRF- new client 111252f50 with con 30dd, requested num -1, flags 0x4000e00

2013-05-07 14:57:20.645: [    CSSD][1029]clssgmJoinGrock: ignoring grock join for client not requiring fencing until group information has been received from the master; group name CRF-, member number -1, flags 0x4000e00

2013-05-07 14:57:20.645: [    CSSD][1029]clssgmDiscEndpcl: gipcDestroy 30dd

2013-05-07 14:57:20.646: [    CSSD][1029]clssgmDeadProc: proc 111b16e30

2013-05-07 14:57:20.646: [    CSSD][1029]clssgmDestroyProc: cleaning up proc(111b16e30) con(30b1) skgpid  ospid 8192022 with 0 clients, refcount 0

2013-05-07 14:57:20.646: [    CSSD][1029]clssgmDiscEndpcl: gipcDestroy 30b1

2013-05-07 14:57:20.753: [    CSSD][2577]clssnmvDHBValidateNcopy: node 2, rac2, has a disk HB, but no network HB, DHB has rcfg 262286478, wrtcnt, 120838, LATS 18018385, lastSeqNo 120837, uniqueness 1367900300, timestamp 1367909840/18018130

2013-05-07 14:57:21.043: [    CSSD][4119]clssnmRcfgMgrThread: Local Join

2013-05-07 14:57:21.043: [    CSSD][4119]clssnmLocalJoinEvent: begin on node(1), waittime 193000

2013-05-07 14:57:21.043: [    CSSD][4119]clssnmLocalJoinEvent: set curtime (18018675) for my node

2013-05-07 14:57:21.043: [    CSSD][4119]clssnmLocalJoinEvent: scanning 32 nodes

2013-05-07 14:57:21.043: [    CSSD][4119]clssnmLocalJoinEvent: Node rac2, number 2, is in an existing cluster with disk state 3

2013-05-07 14:57:21.043: [    CSSD][4119]clssnmLocalJoinEvent: takeover aborted due to cluster member node found on disk

2013-05-07 14:57:21.341: [    CSSD][3348]clssgmWaitOnEventValue: after CmInfo State  val 3, eval 1 waited 0

2013-05-07 14:57:21.676: [    CSSD][1029]clssscSelect: cookie accept request 110977d00

2013-05-07 14:57:21.676: [    CSSD][1029]clssgmAllocProc: (111b16e30) allocated

2013-05-07 14:57:21.676: [    CSSD][1029]clssgmClientConnectMsg: properties of cmProc 111b16e30 - 0,1,2,3,4

2013-05-07 14:57:21.676: [    CSSD][1029]clssgmClientConnectMsg: Connect from con(3144) proc(111b16e30) pid(6160390) version 11:2:1:4, properties: 0,1,2,3,4

2013-05-07 14:57:21.676: [    CSSD][1029]clssgmClientConnectMsg: msg flags 0x0000

2013-05-07 14:57:21.678: [    CSSD][1029]clssscSelect: cookie accept request 111b16e30

2013-05-07 14:57:21.678: [    CSSD][1029]clssscevtypSHRCON: getting client with cmproc 111b16e30

2013-05-07 14:57:21.678: [    CSSD][1029]clssgmRegisterClient: proc(4/111b16e30), client(1/111252f50)

2013-05-07 14:57:21.678: [    CSSD][1029]clssgmJoinGrock: global grock CRF- new client 111252f50 with con 3170, requested num -1, flags 0x4000e00

2013-05-07 14:57:21.678: [    CSSD][1029]clssgmJoinGrock: ignoring grock join for client not requiring fencing until group information has been received from the master; group name CRF-, member number -1, flags 0x4000e00

2013-05-07 14:57:21.678: [    CSSD][1029]clssgmDiscEndpcl: gipcDestroy 3170

2013-05-07 14:57:21.678: [    CSSD][1029]clssgmDeadProc: proc 111b16e30

2013-05-07 14:57:21.678: [    CSSD][1029]clssgmDestroyProc: cleaning up proc(111b16e30) con(3144) skgpid  ospid 6160390 with 0 clients, refcount 0

2013-05-07 14:57:21.678: [    CSSD][1029]clssgmDiscEndpcl: gipcDestroy 3144

2013-05-07 14:57:21.765: [    CSSD][2577]clssnmvDHBValidateNcopy: node 2, rac2, has a disk HB, but no network HB, DHB has rcfg 262286478, wrtcnt, 120839, LATS 18019397, lastSeqNo 120838, uniqueness 1367900300, timestamp 1367909841/18019134

2013-05-07 14:57:22.035: [    CSSD][3862]clssnmSendingThread: sending join msg to all nodes

2013-05-07 14:57:22.035: [    CSSD][3862]clssnmSendingThread: sent 4 join msgs to all nodes

2013-05-07 14:57:22.351: [    CSSD][3348]clssgmWaitOnEventValue: after CmInfo State  val 3, eval 1 waited 0

2013-05-07 14:57:22.770: [    CSSD][2577]clssnmvDHBValidateNcopy: node 2, rac2, has a disk HB, but no network HB, DHB has rcfg 262286478, wrtcnt, 120840, LATS 18020402, lastSeqNo 120839, uniqueness 1367900300, timestamp 1367909842/18020139

错误日志不断循环改报错，查看网络，两台机子之间通讯没有问题