今天同事报他们大区的RAC死活只能启动一个节点,第一反应就是是不是心跳或者网络什么问题。问他们要了远程登陆密码连接上查看日志错误信息:
[ CSSD]2013-09-18 13:45:11.587 [2917116832] >TRACE: clssnmRcfgMgrThread: Local Join
[ CSSD]2013-09-18 13:45:11.587 [2917116832] >WARNING: clssnmLocalJoinEvent: takeover aborted due to ALIVE node on Disk
[ CSSD]2013-09-18 13:45:12.476 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(183) LATS(3001326890) Disk lastSeqNo(183)
[ CSSD]2013-09-18 13:45:12.544 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(183) LATS(3001326960) Disk lastSeqNo(183)
[ CSSD]2013-09-18 13:45:12.560 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(183) LATS(3001326970) Disk lastSeqNo(183)
[ CSSD]2013-09-18 13:45:13.480 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(184) LATS(3001327890) Disk lastSeqNo(184)
[ CSSD]2013-09-18 13:45:13.548 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(184) LATS(3001327960) Disk lastSeqNo(184)
[ CSSD]2013-09-18 13:45:13.564 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(184) LATS(3001327980) Disk lastSeqNo(184)
[ CSSD]2013-09-18 13:45:14.484 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(185) LATS(3001328900) Disk lastSeqNo(185)
[ CSSD]2013-09-18 13:45:14.552 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(185) LATS(3001328960) Disk lastSeqNo(185)
[ CSSD]2013-09-18 13:45:14.568 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(185) LATS(3001328980) Disk lastSeqNo(185)
[ CSSD]2013-09-18 13:45:15.488 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(186) LATS(3001329900) Disk lastSeqNo(186)
[ CSSD]2013-09-18 13:45:15.556 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(186) LATS(3001329970) Disk lastSeqNo(186)
[ CSSD]2013-09-18 13:45:15.572 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(186) LATS(3001329980) Disk lastSeqNo(186)
[ CSSD]2013-09-18 13:45:16.493 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(187) LATS(3001330900) Disk lastSeqNo(187)
[ CSSD]2013-09-18 13:45:16.560 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(187) LATS(3001330970) Disk lastSeqNo(187)
[ CSSD]2013-09-18 13:45:16.577 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(187) LATS(3001330990) Disk lastSeqNo(187)
[ CSSD]2013-09-18 13:45:17.497 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(188) LATS(3001331910) Disk lastSeqNo(188)
[ CSSD]2013-09-18 13:45:17.565 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(188) LATS(3001331980) Disk lastSeqNo(188)
[ CSSD]2013-09-18 13:45:17.581 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(188) LATS(3001331990) Disk lastSeqNo(188)
[ CSSD]2013-09-18 13:45:18.501 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(189) LATS(3001332910) Disk lastSeqNo(189)
[ CSSD]2013-09-18 13:45:18.569 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(189) LATS(3001332980) Disk lastSeqNo(189)
[ CSSD]2013-09-18 13:45:18.585 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(189) LATS(3001333000) Disk lastSeqNo(189)
[ CSSD]2013-09-18 13:45:18.617 [2917116832] >TRACE: clssnmRcfgMgrThread: Local Join
[ CSSD]2013-09-18 13:45:18.617 [2917116832] >WARNING: clssnmLocalJoinEvent: takeover aborted due to ALIVE node on Disk
[ CSSD]2013-09-18 13:45:19.505 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(190) LATS(3001333920) Disk lastSeqNo(190)
[ CSSD]2013-09-18 13:45:19.573 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(190) LATS(3001333980) Disk lastSeqNo(190)
[ CSSD]2013-09-18 13:45:19.589 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(190) LATS(3001334000) Disk lastSeqNo(190)
[ CSSD]2013-09-18 13:45:20.509 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(191) LATS(3001334920) Disk lastSeqNo(191)
[ CSSD]2013-09-18 13:45:20.577 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(191) LATS(3001334990) Disk lastSeqNo(191)
[ CSSD]2013-09-18 13:45:20.593 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(191) LATS(3001335000) Disk lastSeqNo(191)
[ CSSD]2013-09-18 13:45:21.517 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(192) LATS(3001335930) Disk lastSeqNo(192)
[ CSSD]2013-09-18 13:45:21.581 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(192) LATS(3001336000) Disk lastSeqNo(192)
[ CSSD]2013-09-18 13:45:21.597 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(192) LATS(3001336010) Disk lastSeqNo(192)
[ CSSD]2013-09-18 13:45:22.525 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(193) LATS(3001336940) Disk lastSeqNo(193)
[ CSSD]2013-09-18 13:45:22.589 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(193) LATS(3001337000) Disk lastSeqNo(193)
[ CSSD]2013-09-18 13:45:22.605 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(193) LATS(3001337020) Disk lastSeqNo(193)
[ CSSD]2013-09-18 13:45:23.534 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(194) LATS(3001337950) Disk lastSeqNo(194)
[ CSSD]2013-09-18 13:45:24.533 [2977405856] >TRACE: clssnmHandleSync: diskTimeout set to (57000)ms
[ CSSD]2013-09-18 13:45:11.587 [2917116832] >WARNING: clssnmLocalJoinEvent: takeover aborted due to ALIVE node on Disk
[ CSSD]2013-09-18 13:45:12.476 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(183) LATS(3001326890) Disk lastSeqNo(183)
[ CSSD]2013-09-18 13:45:12.544 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(183) LATS(3001326960) Disk lastSeqNo(183)
[ CSSD]2013-09-18 13:45:12.560 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(183) LATS(3001326970) Disk lastSeqNo(183)
[ CSSD]2013-09-18 13:45:13.480 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(184) LATS(3001327890) Disk lastSeqNo(184)
[ CSSD]2013-09-18 13:45:13.548 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(184) LATS(3001327960) Disk lastSeqNo(184)
[ CSSD]2013-09-18 13:45:13.564 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(184) LATS(3001327980) Disk lastSeqNo(184)
[ CSSD]2013-09-18 13:45:14.484 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(185) LATS(3001328900) Disk lastSeqNo(185)
[ CSSD]2013-09-18 13:45:14.552 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(185) LATS(3001328960) Disk lastSeqNo(185)
[ CSSD]2013-09-18 13:45:14.568 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(185) LATS(3001328980) Disk lastSeqNo(185)
[ CSSD]2013-09-18 13:45:15.488 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(186) LATS(3001329900) Disk lastSeqNo(186)
[ CSSD]2013-09-18 13:45:15.556 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(186) LATS(3001329970) Disk lastSeqNo(186)
[ CSSD]2013-09-18 13:45:15.572 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(186) LATS(3001329980) Disk lastSeqNo(186)
[ CSSD]2013-09-18 13:45:16.493 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(187) LATS(3001330900) Disk lastSeqNo(187)
[ CSSD]2013-09-18 13:45:16.560 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(187) LATS(3001330970) Disk lastSeqNo(187)
[ CSSD]2013-09-18 13:45:16.577 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(187) LATS(3001330990) Disk lastSeqNo(187)
[ CSSD]2013-09-18 13:45:17.497 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(188) LATS(3001331910) Disk lastSeqNo(188)
[ CSSD]2013-09-18 13:45:17.565 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(188) LATS(3001331980) Disk lastSeqNo(188)
[ CSSD]2013-09-18 13:45:17.581 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(188) LATS(3001331990) Disk lastSeqNo(188)
[ CSSD]2013-09-18 13:45:18.501 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(189) LATS(3001332910) Disk lastSeqNo(189)
[ CSSD]2013-09-18 13:45:18.569 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(189) LATS(3001332980) Disk lastSeqNo(189)
[ CSSD]2013-09-18 13:45:18.585 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(189) LATS(3001333000) Disk lastSeqNo(189)
[ CSSD]2013-09-18 13:45:18.617 [2917116832] >TRACE: clssnmRcfgMgrThread: Local Join
[ CSSD]2013-09-18 13:45:18.617 [2917116832] >WARNING: clssnmLocalJoinEvent: takeover aborted due to ALIVE node on Disk
[ CSSD]2013-09-18 13:45:19.505 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(190) LATS(3001333920) Disk lastSeqNo(190)
[ CSSD]2013-09-18 13:45:19.573 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(190) LATS(3001333980) Disk lastSeqNo(190)
[ CSSD]2013-09-18 13:45:19.589 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(190) LATS(3001334000) Disk lastSeqNo(190)
[ CSSD]2013-09-18 13:45:20.509 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(191) LATS(3001334920) Disk lastSeqNo(191)
[ CSSD]2013-09-18 13:45:20.577 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(191) LATS(3001334990) Disk lastSeqNo(191)
[ CSSD]2013-09-18 13:45:20.593 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(191) LATS(3001335000) Disk lastSeqNo(191)
[ CSSD]2013-09-18 13:45:21.517 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(192) LATS(3001335930) Disk lastSeqNo(192)
[ CSSD]2013-09-18 13:45:21.581 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(192) LATS(3001336000) Disk lastSeqNo(192)
[ CSSD]2013-09-18 13:45:21.597 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(192) LATS(3001336010) Disk lastSeqNo(192)
[ CSSD]2013-09-18 13:45:22.525 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(193) LATS(3001336940) Disk lastSeqNo(193)
[ CSSD]2013-09-18 13:45:22.589 [3036687264] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(193) LATS(3001337000) Disk lastSeqNo(193)
[ CSSD]2013-09-18 13:45:22.605 [3028294560] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(193) LATS(3001337020) Disk lastSeqNo(193)
[ CSSD]2013-09-18 13:45:23.534 [3019635616] >TRACE: clssnmReadDskHeartbeat: node(1) is down. rcfg(2) wrtcnt(194) LATS(3001337950) Disk lastSeqNo(194)
[ CSSD]2013-09-18 13:45:24.533 [2977405856] >TRACE: clssnmHandleSync: diskTimeout set to (57000)ms
检查网络和/etc/hosts发现没有问题,然后检查等效性,发现连接点见相互不通,查看防火墙,有一个节点防火墙开启,关闭防火墙,在此验证等效性,节点一不能直接通过,节点二没问题,将节点2
authorized_keys拷贝到节点1,再次验证,正确。重新启动未启动节点,正常!
来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/22779291/viewspace-772897/,如需转载,请注明出处,否则将追究法律责任。
转载于:http://blog.itpub.net/22779291/viewspace-772897/