Oracle 11g RAC表决盘和OCR盘挂载失败引发的节点故障

最新推荐文章于 2024-01-18 09:53:05 发布

cuantangnie3915

最新推荐文章于 2024-01-18 09:53:05 发布

阅读量456

点赞数

文章标签：数据库

测试环境的一套两节点RAC，一个节点出现故障，启动不了实例

查看节点1的grid日志，发现找不到表决盘
tail -100 /opt/ora11grid/log/vgerndpud853/alertvgerndpud853.log
..
[cssd(3824)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /opt/ora11grid/log/vgerndpud853/cssd/ocssd.log
2016-10-17 07:56:11.768
[cssd(3824)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /opt/ora11grid/log/vgerndpud853/cssd/ocssd.log
2016-10-17 07:56:26.781
[cssd(3824)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /opt/ora11grid/log/vgerndpud853/cssd/ocssd.log
2016-10-17 07:56:41.794
[cssd(3824)]CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds; Details at (:CSSNM00070:) in /opt/ora11grid/log/vgerndpud853/cssd/ocssd.log
2016-10-17 07:56:55.095
[/opt/ora11grid/bin/cssdagent(3812)]CRS-5818:Aborted command 'start' for resource 'ora.cssd'. Details at (:CRSAGF00113:) {0:34:3} in /opt/ora11grid/log/vgerndpud853/agent/ohasd/oracssdagent_root/oracssdagent_root.log.
2016-10-17 07:56:55.096
[cssd(3824)]CRS-1656:The CSS daemon is terminating due to a fatal error; Details at (:CSSSC00012:) in /opt/ora11grid/log/vgerndpud853/cssd/ocssd.log

查看grid日志中提到的详细日志，进一步验证了表决盘丢失
tail -100 /opt/ora11grid/log/vgerndpud853/cssd/ocssd.log
..
222499 2016-10-17 08:06:58.959: [   SKGFD][257890048]Discovery with str:/voting/vot1/vot1.data,/voting/vot2/vot2.data,/voting/vot3/vot3.data:
222500
222501 2016-10-17 08:06:58.959: [   SKGFD][257890048]UFS discovery with :/voting/vot1/vot1.data:
222502
222503 2016-10-17 08:06:58.959: [   SKGFD][257890048]OSS discovery with :/voting/vot1/vot1.data:
222504
222505 2016-10-17 08:06:58.959: [   SKGFD][257890048]Discovery advancing to nxt string :/voting/vot2/vot2.data:
222506
222507 2016-10-17 08:06:58.959: [   SKGFD][257890048]UFS discovery with :/voting/vot2/vot2.data:
222508
222509 2016-10-17 08:06:58.959: [   SKGFD][257890048]OSS discovery with :/voting/vot2/vot2.data:
222510
222511 2016-10-17 08:06:58.959: [   SKGFD][257890048]Discovery advancing to nxt string :/voting/vot3/vot3.data:
222512
222513 2016-10-17 08:06:58.959: [   SKGFD][257890048]UFS discovery with :/voting/vot3/vot3.data:
222514
222515 2016-10-17 08:06:58.959: [   SKGFD][257890048]OSS discovery with :/voting/vot3/vot3.data:
222516
222517 2016-10-17 08:06:58.959: [    CSSD][257890048]clssnmvDiskVerify: Successful discovery of 0 disks
222518 2016-10-17 08:06:58.959: [    CSSD][257890048]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery

登陆节点2
查看集群状态，发现节点2的状态正常
crs_stat -t -v

查看表决盘和crs盘的挂载情况，正常
vgerndpud852: /opt/ora11grid/bin # mount | grep vot
/dev/vx/dsk/dg_orabe/v_vot2 on /voting/vot2 type vxfs (rw,mntlock=VCS,cluster,crw,delaylog,largefiles,ioerror=mdisable)
/dev/vx/dsk/dg_orabe/v_vot1 on /voting/vot1 type vxfs (rw,mntlock=VCS,cluster,crw,delaylog,largefiles,ioerror=mdisable)
/dev/vx/dsk/dg_orabe/v_vot3 on /voting/vot3 type vxfs (rw,mntlock=VCS,cluster,crw,delaylog,largefiles,ioerror=mdisable)

vgerndpud852: /opt/ora11grid/bin # mount | grep ocr
/dev/vx/dsk/dg_orabe/v_ocr2 on /ocr/ocr2 type vxfs (rw,mntlock=VCS,cluster,crw,delaylog,largefiles,ioerror=mdisable)
/dev/vx/dsk/dg_orabe/v_ocr3 on /ocr/ocr3 type vxfs (rw,mntlock=VCS,cluster,crw,delaylog,largefiles,ioerror=mdisable)
/dev/vx/dsk/dg_orabe/v_ocr1 on /ocr/ocr1 type vxfs (rw,mntlock=VCS,cluster,crw,delaylog,largefiles,ioerror=mdisable)

在节点1，查看表决盘和crs盘的挂载情况，发现盘没有挂载上
/opt/ora11grid[FRWK]:mount | grep vot
/opt/ora11grid[FRWK]:mount | grep ocr

重新挂载盘，重启集群，RAC恢复正常

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/26506993/viewspace-2126822/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/26506993/viewspace-2126822/