RAC10g下crs状态为unknown的解决办法

crs_stop -all 命令后遇到 UNKNOWN

crs_stop -all 命令后遇到 UNKNOWN crs_stop -all 命令后遇到 UNKNOWN RAC中执行crs_stop -all后提示一些服务停不了，再用crs_stat -t查看状态的时候，停不掉的服务会显示状态是UNKNOWN， 服务在UNKNOWN状态下用 crs_start 不能启动服务 例如： [oracle@rac1 ~]$ crs_stop -all Attempting to stop `ora.rac1.gsd` on member `rac1` Attempting to stop `ora.rac1.ons` on member `rac1` Attempting to stop `ora.rac2.gsd` on member `rac2` Attempting to stop `ora.ora9i.db` on member `rac1` Attempting to stop `ora.rac2.ons` on member `rac2` Stop of `ora.rac1.gsd` on member `rac1` succeeded. Stop of `ora.rac2.gsd` on member `rac2` succeeded. Stop of `ora.rac2.ons` on member `rac2` succeeded. Stop of `ora.rac1.ons` on member `rac1` succeeded. Stop of `ora.ora9i.db` on member `rac1` succeeded. Attempting to stop `ora.rac1.LISTENER_RAC1.lsnr` on member `rac1` Attempting to stop `ora.rac2.LISTENER_RAC2.lsnr` on member `rac2` `ora.rac1.LISTENER_RAC1.lsnr` on member `rac1` has experienced an unrecoverable failure. Human intervention required to resume its availability. `ora.ora9i.ora9i1.inst` is already OFFLINE. `ora.rac2.LISTENER_RAC2.lsnr` on member `rac2` has experienced an unrecoverable failure. Human intervention required to resume its availability. `ora.ora9i.ora9i2.inst` is already OFFLINE. Attempting to stop `ora.rac1.ASM1.asm` on member `rac1` `ora.rac1.ASM1.asm` on member `rac1` has experienced an unrecoverable failure. Human intervention required to resume its availability. Attempting to stop `ora.rac2.ASM2.asm` on member `rac2` Attempting to stop `ora.rac1.LISTENER_RAC1.lsnr` on member `rac1` `ora.rac2.ASM2.asm` on member `rac2` has experienced an unrecoverable failure. Human intervention required to resume its availability. `ora.rac1.LISTENER_RAC1.lsnr` on member `rac1` has experienced an unrecoverable failure. Human intervention required to resume its availability. Attempting to stop `ora.rac1.ASM1.asm` on member `rac1` `ora.rac1.ASM1.asm` on member `rac1` has experienced an unrecoverable failure. Human intervention required to resume its availability. Attempting to stop `ora.rac2.LISTENER_RAC2.lsnr` on member `rac2` Attempting to stop `ora.rac1.vip` on member `rac1` `ora.rac2.LISTENER_RAC2.lsnr` on member `rac2` has experienced an unrecoverable failure. Human intervention required to resume its availability. Attempting to stop `ora.rac2.ASM2.asm` on member `rac2` `ora.rac2.ASM2.asm` on member `rac2` has experienced an unrecoverable failure. Human intervention required to resume its availability. Attempting to stop `ora.rac2.vip` on member `rac2` Stop of `ora.rac1.vip` on member `rac1` succeeded. Stop of `ora.rac2.vip` on member `rac2` succeeded. CRS-0216: Could not stop resource 'ora.ora9i.ora9i1.inst'. CRS-0216: Could not stop resource 'ora.ora9i.ora9i2.inst'. CRS-0216: Could not stop resource 'ora.rac1.ASM1.asm'. CRS-0216: Could not stop resource 'ora.rac1.LISTENER_RAC1.lsnr'. CRS-0216: Could not stop resource 'ora.rac1.vip'. CRS-0216: Could not stop resource 'ora.rac2.ASM2.asm'. CRS-0216: Could not stop resource 'ora.rac2.LISTENER_RAC2.lsnr'. CRS-0216: Could not stop resource 'ora.rac2.vip'. 在UNKNOWN状态下，使用crs_start -all命令不会把RAC启动起来，启动的时候会报错。 [oracle@rac1 ~]$ crs_start -all Attempting to start `ora.rac1.vip` on member `rac1` Attempting to start `ora.rac2.vip` on member `rac2` Attempting to start `ora.rac2.gsd` on member `rac2` Attempting to start `ora.rac2.ons` on member `rac2` Attempting to start `ora.rac1.gsd` on member `rac1` Attempting to start `ora.rac1.ons` on member `rac1` `ora.rac2.gsd` on member `rac2` has experienced an unrecoverable failure. Human intervention required to resume its availability. `ora.rac2.ons` on member `rac2` has experienced an unrecoverable failure. Human intervention required to resume its availability. Attempting to start `ora.ora9i.db` on member `rac1` `ora.rac1.ons` on member `rac1` has experienced an unrecoverable failure. Human intervention required to resume its availability. `ora.rac1.gsd` on member `rac1` has experienced an unrecoverable failure. Human intervention required to resume its availability. `ora.ora9i.db` on member `rac1` has experienced an unrecoverable failure. Human intervention required to resume its availability. Start of `ora.rac1.vip` on member `rac1` succeeded. CRS-1028: Dependency analysis failed because of: 'Resource in UNKNOWN state: ora.rac1.ASM1.asm' Start of `ora.rac2.vip` on member `rac2` succeeded. CRS-1028: Dependency analysis failed because of: 'Resource in UNKNOWN state: ora.rac2.ASM2.asm' CRS-0215: Could not start resource 'ora.ora9i.db'. CRS-0223: Resource 'ora.ora9i.ora9i1.inst' has placement error. CRS-0223: Resource 'ora.ora9i.ora9i2.inst' has placement error. CRS-0215: Could not start resource 'ora.rac1.gsd'. CRS-0215: Could not start resource 'ora.rac1.ons'. CRS-0215: Could not start resource 'ora.rac2.gsd'. CRS-0215: Could not start resource 'ora.rac2.ons'. 此时crs_stat -t的状态还是有很多的UNKNOWN （烦、烦、烦，rac中真奇怪，正常启动关闭服务时， 状态只有online和offline两种（启动与未启动），结果RAC中给你来了个UNKNOWN）： [oracle@rac1 ~]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.ora9i.db application ONLINE UNKNOWN rac1 ora....i1.inst application OFFLINE OFFLINE ora....i2.inst application OFFLINE OFFLINE ora....SM1.asm application OFFLINE UNKNOWN rac1 ora....C1.lsnr application OFFLINE UNKNOWN rac1 ora.rac1.gsd application ONLINE UNKNOWN rac1 ora.rac1.ons application ONLINE UNKNOWN rac1 ora.rac1.vip application ONLINE ONLINE rac1 ora....SM2.asm application OFFLINE UNKNOWN rac2 ora....C2.lsnr application OFFLINE UNKNOWN rac2 ora.rac2.gsd application ONLINE UNKNOWN rac2 ora.rac2.ons application ONLINE UNKNOWN rac2 ora.rac2.vip application ONLINE ONLINE rac2 [oracle@rac1 ~]$ sqlplus /nolog SQL*Plus: Release 10.2.0.1.0 - Production on Thu Aug 7 13:42:36 2008 Copyright (c) 1982, 2005, Oracle. All rights reserved. SQL> conn /as sysdba Connected to an idle instance. 显示数据库已经连接不上； 此时可有两种方法来解决： 方法一：先禁用crs服务，后启用crs服务 （相当于从新启动系统，因为crs是随系统自动启动的），需要在两台节点上都执行； 具体步骤： 节点1： 1、[root@rac1 ~]# /etc/init.d/init.crs stop Shutting down Oracle Cluster Ready Services (CRS): Stopping resources. Successfully stopped CRS resources Stopping CSSD. Shutting down CSS daemon. Shutdown request successfully issued. Shutdown has begun. The daemons should exit soon. 2、[root@rac1 ~]# su - oracle [oracle@rac1 ~]$ crs_stat -t CRS-0184: Cannot communicate with the CRS daemon. 3、[oracle@rac1 ~]$ exit logout [root@rac1 ~]# /etc/init.d/init.crs start Startup will be queued to init within 90 seconds. 4、[root@rac1 ~]# su - oracle [oracle@rac1 ~]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.ora9i.db application ONLINE ONLINE rac1 ora....i1.inst application ONLINE ONLINE rac1 ora....i2.inst application ONLINE ONLINE rac2 ora....SM1.asm application ONLINE ONLINE rac1 ora....C1.lsnr application ONLINE ONLINE rac1 ora.rac1.gsd application ONLINE ONLINE rac1 ora.rac1.ons application ONLINE ONLINE rac1 ora.rac1.vip application ONLINE ONLINE rac1 ora....SM2.asm application ONLINE ONLINE rac2 ora....C2.lsnr application ONLINE ONLINE rac2 ora.rac2.gsd application ONLINE ONLINE rac2 ora.rac2.ons application ONLINE ONLINE rac2 ora.rac2.vip application ONLINE ONLINE rac2 [oracle@rac1 ~]$ 节点2： 1、[root@rac2 ~]# /etc/init.d/init.crs stop Shutting down Oracle Cluster Ready Services (CRS): Stopping resources. Successfully stopped CRS resources Stopping CSSD. Shutting down CSS daemon. Shutdown request successfully issued. Shutdown has begun. The daemons should exit soon. 2、[root@rac2 ~]# su - oracle [oracle@rac2 ~]$ crs_stat -t CRS-0184: Cannot communicate with the CRS daemon. [oracle@rac2 ~]$ exit logout 3、[root@rac2 ~]# /etc/init.d/init.crs start Startup will be queued to init within 90 seconds. 4、[root@rac2 ~]# su - oracle [oracle@rac2 ~]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.ora9i.db application ONLINE ONLINE rac1 ora....i1.inst application ONLINE ONLINE rac1 ora....i2.inst application ONLINE ONLINE rac2 ora....SM1.asm application ONLINE ONLINE rac1 ora....C1.lsnr application ONLINE ONLINE rac1 ora.rac1.gsd application ONLINE ONLINE rac1 ora.rac1.ons application ONLINE ONLINE rac1 ora.rac1.vip application ONLINE ONLINE rac1 ora....SM2.asm application ONLINE ONLINE rac2 ora....C2.lsnr application ONLINE ONLINE rac2 ora.rac2.gsd application ONLINE ONLINE rac2 ora.rac2.ons application ONLINE ONLINE rac2 ora.rac2.vip application ONLINE ONLINE rac2 [oracle@rac2 ~]$ 方法二：可以使用crs_stop -f先强制停止服务，在启动服务（在一台节点上运行即可）； 具体步骤： [oracle@rac2 ~]$ crs_stop -f ora.rac1.ons Attempting to stop `ora.rac1.ons` on member `rac1` Stop of `ora.rac1.ons` on member `rac1` succeeded. [oracle@rac2 ~]$ crs_start -f ora.rac1.ons Attempting to start `ora.rac1.ons` on member `rac1` Start of `ora.rac1.ons` on member `rac1` succeeded. [oracle@rac2 ~]$ crs_stop -f ora.rac2.ons Attempting to stop `ora.rac2.ons` on member `rac2` Stop of `ora.rac2.ons` on member `rac2` succeeded. [oracle@rac2 ~]$ crs_start -f ora.rac2.ons Attempting to start `ora.rac2.ons` on member `rac2` Start of `ora.rac2.ons` on member `rac2` succeeded. [oracle@rac2 ~]$ crs_stop -f ora.rac1.gsd Attempting to stop `ora.rac1.gsd` on member `rac1` Stop of `ora.rac1.gsd` on member `rac1` succeeded. [oracle@rac2 ~]$ crs_start -f ora.rac1.gsd Attempting to start `ora.rac1.gsd` on member `rac1` Start of `ora.rac1.gsd` on member `rac1` succeeded. [oracle@rac2 ~]$ crs_stop -f ora.rac2.gsd Attempting to stop `ora.rac2.gsd` on member `rac2` Stop of `ora.rac2.gsd` on member `rac2` succeeded. [oracle@rac2 ~]$ crs_start -f ora.rac2.gsd Attempting to start `ora.rac2.gsd` on member `rac2` Start of `ora.rac2.gsd` on member `rac2` succeeded. [oracle@rac2 ~]$ crs_stop -f ora.rac1.LISTENER_RAC1.lsnr Attempting to stop `ora.rac1.LISTENER_RAC1.lsnr` on member `rac1` Stop of `ora.rac1.LISTENER_RAC1.lsnr` on member `rac1` succeeded. [oracle@rac2 ~]$ crs_start -f ora.rac1.LISTENER_RAC1.lsnr Attempting to start `ora.rac1.LISTENER_RAC1.lsnr` on member `rac1` Start of `ora.rac1.LISTENER_RAC1.lsnr` on member `rac1` succeeded. [oracle@rac2 ~]$ crs_stop -f ora.rac2.LISTENER_RAC2.lsnr Attempting to stop `ora.rac2.LISTENER_RAC2.lsnr` on member `rac2` Stop of `ora.rac2.LISTENER_RAC2.lsnr` on member `rac2` succeeded. [oracle@rac2 ~]$ crs_start -f ora.rac2.LISTENER_RAC2.lsnr Attempting to start `ora.rac2.LISTENER_RAC2.lsnr` on member `rac2` Start of `ora.rac2.LISTENER_RAC2.lsnr` on member `rac2` succeeded. [oracle@rac2 ~]$ crs_stop -f ora.rac1.ASM1.asm Attempting to stop `ora.rac1.ASM1.asm` on member `rac1` Stop of `ora.rac1.ASM1.asm` on member `rac1` succeeded. [oracle@rac2 ~]$ crs_start -f ora.rac1.ASM1.asm Attempting to start `ora.rac1.ASM1.asm` on member `rac1` Start of `ora.rac1.ASM1.asm` on member `rac1` succeeded. [oracle@rac2 ~]$ crs_stop -f ora.rac2.ASM2.asm Attempting to stop `ora.rac2.ASM2.asm` on member `rac2` Stop of `ora.rac2.ASM2.asm` on member `rac2` succeeded. [oracle@rac2 ~]$ crs_start -f ora.rac2.ASM2.asm Attempting to start `ora.rac2.ASM2.asm` on member `rac2` Start of `ora.rac2.ASM2.asm` on member `rac2` succeeded. [oracle@rac2 ~]$ crs_stop -f ora.ora9i.ora9i1.inst `ora.ora9i.ora9i1.inst` is already OFFLINE. CRS-0216: Could not stop resource 'ora.ora9i.ora9i1.inst'. [oracle@rac2 ~]$ crs_start -f ora.ora9i.ora9i1.inst Attempting to start `ora.ora9i.ora9i1.inst` on member `rac1` Start of `ora.ora9i.ora9i1.inst` on member `rac1` succeeded. [oracle@rac2 ~]$ crs_stop -f ora.ora9i.ora9i2.inst `ora.ora9i.ora9i2.inst` is already OFFLINE. CRS-0216: Could not stop resource 'ora.ora9i.ora9i2.inst'. [oracle@rac2 ~]$ crs_start -f ora.ora9i.ora9i2.inst Attempting to start `ora.ora9i.ora9i2.inst` on member `rac2` Start of `ora.ora9i.ora9i2.inst` on member `rac2` succeeded. [oracle@rac2 ~]$ crs_stop -f ora.ora9i.db Attempting to stop `ora.ora9i.db` on member `rac2` Stop of `ora.ora9i.db` on member `rac2` succeeded. [oracle@rac2 ~]$ crs_start -f ora.ora9i.db Attempting to start `ora.ora9i.db` on member `rac1` Start of `ora.ora9i.db` on member `rac1` succeeded. [oracle@rac2 ~]$ crs_stat -t Name Type Target State Host ------------------------------------------------------------ ora.ora9i.db application ONLINE ONLINE rac1 ora....i1.inst application ONLINE ONLINE rac1 ora....i2.inst application ONLINE ONLINE rac2 ora....SM1.asm application ONLINE ONLINE rac1 ora....C1.lsnr application ONLINE ONLINE rac1 ora.rac1.gsd application ONLINE ONLINE rac1 ora.rac1.ons application ONLINE ONLINE rac1 ora.rac1.vip application ONLINE ONLINE rac1 ora....SM2.asm application ONLINE ONLINE rac2 ora....C2.lsnr application ONLINE ONLINE rac2 ora.rac2.gsd application ONLINE ONLINE rac2 ora.rac2.ons application ONLINE ONLINE rac2 ora.rac2.vip application ONLINE ONLINE rac2 总结：在rac环境中如果使用crs_stop -all去停rac服务时，很有可能会出现crs中的一些服务的状态是UNKNOWN，这时我们可以采用两种方法去重新启动crs服务： 1：转换到root用户下用/etc/init.d/init.crs stop先禁用crs，然后再用/etc/init.d/init.crs start去启用crs，启用crs后会自动启动crs的一系列服务，注意此种方法需要在两台节点上都执行； 2：是用crs_stop -f 参数把crs中状态是UNKNOWN的服务关掉，然后再用crs_start -f （加一个-f的参数）启动所有的服务就可以。参考：http://www.rachelp.nl/index_kb.php?menu=articles&actie=show&id=31 要分别在两个节点上执行； 在启动的过程中最好检测着crs、ASM和数据库的日志： crs日志： [oracle@rac1 ~]$ tail -f /u01/app/oracle/product/10.2.0/crs_1/log/rac1/alertrac1.log [oracle@rac2 ~]$ tail -f /u01/app/oracle/product/10.2.0/crs_1/log/rac2/alertrac2.log ASM日志： [oracle@rac1 ~]$ tail -f /u01/app/oracle/admin/+ASM/bdump/alert_+ASM1.log [oracle@rac2 ~]$ tail -f /u01/app/oracle/admin/+ASM/bdump/alert_+ASM2.log 数据库日志： [oracle@rac1 ~]$ tail -f /u01/app/oracle/admin/ora9i/bdump/alert_ora9i1.log [oracle@rac2 ~]$ tail -f /u01/app/oracle/admin/ora9i/bdump/alert_ora9i2.log