Oracle 10gR2 RAC 没有备份的情况下恢复ocr和vote

本例使用重新初始化OCR和VOTE的办法

1.停止所有节点的Clusterware stack
本例在一个节点rac3上操作

[oracle@rac3 rac3]$ srvctl stop database -d ccupdb
[oracle@rac3 rac3]$ srvctl stop asm -n rac3
[oracle@rac3 rac3]$ srvctl stop asm -n rac4
[oracle@rac3 rac3]$ srvctl stop nodeapps -n rac3
[oracle@rac3 rac3]$ srvctl stop nodeapps -n rac4
[oracle@rac3 rac3]$ crs_stat -t -v
Name Type R/RA F/FT Target State Host
———————————————————————-
ora….b1.inst application 0/5 0/0 OFFLINE OFFLINE
ora….b2.inst application 0/5 0/0 OFFLINE OFFLINE
ora.ccupdb.db application 0/0 0/1 OFFLINE OFFLINE
ora….SM1.asm application 0/5 0/0 OFFLINE OFFLINE
ora….C3.lsnr application 0/5 0/0 OFFLINE OFFLINE
ora.rac3.gsd application 0/5 0/0 OFFLINE OFFLINE
ora.rac3.ons application 0/3 0/0 OFFLINE OFFLINE
ora.rac3.vip application 0/0 0/0 OFFLINE OFFLINE
ora….SM2.asm application 0/5 0/0 OFFLINE OFFLINE
ora….C4.lsnr application 0/5 0/0 OFFLINE OFFLINE
ora.rac4.gsd application 0/5 0/0 OFFLINE OFFLINE
ora.rac4.ons application 0/3 0/0 OFFLINE OFFLINE
ora.rac4.vip application 0/0 0/0 OFFLINE OFFLINE

[root@rac3 bin]# ./crsctl stop crs
Stopping resources.
Successfully stopped CRS resources
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.

[root@rac3 bin]# ./crsctl check crs
Failure 1 contacting CSS daemon
Cannot communicate with CRS
Cannot communicate with EVM

[root@rac4 bin]# ./crsctl stop crs
Stopping resources.
Successfully stopped CRS resources
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.

[root@rac4 bin]# ./crsctl check crs
Failure 1 contacting CSS daemon
Cannot communicate with CRS
Cannot communicate with EVM

上述演示的是正常停止crs的顺序，也可以使用最后的命令直接停止

2.用root用户在每个节点上执行$ORA_CRS_HOME/install/rootdelete.sh
[root@rac3 install]# sh rootdelete.sh
Shutting down Oracle Cluster Ready Services (CRS):
Stopping resources.
Error while stopping resources. Possible cause: CRSD is down.
Stopping CSSD.
Unable to communicate with the CSS daemon.
Shutdown has begun. The daemons should exit soon.
Checking to see if Oracle CRS stack is down…
Oracle CRS stack is not running.
Oracle CRS stack is down now.
Removing script. for Oracle Cluster Ready services
Updating ocr file for downgrade
Cleaning up SCR settings in ‘/etc/oracle/scls_scr’

[root@rac4 install]# sh rootdelete.sh
Shutting down Oracle Cluster Ready Services (CRS):
Stopping resources.
Error while stopping resources. Possible cause: CRSD is down.
Stopping CSSD.
Unable to communicate with the CSS daemon.
Shutdown has begun. The daemons should exit soon.
Checking to see if Oracle CRS stack is down…
Oracle CRS stack is not running.
Oracle CRS stack is down now.
Removing script. for Oracle Cluster Ready services
Updating ocr file for downgrade
Cleaning up SCR settings in ‘/etc/oracle/scls_scr’

3.只在其中一个节点上执行$ORA_CRS_HOME/install/rootdeinstall.sh
[root@rac3 install]# sh rootdeinstall.sh

Removing contents from OCR device
读入了 2560+0 个块
输出了 2560+0 个块

—上述命令会清除相关信息，本例手动再次清除确认
修改 /etc/inittab, 删除以下三行.
h1:2:respawn:/etc/init.evmd run >/dev/null 2>&1 h2:2:respawn:/etc/init.cssd fatal >/dev/null 2>&1 h3:2:respawn:/etc/init.crsd run >/dev/null 2>&1  
rm -rf /etc/oracle/* 
rm -f /etc/init.d/init.cssd
rm -f /etc/init.d/init.crs
rm -f /etc/init.d/init.crsd
rm -f /etc/init.d/init.evmd
rm -f /etc/inittab.crs
cp /etc/inittab.orig /etc/inittab 
rm -rf /var/tmp/.oracle
rm -rf /tmp/.oracle 

4.在和步骤3中的同一个节点上执行$ORA_CRS_HOME/root.sh
[root@rac3 crs]# pwd
/oracle/crs
[root@rac3 crs]# sh root.sh
WARNING: directory '/oracle' is not owned by root
Checking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory '/oracle' is not owned by root
assigning default hostname rac3 for node 1.
assigning default hostname rac4 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node : 
node 1: rac3 rac3-priv rac3
node 2: rac4 rac4-priv rac4
Creating OCR keys for user ‘root’, privgrp ‘root’..
Operation successful.
Now formatting voting device: /dev/raw/raw2
Format of 1 voting devices complete.
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
rac3
CSS is inactive on these nodes.
rac4
Local node checking complete.
Run root.sh on remaining nodes to start CRS daemons.

—这里注意最后的输出，要求在另一个节点上也运行root.sh

5.在另一个节点上执行$ORA_CRS_HOME/root.sh
[root@rac4 crs]# pwd
/oracle/crs
[root@rac4 crs]# sh root.sh
WARNING: directory ‘/oracle’ is not owned by root
Checking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directory
Setting up NS directories
Oracle Cluster Registry configuration upgraded successfully
WARNING: directory ‘/oracle’ is not owned by root
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
assigning default hostname rac3 for node 1.
assigning default hostname rac4 for node 2.
Successfully accumulated necessary OCR keys.
Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.
node : 
node 1: rac3 rac3-priv rac3
node 2: rac4 rac4-priv rac4
clscfg: Arguments check out successfully.

NO KEYS WERE WRITTEN. Supply -force parameter to override.
-force is destructive and will destroy any previous cluster
configuration.
Oracle Cluster Registry for cluster has already been initialized
Startup will be queued to init within 90 seconds.
Adding daemons to inittab
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
rac3
rac4
CSS is active on all nodes.
Waiting for the Oracle CRSD and EVMD to start
Oracle CRS stack installed and running under init(1M)
Running vipca(silent) for configuring nodeapps

给定的接口 “eth0″ 不是公共接口。应使用公共接口来配置虚拟 IP。
—这里注意最后的输出，执行vipca，此处依然在rac4节点上以root用户执行
=======
执行完成后发现已经重建了crs资源
[oracle@rac3 install]$ crs_stat -t -v
Name Type R/RA F/FT Target State Host
———————————————————————-
ora.rac3.gsd application 0/5 0/0 ONLINE ONLINE rac3
ora.rac3.ons application 0/3 0/0 ONLINE ONLINE rac3
ora.rac3.vip application 0/0 0/0 ONLINE ONLINE rac3
ora.rac4.gsd application 0/5 0/0 ONLINE ONLINE rac4
ora.rac4.ons application 0/3 0/0 ONLINE ONLINE rac4

ora.rac4.vip application 0/0 0/0 ONLINE ONLINE rac4

6.然后用netca命令重新配置rac的监听文件即可

并最终确认监听已经注册到crs中：
[oracle@rac3 admin]$ crs_stat -t -v
Name Type R/RA F/FT Target State Host
———————————————————————-
ora….C3.lsnr application 0/5 0/0 ONLINE ONLINE rac3
ora.rac3.gsd application 0/5 0/0 ONLINE ONLINE rac3
ora.rac3.ons application 0/3 0/0 ONLINE ONLINE rac3
ora.rac3.vip application 0/0 0/0 ONLINE ONLINE rac3
ora….C4.lsnr application 0/5 0/0 ONLINE ONLINE rac4
ora.rac4.gsd application 0/5 0/0 ONLINE ONLINE rac4
ora.rac4.ons application 0/3 0/0 ONLINE ONLINE rac4
ora.rac4.vip application 0/0 0/0 ONLINE ONLINE rac4

此时CRS的3个核心进程资源ONS、GSD、VIP以及Listener资源都已经online了，还需要把ASM、数据库实例注册到OCR；
在添加之前，我们需要把OCR重启一下
[oracle@rac3 ~]$ srvctl stop nodeapps -n rac3
[oracle@rac3 ~]$ srvctl stop nodeapps -n rac4
[root@rac3 bin]# ./crsctl stop crs
Stopping resources.
Successfully stopped CRS resources
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.

[root@rac3 bin]# ./crsctl check crs
Failure 1 contacting CSS daemon
Cannot communicate with CRS
Cannot communicate with EVM

[root@rac4 bin]# ./crsctl stop crs
Stopping resources.
Successfully stopped CRS resources
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.

[root@rac4 bin]# ./crsctl check crs
Failure 1 contacting CSS daemon
Cannot communicate with CRS
Cannot communicate with EVM

[root@rac3 bin]# ./crsctl start crs
Attempting to start CRS stack
The CRS stack will be started shortly

[root@rac4 bin]# ./crsctl start crs
Attempting to start CRS stack
The CRS stack will be started shortly

[oracle@rac3 ~]$ crs_stat -t -v
Name Type R/RA F/FT Target State Host
———————————————————————-
ora….C3.lsnr application 0/5 0/0 ONLINE ONLINE rac3
ora.rac3.gsd application 0/5 0/0 ONLINE ONLINE rac3
ora.rac3.ons application 0/3 0/0 ONLINE ONLINE rac3
ora.rac3.vip application 0/0 0/0 ONLINE ONLINE rac3
ora….C4.lsnr application 0/5 0/0 ONLINE ONLINE rac4
ora.rac4.gsd application 0/5 0/0 ONLINE ONLINE rac4
ora.rac4.ons application 0/3 0/0 ONLINE ONLINE rac4
ora.rac4.vip application 0/0 0/0 ONLINE ONLINE rac4

7.下面添加ASM
[oracle@rac3 ~]$ srvctl add asm -n rac3 -i +ASM1 -o $ORACLE_HOME
[oracle@rac3 ~]$ srvctl add asm -n rac4 -i +ASM2 -o $ORACLE_HOME
添加完成，注意这里的$ORACLE_HOME使用的是database的，而不是$ORA_CRS_HOME
[oracle@rac3 admin]$ crs_stat -t -v
Name Type R/RA F/FT Target State Host
———————————————————————-
ora….SM1.asm application 0/5 0/0 OFFLINE OFFLINE
ora….C3.lsnr application 0/5 0/0 ONLINE ONLINE rac3
ora.rac3.gsd application 0/5 0/0 ONLINE ONLINE rac3
ora.rac3.ons application 0/3 0/0 ONLINE ONLINE rac3
ora.rac3.vip application 0/0 0/0 ONLINE ONLINE rac3
ora….SM2.asm application 0/5 0/0 OFFLINE OFFLINE
ora….C4.lsnr application 0/5 0/0 ONLINE ONLINE rac4
ora.rac4.gsd application 0/5 0/0 ONLINE ONLINE rac4
ora.rac4.ons application 0/3 0/0 ONLINE ONLINE rac4
ora.rac4.vip application 0/0 0/0 ONLINE ONLINE rac4

重启OCR，步骤同上，略

启动ASM实例，并监控/oracle/db/admin/+ASM/bdump/alert*.log和/oracle/db/admin/ccupdb/bdump/alert_ccupdb1.log日志
[oracle@rac3 rac3]$ srvctl start asm -n rac3
节点rac3正常启动asm实例
启动rac4节点的asm实例时报错了：
[oracle@rac3 rac3]$ srvctl start asm -n rac4
PRKS-1009 : 无法启动节点 “rac4″, [PRKS-1009 : 无法启动节点 "rac4", [CRS-0215: Could not start resource 'ora.rac4.ASM2.asm'.] 上的 ASM 实例 “+ASM2″] 上的 ASM 实例 “+ASM2″
[PRKS-1009 : 无法启动节点 "rac4", [CRS-0215: Could not start resource 'ora.rac4.ASM2.asm'.] 上的 ASM 实例 “+ASM2″]

分别查看rac3、rac4节点的ASM实例的alertr日志/oracle/db/admin/+ASM/bdump/alert*.log
Tue Apr 16 02:04:50 2013
Error: KGXGN polling error (15)
Tue Apr 16 02:04:50 2013
Errors in file /oracle/db/admin/+ASM/bdump/+asm2_lmon_15526.trc:
ORA-29702: error occurred in Cluster Group Service operation
LMON: terminating instance due to error 29702
Tue Apr 16 02:04:52 2013
Shutting down instance (abort)
License high water mark = 0
Tue Apr 16 02:04:54 2013
Instance terminated by LMON, pid = 15526
Tue Apr 16 02:04:57 2013
Instance terminated by USER, pid = 20339
—信息ORA-29702: error occurred in Cluster Group Service operation

—最重要的信息是这里：
WARNING: No cluster interconnect has been specified. Depending on
the communication driver configured Oracle cluster traffic
may be directed to the public interface of this machine.
Oracle recommends that RAC clustered databases be configured
with a private interconnect for enhanced security and
performance.
—这是因为RAC无法确认使用哪个网卡作为 private interconnect ，因此需要在ASM实例启动的pfile文件中指定

—修改两个节点ASM实例的pfile( init+ASM1.ora -> /oracle/db/admin/+ASM/pfile/init.ora)，添加如下内容:
+ASM1.cluster_interconnects=’10.1.1.3′
+ASM2.cluster_interconnects=’10.1.1.4′
—就是将私有网络心跳ip填进去，再使用srvctl重启asm实例即可

[oracle@rac3 admin]$ crs_stat -t -v
Name Type R/RA F/FT Target State Host
———————————————————————-
ora….SM1.asm application 0/5 0/0 ONLINE ONLINE rac3
ora….C3.lsnr application 0/5 0/0 ONLINE ONLINE rac3
ora.rac3.gsd application 0/5 0/0 ONLINE ONLINE rac3
ora.rac3.ons application 0/3 0/0 ONLINE ONLINE rac3
ora.rac3.vip application 0/0 0/0 ONLINE ONLINE rac3
ora….SM2.asm application 0/5 0/0 ONLINE ONLINE rac4
ora….C4.lsnr application 0/5 0/0 ONLINE ONLINE rac4
ora.rac4.gsd application 0/5 0/0 ONLINE ONLINE rac4
ora.rac4.ons application 0/3 0/0 ONLINE ONLINE rac4
ora.rac4.vip application 0/0 0/0 ONLINE ONLINE rac4

8.添加数据库对象
[oracle@rac3 admin]$ srvctl add database -d ccupdb -o $ORACLE_HOME

9.添加两个实例对象
[oracle@rac3 admin]$ srvctl add instance -d ccupdb -i ccupdb1 -n rac3
[oracle@rac3 admin]$ srvctl add instance -d ccupdb -i ccupdb2 -n rac4
[oracle@rac3 admin]$ crs_stat -t -v
Name Type R/RA F/FT Target State Host
———————————————————————-
ora….b1.inst application 0/5 0/0 OFFLINE OFFLINE
ora….b2.inst application 0/5 0/0 OFFLINE OFFLINE
ora.ccupdb.db application 0/0 0/1 OFFLINE OFFLINE
ora….SM1.asm application 0/5 0/0 ONLINE ONLINE rac3
ora….C3.lsnr application 0/5 0/0 ONLINE ONLINE rac3
ora.rac3.gsd application 0/5 0/0 ONLINE ONLINE rac3
ora.rac3.ons application 0/3 0/0 ONLINE ONLINE rac3
ora.rac3.vip application 0/0 0/0 ONLINE ONLINE rac3
ora….SM2.asm application 0/5 0/0 ONLINE ONLINE rac4
ora….C4.lsnr application 0/5 0/0 ONLINE ONLINE rac4
ora.rac4.gsd application 0/5 0/0 ONLINE ONLINE rac4
ora.rac4.ons application 0/3 0/0 ONLINE ONLINE rac4
ora.rac4.vip application 0/0 0/0 ONLINE ONLINE rac4

10.启动数据库
[oracle@rac3 admin]$ srvctl start database -d ccupdb
并关注数据库的alert日志，发现同上相同的错误信息，使用如下语句修改：

SQL> alter system set cluster_interconnects = ’10.1.1.3′ scope=spfile sid=’ccupdb1′;

System altered.

SQL> alter system set cluster_interconnects = ’10.1.1.4′ scope=spfile sid=’ccupdb2′;

System altered.

—注释：此处修改了spfile，建议也同步修改pfile的备份文件，添加如下：
ccupdb1.cluster_interconnects = ’10.1.1.3′
ccupdb2.cluster_interconnects = ’10.1.1.4′

并重启实例2
[oracle@rac3 admin]$ srvctl start instance -d ccupdb -i ccupdb2
[oracle@rac3 admin]$ crs_stat -t -v
Name Type R/RA F/FT Target State Host
———————————————————————-
ora….b1.inst application 1/5 0/0 ONLINE ONLINE rac3
ora….b2.inst application 0/5 0/0 ONLINE ONLINE rac4
ora.ccupdb.db application 0/0 0/1 ONLINE ONLINE rac4
ora….SM1.asm application 0/5 0/0 ONLINE ONLINE rac3
ora….C3.lsnr application 0/5 0/0 ONLINE ONLINE rac3
ora.rac3.gsd application 0/5 0/0 ONLINE ONLINE rac3
ora.rac3.ons application 0/3 0/0 ONLINE ONLINE rac3
ora.rac3.vip application 0/0 0/0 ONLINE ONLINE rac3
ora….SM2.asm application 0/5 0/0 ONLINE ONLINE rac4
ora….C4.lsnr application 0/5 0/0 ONLINE ONLINE rac4
ora.rac4.gsd application 0/5 0/0 ONLINE ONLINE rac4
ora.rac4.ons application 0/3 0/0 ONLINE ONLINE rac4
ora.rac4.vip application 0/0 0/0 ONLINE ONLINE rac4

至此OCR恢复完成。

http://www.oraclers.cn

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/26725802/viewspace-769440/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/26725802/viewspace-769440/