重建表决磁盘和OCR文件

在没有备份的情况下，重建表决磁盘和OCR文件，创建成功后可以通过SRVCTL工具

重新添加资源到OCR中，下面以两节点RAC为例

1）再节点1执行如下命令清空clusterware的配置信息：

[root@vmac1 grid]# cd $GRID_HOME/crs/install/
[root@vmac1 install]# ./rootcrs.pl -deconfig -force
2015-03-24 18:20:08: Parsing the host name
2015-03-24 18:20:08: Checking for super user privileges
2015-03-24 18:20:08: User has super user privileges
Using configuration parameter file: ./crsconfig_params
VIP exists.:vmac1
VIP exists.: /vmac1-vip/192.168.1.182/255.255.255.0/eth1
VIP exists.:vmac2
VIP exists.: /vmac2-vip/192.168.1.184/255.255.255.0/eth1
GSD exists.
ONS daemon exists. Local port 6100, remote port 6200
eONS daemon exists. Multicast port 17832, multicast IP address 234.120.107.221, listening port 2016
ACFS-9200: Supported
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'vmac1'
CRS-2677: Stop of 'ora.registry.acfs' on 'vmac1' succeeded
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'vmac1'
CRS-2673: Attempting to stop 'ora.crsd' on 'vmac1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'vmac1'
CRS-2673: Attempting to stop 'ora.SYSTEMDG.dg' on 'vmac1'
CRS-2673: Attempting to stop 'ora.orcl.db' on 'vmac1'
CRS-2677: Stop of 'ora.SYSTEMDG.dg' on 'vmac1' succeeded
CRS-2677: Stop of 'ora.orcl.db' on 'vmac1' succeeded
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'vmac1'
CRS-2677: Stop of 'ora.DATA.dg' on 'vmac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'vmac1'
CRS-2677: Stop of 'ora.asm' on 'vmac1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'vmac1' has completed
CRS-2677: Stop of 'ora.crsd' on 'vmac1' succeeded
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'vmac1'
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'vmac1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'vmac1'
CRS-2673: Attempting to stop 'ora.evmd' on 'vmac1'
CRS-2673: Attempting to stop 'ora.asm' on 'vmac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'vmac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'vmac1' succeeded
CRS-2677: Stop of 'ora.evmd' on 'vmac1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'vmac1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'vmac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'vmac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'vmac1'
CRS-2677: Stop of 'ora.cssd' on 'vmac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'vmac1'
CRS-2673: Attempting to stop 'ora.diskmon' on 'vmac1'
CRS-2677: Stop of 'ora.drivers.acfs' on 'vmac1' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'vmac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'vmac1'
CRS-2677: Stop of 'ora.gipcd' on 'vmac1' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'vmac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'vmac1' has completed
CRS-4133: Oracle High Availability Services 已停止。
Successfully deconfigured Oracle clusterware stack on this node
[root@vmac1 install]# 

2）再节点2执行同样的清空clusterware的命令，由于该节点是最后一个节点，所以命令后面加上-lastnode

[root@vmac2 ~]#  cd /g01/11ggrid/app/11.2.0/grid/crs/install/
[root@vmac2 install]# ./rootcrs.pl -deconfig -force -lastnode

3) 再节点1执行root.sh脚本以重建clusterware配置信息

[root@vmac1 install]# cd $GRID_HOME
[root@vmac1 grid]# ./root.sh 
Running Oracle 11g root.sh script...

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /g01/11ggrid/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]: 
The file "dbhome" already exists in /usr/local/bin.  Overwrite it? (y/n) 
[n]: y
   Copying dbhome to /usr/local/bin ...
The file "oraenv" already exists in /usr/local/bin.  Overwrite it? (y/n) 
[n]: y
   Copying oraenv to /usr/local/bin ...
The file "coraenv" already exists in /usr/local/bin.  Overwrite it? (y/n) 
[n]: y
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2015-03-24 18:29:18: Parsing the host name
2015-03-24 18:29:18: Checking for super user privileges
2015-03-24 18:29:18: User has super user privileges
Using configuration parameter file: /g01/11ggrid/app/11.2.0/grid/crs/install/crsconfig_params
LOCAL ADD MODE 
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Adding daemon to inittab
CRS-4123: Oracle High Availability Services 已启动。
ohasd is starting
CRS-2672: Attempting to start 'ora.gipcd' on 'vmac1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'vmac1'
CRS-2676: Start of 'ora.gipcd' on 'vmac1' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'vmac1'
CRS-2676: Start of 'ora.gpnpd' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'vmac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'vmac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'vmac1'
CRS-2676: Start of 'ora.diskmon' on 'vmac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'vmac1'
CRS-2676: Start of 'ora.ctssd' on 'vmac1' succeeded

ASM created and started successfully.

DiskGroup SYSTEMDG created successfully.

clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-2672: Attempting to start 'ora.crsd' on 'vmac1'
CRS-2676: Start of 'ora.crsd' on 'vmac1' succeeded
已成功添加表决磁盘 7159b43ca62a4f4cbf257357edb7d5a9                                                                                                                                                                                                                                。
已成功添加表决磁盘 b1984602f9604ff4bfeab637904d3bbd                                                                                                                                                                                                                                。
已成功添加表决磁盘 8fe7173ed0564f38bfe64d44cc3da792                                                                                                                                                                                                                                。
已成功将表决磁盘组替换为 +SYSTEMDG。
CRS-4266: 已成功替换表决文件
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   7159b43ca62a4f4cbf257357edb7d5a9 (/dev/asm-diskb) [SYSTEMDG]
 2. ONLINE   b1984602f9604ff4bfeab637904d3bbd (/dev/asm-diskc) [SYSTEMDG]
 3. ONLINE   8fe7173ed0564f38bfe64d44cc3da792 (/dev/asm-diskd) [SYSTEMDG]
找到了 3 个表决磁盘。
CRS-2673: Attempting to stop 'ora.crsd' on 'vmac1'
CRS-2677: Stop of 'ora.crsd' on 'vmac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'vmac1'
CRS-2677: Stop of 'ora.asm' on 'vmac1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'vmac1'
CRS-2677: Stop of 'ora.ctssd' on 'vmac1' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'vmac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'vmac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'vmac1'
CRS-2677: Stop of 'ora.cssd' on 'vmac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'vmac1'
CRS-2677: Stop of 'ora.gpnpd' on 'vmac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'vmac1'
CRS-2677: Stop of 'ora.gipcd' on 'vmac1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'vmac1'
CRS-2677: Stop of 'ora.mdnsd' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.mdnsd' on 'vmac1'
CRS-2676: Start of 'ora.mdnsd' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'vmac1'
CRS-2676: Start of 'ora.gipcd' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'vmac1'
CRS-2676: Start of 'ora.gpnpd' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'vmac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'vmac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'vmac1'
CRS-2676: Start of 'ora.diskmon' on 'vmac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'vmac1'
CRS-2676: Start of 'ora.ctssd' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'vmac1'
CRS-2676: Start of 'ora.asm' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'vmac1'
CRS-2676: Start of 'ora.crsd' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'vmac1'
CRS-2676: Start of 'ora.evmd' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'vmac1'
CRS-2676: Start of 'ora.asm' on 'vmac1' succeeded
CRS-2672: Attempting to start 'ora.SYSTEMDG.dg' on 'vmac1'
CRS-2676: Start of 'ora.SYSTEMDG.dg' on 'vmac1' succeeded
<span style="color:#ff0000;">PRCR-1013 : Failed to start resource ora.SYSTEMDG.dg
PRCR-1064 : Failed to start resource ora.SYSTEMDG.dg on node vmac2</span>
CRS-2546: Server 'vmac2' is not online

start diskgroup resource ... failed
CRS-2672: Attempting to start 'ora.registry.acfs' on 'vmac1'
CRS-2676: Start of 'ora.registry.acfs' on 'vmac1' succeeded

vmac1     2015/03/24 18:36:24     /g01/11ggrid/app/11.2.0/grid/cdata/vmac1/backup_20150324_183624.olr
Preparing packages for installation...
cvuqdisk-1.0.7-1
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Updating inventory properties for clusterware
Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 3855 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /g01/oraInventory
'UpdateNodeList' was successful.
[root@vmac1 grid]# 

4）在节点2执行root.sh脚本以重建clusterware配置信息

5) 加载磁盘组

SQL> select name,state from v$asm_diskgroup;

NAME
--------------------------------------------------------------------------------
STATE
---------------------------------
SYSTEMDG
MOUNTED

DATA
DISMOUNTED


SQL> alter diskgroup data mount;

Diskgroup altered.

[root@vmac1 grid]# srvctl status diskgroup -g +systemdg
PRCA-1000 : ASM Disk Group +systemdg does not exist
PRCR-1070 : Failed to check if resource ora.+SYSTEMDG.dg is registered
Invalid resource name
[root@vmac1 grid]# srvctl status diskgroup -g systemdg
Disk Group systemdg is running on vmac1,vmac2
[root@vmac1 grid]# srvctl status diskgroup -g data
PRCR-1001 : Resource ora.DATA.dg does not exist
[root@vmac1 grid]# srvctl status diskgroup -g data
Disk Group data is running on vmac1
[root@vmac1 grid]# srvctl start diskgroup -g data -n vmac2
[root@vmac1 grid]# srvctl status diskgroup -g data
Disk Group data is running on vmac1,vmac2

如果磁盘组在重建OCR之后创建，需要执行SRVCTL ADD DISKGROUP命令将ASM磁盘组的信息注册到OCR中。

6）使用操作系统grid用户执行以下命令，重新注册本地监听器到OCR文件中：

[grid@vmac1 ~]$ srvctl add listener -l listener
PRCN-2061 : Failed to add listener ora.LISTENER.lsnr
PRCN-2065 : Port(s) 1521 are not available on the nodes given
PRCN-2035 : Port 1521 is not available on node(s) or virtual host(s): vmac2-vip

[grid@vmac1 ~]$ srvctl status listener
PRCN-2044 : No listener exists

监听添加不上去

7） 

[oracle@vmac1 ~]$  srvctl add database -d orcl -o /s01/orabase/product/11.2.0/dbhome_1
[oracle@vmac1 ~]$ srvctl add instance -d orcl -i orcl1 -n vmac1
[oracle@vmac1 ~]$  srvctl add instance -d orcl -i orcl2 -n vmac2
[oracle@vmac1 ~]$ srvctl config database -d orcl
Database unique name: orcl
Database name: 
Oracle home: /s01/orabase/product/11.2.0/dbhome_1
Oracle user: oracle
Spfile: 
Domain: 
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools: orcl
Database instances: orcl1,orcl2
Disk Groups: 
Services: 
Database is administrator managed
[oracle@vmac1 ~]$ 

[oracle@vmac2 ~]$ ps -ef | grep ora_ 
oracle   14314 13541  0 19:31 pts/2    00:00:00 grep ora_

此时两实例是down的

[oracle@vmac1 ~]$ srvctl start database -d orcl

[oracle@vmac2 ~]$ srvctl status database -d orcl
Instance orcl1 is running on node vmac1
Instance orcl2 is running on node vmac2