In this Document
Goal
Solution
References
Applies to:
Oracle Server - Enterprise Edition -
Version: 11.2.0.1 to 11.2.0.2 [Release: 11.2 to
11.2]
Microsoft Windows
(32-bit)
Microsoft Windows x64
(64-bit)
Goal
The goal of this document is to provide
the steps necessary to restore the OCR after the 1st ASM diskgroup,
which contains the OCR, Voting Disks, and ASM SPFILE is
lost.
The general process of restoring the OCR
from backup is documented here: Restoring Oracle Cluster
Repository.
Solution
When using an ASM disk group for CRS there
are typically 3 different types of files located in the disk group
that potentially need to be restored/recreated:
The Oracle Cluster Registry file
(OCR)
The Voting file(s)
The shared SPFILE for the ASM
instance(s)
1. Locate the latest automatic OCR
backup
When using a non-shared CRS home,
automatic OCR backups can be located on any node of the cluster,
consequently all nodes need to be checked for the most recent
backup:
$CRS_HOME\bin\ocrconfig
-showbackup
PROT-26: Oracle Cluster Registry backup
locations were retrieved from a local copy
RACNODE01 2010/08/20 12:25:00
C:\app\11.2.0\grid\cdata\cluster_name\backup00.ocr
RACNODE01 2010/08/20 08:24:58
C:\app\11.2.0\grid\cdata\cluster_name\backup01.ocr
RACNODE01 2010/08/20 04:24:55
C:\app\11.2.0\grid\cdata\cluster_name\backup02.ocr
RACNODE01 2010/08/19 04:24:41
C:\app\11.2.0\grid\cdata\cluster_name\day.ocr
RACNODE01 2010/08/09 04:22:07
C:\app\11.2.0\grid\cdata\cluster_name\week.ocr
2. Clear the partitions of the disks that
were used in the 1st diskgroup:
$CRS_HOME\bin\logpartformat /q
\\.\ORCLDISKOCRVOTE0
$CRS_HOME\bin\logpartformat /q
\\.\ORCLDISKOCRVOTE1
$CRS_HOME\bin\logpartformat /q
\\.\ORCLDISKOCRVOTE2
3. Re-Stamp the 1st diskgroup's (+OCRVOTE)
disks using ASMTOOLG:
4. Open the server services console: Start
--> Run --> services.msc
5. Manually stop the OracleOHService
service
6. Start the CRS stack in exclusive
mode:
11.2.0.1:
$CRS_HOME\bin\crsctl start crs
-excl
...
CRS-2672: Attempting to start 'ora.asm' on
'racnode1'
CRS-2676: Start of 'ora.asm' on 'racnode1'
succeeded
CRS-2672: Attempting to start 'ora.crsd'
on 'racnode1'
CRS-2676: Start of 'ora.crsd' on
'racnode1' succeeded
11.2.0.2:
$CRS_HOME\bin\crsctl start crs -excl
-nocrs
...
CRS-2672: Attempting to start
'ora.cluster_interconnect.haip' on 'racnode1'
CRS-2672: Attempting to start 'ora.ctssd'
on 'racnode1'
CRS-2676: Start of 'ora.drivers.acfs' on
'racnode1' succeeded
CRS-2676: Start of 'ora.ctssd' on
'racnode1' succeeded
CRS-2676: Start of
'ora.cluster_interconnect.haip' on 'racnode1'
succeeded
CRS-2672: Attempting to start 'ora.asm' on
'racnode1'
CRS-2676: Start of 'ora.asm' on 'racnode1'
succeeded
IMPORTANT:
A new option '-nocrs' has been introduced
with 11.2.0.2, which prevents the start of the
ora.crsd resource. It is vital that this option is specified,
otherwise the failure to start the ora.crsd resource will tear down
ora.cluster_interconnect.haip, which in turn will cause ASM to
crash.
7. Re-create the OCRVOTE diskgroup via
sqlplus
The disk group can now be (re-)created via
sqlplus from the grid user. The compatible.asm attribute must be
set to 11.2 in order for the disk group to be used by
CRS:
sqlplus / as sysasm
SQL*Plus: Release 11.2.0.1.0 Production on
Thu Sep 2 08:39:18 2010
Copyright (c) 1982, 2010, Oracle. All
rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition
Release 11.2.0.1.0 - 64bit Production
With the Real Application Clusters and
Automatic Storage Management options
SQL> CREATE DISKGROUP OCRVOTE NORMAL
REDUNDANCY
DISK
'\\.\ORCLDISKOCRVOTE0','\\.\ORCLDISKOCRVOTE1','\\.\ORCLDISKOCRVOTE2'
ATTRIBUTE
'compatible.asm'='11.2.0.0.0';
Diskgroup created.
SQL> exit
8. Restore the latest OCR
backup
Now that the CRS disk group is created
& mounted the OCR can be restored:
$CRS_HOME\bin\ocrconfig -restore
C:\app\11.2.0\grid\cdata\cluster_name\backup00.ocr
9. Start the CRS daemon on the current
node (11.2.0.1 only !)
Now that the OCR has been restored the CRS
daemon can be started, this is needed to recreate the Voting file.
Skip this step for 11.2.0.2.0.
$CRS_HOME\bin\crsctl start res ora.crsd
-init
CRS-2672: Attempting to start 'ora.crsd'
on 'racnode1'
CRS-2676: Start of 'ora.crsd' on
'racnode1' succeeded
10. Recreate the Voting
file
The Voting file needs to be initialized in
the CRS disk group:
$CRS_HOME\bin\crsctl replace votedisk
+OCRVOTE
Successful addition of voting disk
401738d9d75e4f7abf60d6bac7f9119c.
Successful addition of voting disk
7449ffaed19b4f22bf643553ea396683.
Successful addition of voting disk
5be68de758a74fbbbf9b8a4240d5e32d.
Successfully replaced voting disk group
with +OCRVOTE.
CRS-4266: Voting file(s) successfully
replaced
11. Recreate the SPFILE for
ASM
Prepare a pfile (e.g.
c:\tmp\asm_pfile.ora) with the ASM startup parameters - these may
vary from the example below. If in doubt consult the ASM alert log
as the ASM instance startup should list all
non-default parameter values. Please note the last startup of ASM
(in step 2 via CRS start) will not have used an SPFILE, so a
startup prior to the loss of the CRS disk group would need to be
located.
*.asm_power_limit=1
*.diagnostic_dest='C:\app\11.2.0\grid'
*.instance_type='asm'
*.large_pool_size=12M
*.remote_login_passwordfile='EXCLUSIVE'
Now the SPFILE can be created using this
PFILE:
sqlplus / as sysasm
SQL*Plus: Release 11.2.0.1.0 Production on
Tue Mar 30 11:52:39 2010
Copyright (c) 1982, 2009, Oracle. All
rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition
Release 11.2.0.1.0 - Production
With the Real Application Clusters and
Automatic Storage Management options
SQL> create spfile='+OCRVOTE' from
pfile='c:\tmp\asm_pfile.ora';
File created.
SQL> exit
12. shutdown
CRS
Since CRS is running in exclusive mode, it
needs to be shutdown to allow CRS to run on all
nodes again. Use of the force (-f) option may be
required:
$CRS_HOME/bin/crsctl stop crs
-f
...
CRS-2793: Shutdown of Oracle High
Availability Services-managed resources on 'auw2k3' has
completed
CRS-4133: Oracle High Availability
Services has been stopped.
13. start
CRS
As the root user submit the CRS startup on
all cluster nodes:
# $GRID_HOME/bin/crsctl start
crs
CRS-4123: Oracle High Availability
Services has been started.
** This step can take several minutes to
run as a result of the changes we have made to the RAC
configuration. Please wait about 5 minutes, and then check that all
the CRS services are running:
$CRS_HOME\bin\crsctl status resource
-t
14. Verify CRS
$CRS_HOME\bin\crsctl check cluster
-all
$CRS_HOME\bin\crsctl status resource
-t
Errors:
PROT-602: Failed to retrieve data from the
cluster registry
PROC-26: Error while accessing the
physical storage ASM error [SLOS: cat=8, pn=kgfolclcpi1, dep=210,
loc=kgfokge
AMDU-00210: No disks found in diskgroup
OCRVOTE
AMDU-00210: No disks found in diskgroup
OCRVOTE