RDBMS 11.2.0.4
参考文档:
OCR/Vote disk 维护操作: (添加/删除/替换/移动) (Doc ID 1674859.1)
CRS-4258: Addition and Deletion of Voting Files are not Allowed Because the Voting Files are on ASM in 11gR2 (Doc ID 1060146.1)
1 查看当前ocr的备份
ocrconfig -showbackup
ocrconfig -manualbackup
[grid@o11gr21 ~]$ ocrconfig -showbackup
o11gr21 2019/08/24 13:50:40 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/backup00.ocr
o11gr21 2019/08/10 13:55:54 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/backup01.ocr
o11gr21 2019/08/24 13:50:40 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/day.ocr
o11gr21 2019/08/24 13:50:40 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/week.ocr
PROT-25: Manual backups for the Oracle Cluster Registry are not available
[grid@o11gr21 ~]$
[grid@o11gr21 ~]$ ocrconfig -manualbackup
PROT-20: Insufficient permission to proceed. Require privileged user
[grid@o11gr21 ~]$ exit
logout
[root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/ocrconfig -manualbackup
o11gr22 2022/03/18 16:45:11 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/backup_20220318_164511.ocr
[root@o11gr21 ~]#
2 检查ocr是否健康,可以看到有Device/File not configured。表示可以最多添加4个ocrmirrors
ocrcheck
[root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3304
Available space (kbytes) : 258816
ID : 953786240
Device/File Name : +OCR
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
[root@o11gr21 ~]#
3 添加一个ocr mirror设备,这里把/dev/asm-test6添加进去,可以看到多了一个/dev/asm-test6
ocrconfig -add /dev/asm-test6
[root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/ocrconfig -add /dev/asm-test6
[root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3304
Available space (kbytes) : 258816
ID : 953786240
Device/File Name : +OCR
Device/File integrity check succeeded
Device/File Name : /dev/asm-test6
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
[root@o11gr21 ~]#
4 替换或移动ocr设备位置, 使用/dev/asm-test5 ,可以看到原来的ocr由 /dev/asm-test6替换成了/asm-test5
ocrconfig -replace /dev/asm-test6 -replacement /dev/asm-test5
[root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/ocrconfig -replace /dev/asm-test6 -replacement /dev/asm-test5
[root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3304
Available space (kbytes) : 258816
ID : 953786240
Device/File Name : +OCR
Device/File integrity check succeeded
Device/File Name : /dev/asm-test5
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
[root@o11gr21 ~]#
5 查看当前vote disk
[root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 426085b261694f1cbf44c4a054c183f0 (/dev/asm-ocr1) [OCR]
2. ONLINE 0fd49f1e7e804f3abf4a67d45271b2ce (/dev/asm-ocr2) [OCR]
3. ONLINE 4d145b12b6024fe7bfeb0730de5c53a7 (/dev/asm-ocr3) [OCR]
Located 3 voting disk(s).
[root@o11gr21 ~]#
6 添加一个vote disk。 将/dev/asm-test4添加到votedisk ,报错,可以参考CRS-4258: Addition and Deletion of Voting Files are not Allowed Because the Voting Files are on ASM in 11gR2 (Doc ID 1060146.1)
crsctl add css votedisk /dev/asm-test4
[grid@o11gr21 ~]$ crsctl add css votedisk /dev/asm-test4
CRS-4258: Addition and deletion of voting files are not allowed because the voting files are on ASM
[grid@o11gr21 ~]$
WORKAROUND:
Do something to trigger ASM to try to relocate the voting file.
e.g) $ crsctl replace votedisk +asm_disk_group — Put available ASM diskgroup
$ crsctl query css votedisk — Check if voting files are all online on the new ASM diskgroup
$ crsctl replace votedisk + – Put the original ASM diskgroup where voting files were
创建一个磁盘组,vote_test
create diskgroup vote_test external redundancy disk '/dev/asm-test4';
SQL> create diskgroup vote_test external redundancy disk '/dev/asm-test4';
Diskgroup created.
SQL>
进行replance操作
crsctl replace votedisk +vote_test
crsctl add css votedisk +vote_test -- 这个在11.2中已经不能执行了。具体可以参考文档中的解释
b. 当 votedisk 位于 ASM 磁盘组,没有添加选项。
Votedisk 数量是由磁盘组冗余度决定。如果希望更多 votedisk 副本,可以移动 votedisk 到更高冗余度的磁盘组。见步骤4。
如果 votedisk 从一个正常或者高冗余度的磁盘组被删除,可以使用下面的命令添加回来:
[root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/crsctl replace votedisk +vote_test
Failed to create voting files on disk group vote_test.
Change to configuration failed, but was successfully rolled back.
CRS-4000: Command Replace failed, or completed with errors.
[root@o11gr21 ~]#
NOTE: cache ending mount (success) of group VOTE_TEST number=3 incarn=0xdb08899f
Fri Mar 18 17:11:36 2022
NOTE: Instance updated compatible.asm to 10.1.0.0.0 for grp 3
SUCCESS: diskgroup VOTE_TEST was mounted
Fri Mar 18 17:11:37 2022
NOTE: diskgroup resource ora.VOTE_TEST.dg is online
Fri Mar 18 17:11:37 2022
SUCCESS: create diskgroup vote_test external redundancy disk '/dev/asm-test4'
ERROR: failed to update diskgroup resource ora.VOTE_TEST.dg
Fri Mar 18 17:12:13 2022
NOTE: enlarging ACD to 3 threads for group 3/0xdb08899f (VOTE_TEST)
SUCCESS: ACD enlarged for group 3/0xdb08899f (VOTE_TEST)
Fri Mar 18 17:14:23 2022
NOTE: [crsctl.bin@o11gr21 (TNS V1-V3) 11778] opening OCR file
Fri Mar 18 17:14:24 2022
NOTE: updated gpnp profile ASM diskstring: /dev/asm*
Fri Mar 18 17:14:24 2022
NOTE: Creating voting files in diskgroup VOTE_TEST
Fri Mar 18 17:14:24 2022
NOTE: Voting File refresh pending for group 3/0xdb08899f (VOTE_TEST)
NOTE: Attempting voting file creation in diskgroup VOTE_TEST
ERROR: Voting file allocation failed for group VOTE_TEST
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_11790.trc:
ORA-15221: ASM operation requires compatible.asm of 11.2.0.0.0 or higher
NOTE: Attempting voting file refresh on diskgroup VOTE_TEST
NOTE: Refresh completed on diskgroup VOTE_TEST. No voting file found.
Fri Mar 18 17:18:50 2022
NOTE: [crsctl.bin@o11gr21 (TNS V1-V3) 12459] opening OCR file
Fri Mar 18 17:18:50 2022
NOTE: updated gpnp profile ASM diskstring: /dev/asm*
Fri Mar 18 17:18:50 2022
NOTE: Creating voting files in diskgroup VOTE_TEST
Fri Mar 18 17:18:50 2022
NOTE: Voting File refresh pending for group 3/0xdb08899f (VOTE_TEST)
NOTE: Attempting voting file creation in diskgroup VOTE_TEST
ERROR: Voting file allocation failed for group VOTE_TEST
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12467.trc:
ORA-15221: ASM operation requires compatible.asm of 11.2.0.0.0 or higher
NOTE: Attempting voting file refresh on diskgroup VOTE_TEST
NOTE: Refresh completed on diskgroup VOTE_TEST. No voting file found.
– 查看磁盘组的兼容性,是10.1.0.0.0 。改成11.2
SQL> set linesize 200
SQL> select inst_id, name, state, type, free_mb, substr(compatibility,1,10) compatibility from gv$asm_diskgroup;
INST_ID NAME STATE TYPE FREE_MB COMPATIBIL
---------- ------------------------------ ----------- ------ ---------- ----------
2 DATA MOUNTED NORMAL 187744 11.2.0.0.0
2 OCR MOUNTED NORMAL 5089 11.2.0.0.0
2 VOTE_TEST MOUNTED EXTERN 1912 10.1.0.0.0
1 DATA MOUNTED NORMAL 187744 11.2.0.0.0
1 OCR MOUNTED NORMAL 5089 11.2.0.0.0
1 VOTE_TEST MOUNTED EXTERN 1912 10.1.0.0.0
3 DATA MOUNTED NORMAL 187744 11.2.0.0.0
3 OCR MOUNTED NORMAL 5089 11.2.0.0.0
3 VOTE_TEST MOUNTED EXTERN 1912 10.1.0.0.0
9 rows selected.
SQL>
– 修改兼容性
ALTER DISKGROUP VOTE_TEST set ATTRIBUTE 'compatible.asm' = '11.2.0.0.0';
SQL> ALTER DISKGROUP VOTE_TEST set ATTRIBUTE 'compatible.asm' = '11.2.0.0.0';
Diskgroup altered.
SQL> select inst_id, name, state, type, free_mb, substr(compatibility,1,10) compatibility from gv$asm_diskgroup;
INST_ID NAME STATE TYPE FREE_MB COMPATIBIL
---------- ------------------------------ ----------- ------ ---------- ----------
3 DATA MOUNTED NORMAL 187744 11.2.0.0.0
3 OCR MOUNTED NORMAL 5089 11.2.0.0.0
3 VOTE_TEST MOUNTED EXTERN 1910 11.2.0.0.0
2 DATA MOUNTED NORMAL 187744 11.2.0.0.0
2 OCR MOUNTED NORMAL 5089 11.2.0.0.0
2 VOTE_TEST MOUNTED EXTERN 1910 11.2.0.0.0
1 DATA MOUNTED NORMAL 187744 11.2.0.0.0
1 OCR MOUNTED NORMAL 5089 11.2.0.0.0
1 VOTE_TEST MOUNTED EXTERN 1910 11.2.0.0.0
9 rows selected.
SQL>
– 重新 replace votedisk ,replace成功 。
crsctl replace votedisk +vote_test
[grid@o11gr21 trace]$ crsctl replace votedisk +vote_test
Successful addition of voting disk 332ff99c6ff94f99bfbf079250061239.
Successful deletion of voting disk 426085b261694f1cbf44c4a054c183f0.
Successful deletion of voting disk 0fd49f1e7e804f3abf4a67d45271b2ce.
Successful deletion of voting disk 4d145b12b6024fe7bfeb0730de5c53a7.
Successfully replaced voting disk group with +vote_test.
CRS-4266: Voting file(s) successfully replaced
[grid@o11gr21 trace]$
[grid@o11gr21 trace]$ crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 332ff99c6ff94f99bfbf079250061239 (/dev/asm-test4) [VOTE_TEST]
Located 1 voting disk(s).
[grid@o11gr21 trace]$
– 检查,发现ocr盘是+OCR和/dev/asm-test5 ,votedisk是/dev/asm-test4
[grid@o11gr21 trace]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3324
Available space (kbytes) : 258796
ID : 953786240
Device/File Name : +OCR
Device/File integrity check succeeded
Device/File Name : /dev/asm-test5
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check bypassed due to non-privileged user
[grid@o11gr21 trace]$ crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 332ff99c6ff94f99bfbf079250061239 (/dev/asm-test4) [VOTE_TEST]
Located 1 voting disk(s).
[grid@o11gr21 trace]$
–将ocr盘变更为+OCR ,也就是删除掉OCR盘 /dev/asm-test5
ocrconfig -delete /dev/asm-test5
[root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/ocrconfig -delete /dev/asm-test5
[root@o11gr21 ~]#
– 将vote disk盘重新替换为+OCR
crsctl replace votedisk +OCR
[grid@o11gr21 trace]$ ocrconfig -delete /dev/asm-test5
PROT-20: Insufficient permission to proceed. Require privileged user
[grid@o11gr21 trace]$ crsctl replace votedisk +OCR
Successful addition of voting disk 2cb64f80fd974f61bf9ce6fa0f0bfbde.
Successful addition of voting disk 8a44f55e4f8f4f8dbf3dbe890877a857.
Successful addition of voting disk 81ffe4523d5b4f24bf14083103925201.
Successful deletion of voting disk 332ff99c6ff94f99bfbf079250061239.
Successfully replaced voting disk group with +OCR.
CRS-4266: Voting file(s) successfully replaced
[grid@o11gr21 trace]$
– 查询结果,恢复到原来的状态了
[grid@o11gr21 trace]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3324
Available space (kbytes) : 258796
ID : 953786240
Device/File Name : +OCR
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check bypassed due to non-privileged user
[grid@o11gr21 trace]$ crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 2cb64f80fd974f61bf9ce6fa0f0bfbde (/dev/asm-ocr1) [OCR]
2. ONLINE 8a44f55e4f8f4f8dbf3dbe890877a857 (/dev/asm-ocr2) [OCR]
3. ONLINE 81ffe4523d5b4f24bf14083103925201 (/dev/asm-ocr3) [OCR]
Located 3 voting disk(s).
[grid@o11gr21 trace]$
– 清理掉VOTE_TEST磁盘 ,将vote_test磁盘,mount到3个节点中的其中一个节点上,然后在drop diskgroup 。
SQL> drop diskgroup vote_test;
drop diskgroup vote_test
*
ERROR at line 1:
ORA-15039: diskgroup not dropped
ORA-15001: diskgroup "VOTE_TEST" does not exist or is not mounted
SQL> alter diskgroup vote_test mount;
Diskgroup altered.
SQL> drop diskgroup vote_test;
Diskgroup dropped.
SQL>
END