RAC的Diskgroup重建

之前安装时diskgroup使用的是external的冗余级别，现在想把它换成normal的冗余级别，但是由于diskgroup的冗余一经创建就无法调整，只能重建。

整个过程的大概步骤如下：

1、备份已有的数据库DCDB，并伤处asm上的文件

2、新建一个磁盘组SFRA作为过渡

3、迁移原磁盘组DATA上的vote和ocr以及asm的spfile至SFRA

4、删除并重建DATA磁盘组为normal冗余

5、回迁vote和ocr以及asm的spfile至DATA磁盘组

6、恢复数据库

详细过程如下：

1、备份已有数据库，此步略过不做介绍

使用dbca删除数据库。

删除ASM上DCDB数据库的参数文件等：

[grid@srcbdb2 ~]$ asmcmd
ASMCMD> cd +DATA/
ASMCMD> ls
DCDB/

ASMCMD> rm -fr DCDB

过程如同在os上执行。

2、新建一个磁盘组SFRA作为过渡

CREATE DISKGROUP SFRA NORMAL REDUNDANCY  
FAILGROUP DGFG02 DISK '/dev/iscsi/11gdisk41' SIZE 20480M ,
'/dev/iscsi/11gdisk42' SIZE 20480M ,
'/dev/iscsi/11gdisk43' SIZE 20480M  
FAILGROUP DGFG01 DISK '/dev/iscsi/11gdisk31' SIZE 20480M ,
'/dev/iscsi/11gdisk32' SIZE 20480M ,
'/dev/iscsi/11gdisk33' SIZE 20480M  
ATTRIBUTE 'compatible.asm'='11.2.0.0.0',
'au_size'='1M';

Diskgroup created.

检查另一节点上改磁盘组是否mount起来的。

+ASM2@SYS>  select * from v$asm_diskgroup;


GROUP_NUMBER NAME                                                         SECTOR_SIZE BLOCK_SIZE ALLOCATION_UNIT_SIZE STATE                  TYPE           TOTAL_MB FREE_MB HOT_USED_MB COLD_USED_MB
------------ ------------------------------------------------------------ ----------- ---------- -------------------- ---------------------- ------------ ---------- ---------- ----------- ------------
REQUIRED_MIRROR_FREE_MB USABLE_FILE_MB OFFLINE_DISKS COMPATIBILITY
----------------------- -------------- ------------- ------------------------------------------------------------------------------------------------------------------------
DATABASE_COMPATIBILITY                                                                                                   VO
------------------------------------------------------------------------------------------------------------------------ --
           0 SFRA                                                                  0       4096                    0DISMOUNTED                               0           0           0            0
                      0              0             0 0.0.0.0.0
0.0.0.0.0                                                                                                                N


           2 DATA                                                                 512       4096              1048576 MOUNTED                NORMAL           122880  121175           0         1705
                    304          60435             0 11.2.0.0.0
10.1.0.0.0                                                                                                               Y

如果没有自动mount起来，则手工mount改磁盘组

+ASM2@SYS> alter diskgroup sfra mount;

Diskgroup altered.

3、迁移SPFILE，OCR和VOTE文件至SFRA磁盘组

3.1、迁移SPFILE

查看当前asm的spfile位置

+ASM1@SYS> show parameter spfile


NAME                                 TYPE                   VALUE
------------------------------------ ---------------------- ------------------------------
spfile                               string                 +DATA/srcbdb-cluster/asmparame
                                                            terfile/registry.253.788224447

创建一个pfile作为spfile迁移的过渡文件

+ASM1@SYS> create pfile from spfile;

File created.

新建spfile文件并指定到过渡磁盘组
+ASM1@SYS> create spfile='+SFRA' from pfile='?/dbs/init+ASM1.ora';

File created.

+ASM1@SYS> 

spfile已经在SFRA磁盘组中

[grid@srcbdb2 ~]$ asmcmd
ASMCMD> cd +SFRA/srcbdb-cluster/ASMPARAMETERFILE
ASMCMD> ls
REGISTRY.253.788224447

在另一节点上执行同样的操作迁移spfile。

3.2、迁移OCR文件

先检查一下当前OCR文件，可知当前OCR文件存放于DATA磁盘组上

[root@srcbdb1 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2936
         Available space (kbytes) :     259184
         ID                       :  894939142
         Device/File Name         :     +DATA
                                    Device/File integrity check succeeded
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
         Cluster registry integrity check succeeded
         Logical corruption check succeeded

新增一个OCR文件到SFRA磁盘组：

[root@srcbdb1 ~]# ocrconfig -add '+SFRA'     

[root@srcbdb1 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2936
         Available space (kbytes) :     259184
         ID                       :  894939142
         Device/File Name         :      +SFRA
                                    Device/File integrity check succeeded
         Device/File Name         :      +DATA
                                    Device/File integrity check succeeded
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
         Cluster registry integrity check succeeded
         Logical corruption check succeeded

然后从DATA磁盘组中删除OCR文件

[root@srcbdb1 ~]# ocrconfig -delete '+DATA'

[root@srcbdb1 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2936
         Available space (kbytes) :     259184
         ID                       :  894939142
         Device/File Name         :      +SFRA
                                    Device/File integrity check succeeded
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
                                    Device/File not configured
         Cluster registry integrity check succeeded
         Logical corruption check succeeded

3.3、迁移VOTE文件

查看当前votedisk的位置

[root@srcbdb2 ~]# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   0524c6e01d244f52bf755b2d6565f576 (/dev/iscsi/11gdisk21) [DATA]
 Located 3 voting disk(s).

停止节点1（其中一个即可）上的crs：

[root@srcbdb1 ~]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'srcbdb1'
CRS-2673: Attempting to stop 'ora.crsd' on 'srcbdb1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'srcbdb1'
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'srcbdb1'
CRS-2673: Attempting to stop 'ora.SFRA.dg' on 'srcbdb1'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'srcbdb1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'srcbdb1'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'srcbdb1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'srcbdb1' succeeded
CRS-2673: Attempting to stop 'ora.srcbdb1.vip' on 'srcbdb1'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'srcbdb1' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'srcbdb1'
CRS-2677: Stop of 'ora.scan1.vip' on 'srcbdb1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'srcbdb2'
CRS-2677: Stop of 'ora.srcbdb1.vip' on 'srcbdb1' succeeded
CRS-2672: Attempting to start 'ora.srcbdb1.vip' on 'srcbdb2'
CRS-2677: Stop of 'ora.registry.acfs' on 'srcbdb1' succeeded
CRS-2676: Start of 'ora.scan1.vip' on 'srcbdb2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'srcbdb2'
CRS-2676: Start of 'ora.srcbdb1.vip' on 'srcbdb2' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'srcbdb2' succeeded
CRS-2677: Stop of 'ora.DATA.dg' on 'srcbdb1' succeeded
CRS-2677: Stop of 'ora.SFRA.dg' on 'srcbdb1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'srcbdb1'
CRS-2677: Stop of 'ora.asm' on 'srcbdb1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'srcbdb1'
CRS-2677: Stop of 'ora.ons' on 'srcbdb1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'srcbdb1'
CRS-2677: Stop of 'ora.net1.network' on 'srcbdb1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'srcbdb1' has completed
CRS-2677: Stop of 'ora.crsd' on 'srcbdb1' succeeded
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'srcbdb1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'srcbdb1'
CRS-2673: Attempting to stop 'ora.evmd' on 'srcbdb1'
CRS-2673: Attempting to stop 'ora.asm' on 'srcbdb1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'srcbdb1'
CRS-2677: Stop of 'ora.asm' on 'srcbdb1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'srcbdb1'
CRS-2677: Stop of 'ora.evmd' on 'srcbdb1' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'srcbdb1' succeeded
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'srcbdb1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'srcbdb1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'srcbdb1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'srcbdb1'
CRS-2677: Stop of 'ora.cssd' on 'srcbdb1' succeeded
CRS-2673: Attempting to stop 'ora.diskmon' on 'srcbdb1'
CRS-2673: Attempting to stop 'ora.crf' on 'srcbdb1'
CRS-2677: Stop of 'ora.crf' on 'srcbdb1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'srcbdb1'
CRS-2677: Stop of 'ora.gipcd' on 'srcbdb1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'srcbdb1'
CRS-2677: Stop of 'ora.gpnpd' on 'srcbdb1' succeeded
CRS-2677: Stop of 'ora.diskmon' on 'srcbdb1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'srcbdb1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@srcbdb1 ~]# 

把votedisk重建至SFRA磁盘组

[root@srcbdb2 ~]# crsctl replace votedisk '+SFRA'
Successful addition of voting disk 61c5620610934f48bfa1d51faa1dca24.
Successful addition of voting disk a002ef4be5a44f99bf5f87da134af0ef.
Successful addition of voting disk 8300c9c2d1b74f18bf49c7a671f236f2.
Successful deletion of voting disk 0524c6e01d244f52bf755b2d6565f576.
Successful deletion of voting disk 53c5e679b6c74f8abf9def1364ff18cd.
Successful deletion of voting disk 17350898fc3b4f91bfbf8d28768709c2.
Successfully replaced voting disk group with +SFRA.
CRS-4266: Voting file(s) successfully replaced

4、删除DATA磁盘组并重建

删除：

+ASM2@SYS> drop diskgroup data;

Diskgroup dropped.

重建，该磁盘组为NORMAL冗余，并划分为DGFG1，DGFG2两个failuregroup：

CREATE DISKGROUP DATA NORMAL REDUNDANCY  
FAILGROUP DGFG02 DISK 
'/dev/iscsi/11gdisk21' SIZE 20480M ,
'/dev/iscsi/11gdisk22' SIZE 20480M ,
'/dev/iscsi/11gdisk23' SIZE 20480M  
FAILGROUP DGFG01 DISK 
'/dev/iscsi/11gdisk11' SIZE 20480M ,
'/dev/iscsi/11gdisk12' SIZE 20480M ,
'/dev/iscsi/11gdisk13' SIZE 20480M  
ATTRIBUTE 'compatible.asm'='11.2.0.0.0',
'au_size'='1M';

注意：如果要在normal的磁盘组上创建votedisk，则至少需要3个failuregroup；而high的磁盘组需要至少5个failgroup。我这里明显是不符合的，所以我会再创建3个小的磁盘来单独存放votedisk，【省略】。

5、至此DATA磁盘组已重建完毕，使用与3、4步相同的方式把配置迁移至DATA磁盘组上，【步骤通，省略】

6、最后一步就是是把之前的数据库被分恢复出来即可。