记一次Oracle 19C RAC 在线更换数据盘和OCR盘操作记录-CSDN博客

本文链接：https://blog.csdn.net/shlei5580/article/details/136879191

欢迎您关注我的公众号【尚雷的驿站】
****************************************************************************
公众号：尚雷的驿站
CSDN ：https://blog.csdn.net/shlei5580
墨天轮：https://www.modb.pro/u/2436
PGFans：https://www.pgfans.cn/user/home?userId=4159
****************************************************************************

一、前言

最近遇到一个需求，有套NetAPP存储已过保一段时间，目前作为测试环境使用，但最近这套存储出现故障，一条链路状态异常。导致其上配置的多路径有两条链路非Active状态，系统组担心会影响到数据库的使用。最近刚好有套过保的3par存储，在配置和性能方面都要优于这套过保的NetAPP存储，希望我配合进行存储的替换。
目前这套NetAPP存储，目前在其上部署的有4套测试环境，其中有三套都是单机在使用，可以方便替换，其中有套Oracle 19C RAC测试库，虽然是测试库，但当前生产对应的很多测试都是在其上进行验证测试，非常重要，需要提前和项目组沟通，并要避免数据的丢失和损坏。

之前我曾验证过EMC存储和3Par存储的替换，因为EMC日常使用的是EMC自带的存储，3Par存储通常使用操作系统自带multipah多路径软件，在同一台服务器上安装时会有不兼容的情况。
但目前这套NetAPP使用的multipah多路径软件,和3par还是可以兼容的。

为防止在替换过程中会出现一些意外，我这边还是提前做了一些相关备份操作，将多路径软及OCR信息进行了手工备份，并将数据库数据进行了备份。

并通知了项目组，计划在下班后进行操作。

二、环境信息

这套19C RAC库操作系统为Centos 7.9，当前存储使用3个多T。

[root@local-test-xxxx~]# cat /etc/redhat-release 
CentOS Linux release 7.9.2009 (Core)
[root@local-test-mlandb1 ~]# su - grid
Last login: Mon Aug 28 13:29:37 CST 2023
[grid@local-test-xxxx~]$ ./asm/asm 

CURDATE
-------------------
2023-08-28 13:44:07


Disk Group        Sector   Block   Allocation
Name                Size    Size    Unit Size State       Type   Total Size (MB) Used Size (MB)    FREE_MB Pct. Used
---------------- ------- ------- ------------ ----------- ------ --------------- -------------- ---------- ---------
CRS                  512   4,096    4,194,304 MOUNTED     NORMAL          30,672          1,132      29540      3.69
DG                   512   4,096    4,194,304 MOUNTED     EXTERN       3,583,968      3,188,400     395568     88.96
                                                                 --------------- --------------
Grand Total:                                                           3,614,640      3,189,532

ASM使用的是Oracle ASMLIB包安装的ASM磁盘。

[root@local-test-xxxx~]# oracleasm -V
oracleasm version 2.1.11
[root@local-test-xxxx~]# oracleasm listdisks
VOL01
VOL02
VOL03
VOL04
VOL05
VOL06
VOL07

OCR信息如下：

[grid@local-test-xxxx ~]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          4
         Total space (kbytes)     :     491684
         Used space (kbytes)      :      84332
         Available space (kbytes) :     407352
         ID                       :  231456364
         Device/File Name         :       +CRS
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check bypassed due to non-privileged user


SQL>  select GROUP_NUMBER,NAME,TYPE,STATE,total_mb/1024 total_gb,free_mb/1024 free_gb,round((free_mb/total_mb)*100,2)||'%' pct_free from v$asm_diskgroup;
           1 CRS                            NORMAL MOUNTED     23.9882813 22.9648438 95.73%
           2 DG                             EXTERN MOUNTED     7000.41016 3983.82813 56.91%

ASM磁盘信息如下

SQL> select GROUP_NUMBER,free_mb,total_mb,FAILGROUP,disk_number,MOUNT_STATUS,mode_status,STATE,HEADER_STATUS,name,PATH from v$asm_disk order by 4,5; 

           1       6844      10224 CRS_0001                  3 CACHED  ONLINE  NORMAL   MEMBER       CRS_0001        /dev/oracleasm/disks/VOL01
           1       6848      10224 CRS_0002                 4 CACHED  ONLINE  NORMAL   MEMBER       CRS_0002        /dev/oracleasm/disks/VOL02
           1       6848      10224 CRS_0003                 5 CACHED  ONLINE  NORMAL   MEMBER       CRS_0003        /dev/oracleasm/disks/VOL03
           2     291368     512076 DG_00004                  0 CACHED  ONLINE  NORMAL   MEMBER       DG_0000         /dev/oracleasm/disks/VOL04
           2     291368     512076 DG_00005                  1 CACHED  ONLINE  NORMAL   MEMBER       DG_0001         /dev/oracleasm/disks/VOL05
           2     874144    1536192 DG_00006                   3 CACHED  ONLINE  NORMAL   MEMBER       DG_0003         /dev/oracleasm/disks/VOL06
           2     582744    1024108 DG_00007                    2 CACHED  ONLINE  NORMAL   MEMBER       DG_00007       /dev/oracleasm/disks/VOL07

7 rows selected.

**多路径信息备份： **

[root@local-test-xxxx~]# multipath -ll > /root/multipath_xxx.txt

数据库备份,使用expdp对数据库进行导出备份

OCR备份：

-- root用户
[root@local-test-mlandbxxx ~]# /u01/app/19.3.0/grid/bin/ocrconfig -export /tmp/ocr_20230808.dmp

然后通知系统管理员为这套RAC 分配3Par存储。为何原存储进行区分，新分配的存储没块大小都和之前不同，便于区分，存储分配后，先逐台重启了两台数据库服务器识别到新加的存储。
然后使用dd if=/dev/zero of=/dev/mapper/xxx bs=1024k count=10000 对新加的存储进行格式化，然后进行fdisk分区。
之后通过如下oracleasm命令创建新的ASM磁盘，新创建的ASM磁盘名称和原磁盘名称区分开来。

[root@local-test-xxxx~]# oracleasm createdisk VOL1 /dev/mapper/mpathi1 
[root@local-test-xxxx~]# oracleasm createdisk VOL2 /dev/mapper/mpathj1 
[root@local-test-xxxx~]# oracleasm createdisk VOL3 /dev/mapper/mpathk1
[root@local-test-xxxx~]# oracleasm createdisk VOL4 /dev/mapper/mpathl1 
[root@local-test-xxxx~]# oracleasm createdisk VOL5 /dev/mapper/mpathm1 

---然后再二个节点扫盘识别到新加的ASM存储盘。
[root@local-test-xxxx2 ~]# oracleasm scandisks

三、磁盘替换

对于数据盘替换，这块之前已经做了好多次，心里有把握，现将新的ASM磁盘通过如下方式添佳到数据磁盘组：

su - grid
sqlplus /as sysasm
SQL> alter diskgroup DG add disk '/dev/oracleasm/disks/VOL4' rebalance power 11;
SQL> alter diskgroup DG add disk '/dev/oracleasm/disks/VOL5' rebalance power 11;

SQL> SQL> select * from v$asm_operation;

           2 REBAL COMPACT   WAIT         11         11          0          0          0           0                                                       0
           2 REBAL REBALANCE RUN          11         11       5962      71713       3900          16                                                       0
           2 REBAL REBUILD   DONE         11         11          0          0          0           0

替换OCR盘，之前替换10G RAC OCR盘的时候比较麻烦，需要修改的配置较多，但替换19C OCR盘还是第一次，心里也是没有把握，于是查询了网上一些资料，看到JikeXu曾写过 11g https://cloud.tencent.com/developer/article/1696479 OCR替换，不知道19C是不是也可以，并向其进行了请教。

SQL> alter diskgroup CRS  add disk '/dev/oracleasm/disks/VOL1';
alter diskgroup CRS  add disk '/dev/oracleasm/disks/VOL1'
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15410: Disks in disk group CRS do not have equal size.

因为新的OCR盘和原盘大小不同，无法直接添加，恩墨彗星性老师给我一个他曾写的文章：https://www.modb.pro/db/543611 对于ORA-15410 Disks in disk group OCR_VOT do not have equal size 有相应的解释和处理办法。我本次没有去尝试。

于是决定采用JiekeXU提到的方法将全部OCR添加进去并替换掉原有的OCR盘。

su - grid
sqlplus / as sysasm
SQL> alter diskgroup CRS add disk '/dev/oracleasm/disks/VOL1','/dev/oracleasm/disks/VOL2','/dev/oracleasm/disks/VOL3' drop disk 'CRS_0000','CRS_0001','CRS_0002';

Diskgroup altered.

SQL> select GROUP_NUMBER,free_mb,total_mb,FAILGROUP,disk_number,MOUNT_STATUS,mode_status,STATE,HEADER_STATUS,name,PATH from v$asm_disk order by 4,5; 

           1       9844      10224 CRS_0003                  3 CACHED  ONLINE  NORMAL   MEMBER       CRS_0003        /dev/oracleasm/disks/VOL1
           1       9848      10224 CRS_0004                  4 CACHED  ONLINE  NORMAL   MEMBER       CRS_0004        /dev/oracleasm/disks/VOL2
           1       9848      10224 CRS_0005                  5 CACHED  ONLINE  NORMAL   MEMBER       CRS_0005        /dev/oracleasm/disks/VOL3
           2     291368     512076 DG_0000                   0 CACHED  ONLINE  NORMAL   MEMBER       DG_0000         /dev/oracleasm/disks/VOL04
           2     291368     512076 DG_0001                   1 CACHED  ONLINE  NORMAL   MEMBER       DG_0001         /dev/oracleasm/disks/VOL05
           2     874144    1536192 DG_0003                   3 CACHED  ONLINE  NORMAL   MEMBER       DG_0003         /dev/oracleasm/disks/VOL07
           2    1107116    1945584 DG_0004                   4 CACHED  ONLINE  NORMAL   MEMBER       DG_0004         /dev/oracleasm/disks/VOL4
           2     932304    1638384 DG_0005                   5 CACHED  ONLINE  NORMAL   MEMBER       DG_0005         /dev/oracleasm/disks/VOL5
           2     582744    1024108 VOL06                     2 CACHED  ONLINE  NORMAL   MEMBER       VOL06           /dev/oracleasm/disks/VOL06
           0          0          0                           0 CLOSED  ONLINE  NORMAL   FORMER                       /dev/oracleasm/disks/VOL02
           0          0          0                           1 CLOSED  ONLINE  NORMAL   FORMER                       /dev/oracleasm/disks/VOL01
           0          0          0                           2 CLOSED  ONLINE  NORMAL   FORMER                       /dev/oracleasm/disks/VOL03

12 rows selected.

原来三块OCR盘确实都被替换了。

四、附录

4.1 替换OCR盘日志信息

以下是OCR盘替换时查看到的ASM日志信息

替换OCR盘的日志信息如下：
2023-08-08T16:00:10.337765+08:00
SQL> alter diskgroup CRS add disk '/dev/oracleasm/disks/VOL1','/dev/oracleasm/disks/VOL2','/dev/oracleasm/disks/VOL3' drop disk 'CRS_0000','CRS_0001','CRS_0002' 
2023-08-08T16:00:10.339009+08:00
NOTE: GroupBlock outside rolling migration privileged region
NOTE: Assigning number (1,3) to disk (/dev/oracleasm/disks/VOL1)
NOTE: Assigning number (1,4) to disk (/dev/oracleasm/disks/VOL2)
NOTE: Assigning number (1,5) to disk (/dev/oracleasm/disks/VOL3)
NOTE: requesting all-instance membership refresh for group=1
2023-08-08T16:00:11.457532+08:00
NOTE: Attempting voting file relocation on diskgroup CRS
WARNING: Read Failed. group:1 disk:3 AU:0 offset:0 size:4096
path:Unknown disk
         incarnation:0xf0f0ed7e asynchronous result:'I/O error' ioreason:2818 why:11
         subsys:Unknown library krq:0x7f5e6fbe5938 bufp:0x7f5e75183000 osderr1:0x434c5344 osderr2:0x0
         IO elapsed time: 0 usec Time waited on I/O: 0 usec
WARNING: Read Failed. group:1 disk:4 AU:0 offset:0 size:4096
path:Unknown disk
         incarnation:0xf0f0ed7f asynchronous result:'I/O error' ioreason:2818 why:11
         subsys:Unknown library krq:0x7f5e6fbe5490 bufp:0x7f5e75181000 osderr1:0x434c5344 osderr2:0x0
         IO elapsed time: 0 usec Time waited on I/O: 0 usec
WARNING: Read Failed. group:1 disk:5 AU:0 offset:0 size:4096
path:Unknown disk
         incarnation:0xf0f0ed80 asynchronous result:'I/O error' ioreason:2818 why:11
         subsys:Unknown library krq:0x7f5e751809e0 bufp:0x7f5e6fc13000 osderr1:0x434c5344 osderr2:0x0
         IO elapsed time: 0 usec Time waited on I/O: 0 usec
NOTE: Successful voting file relocation on diskgroup CRS
2023-08-08T16:00:12.534839+08:00
NOTE: Disk 3 in group 1 is assigned fgnum=4
NOTE: Disk 4 in group 1 is assigned fgnum=5
NOTE: Disk 5 in group 1 is assigned fgnum=6
NOTE: discarding redo for group 1 disk 3
NOTE: discarding redo for group 1 disk 4
NOTE: discarding redo for group 1 disk 5
NOTE: initializing header (replicated) on grp 1 disk CRS_0003
NOTE: initializing header (replicated) on grp 1 disk CRS_0004
NOTE: initializing header (replicated) on grp 1 disk CRS_0005
NOTE: initializing header on grp 1 disk CRS_0003
NOTE: initializing header on grp 1 disk CRS_0004
NOTE: initializing header on grp 1 disk CRS_0005
NOTE: requesting all-instance disk validation for group=1
2023-08-08T16:00:12.605238+08:00
NOTE: skipping rediscovery for group 1/0xd4a01dae (CRS) on local instance.
2023-08-08T16:00:12.727464+08:00
NOTE: running client discovery for group 1 (reqid:15321278565476053588)
NOTE: requesting all-instance disk validation for group=1
2023-08-08T16:00:12.731123+08:00
NOTE: skipping rediscovery for group 1/0xd4a01dae (CRS) on local instance.
2023-08-08T16:00:12.829109+08:00
NOTE: running client discovery for group 1 (reqid:15321278565476065544)
NOTE: Allocated 1 DD reserve extents for group 1 (CRS)
NOTE: Allocated 6 VAT reserve extents for group 1 (CRS).
NOTE: Adding disk 3 (CRS_0003) to grp 1 (CRS) (2556 AUs)
2023-08-08T16:00:13.804894+08:00
NOTE: Adding disk 4 (CRS_0004) to grp 1 (CRS) (2556 AUs)
2023-08-08T16:00:14.387378+08:00
NOTE: Adding disk 5 (CRS_0005) to grp 1 (CRS) (2556 AUs)
2023-08-08T16:00:14.992385+08:00
GMON updating for reconfiguration, group 1 at 21 for pid 34, osid 30275
2023-08-08T16:00:15.003705+08:00
NOTE: group 1 PST updated.
2023-08-08T16:00:15.068630+08:00
NOTE: membership refresh pending for group 1/0xd4a01dae (CRS)
NOTE: Attempting voting file refresh on diskgroup CRS
NOTE: Refresh completed on diskgroup CRS. Found 3 voting file(s).
NOTE: Voting file relocation is required in diskgroup CRS
2023-08-08T16:00:15.129060+08:00
GMON querying group 1 at 22 for pid 27, osid 6491
NOTE: cache opening disk 3 of grp 1: CRS_0003 path:/dev/oracleasm/disks/VOL1
NOTE: cache opening disk 4 of grp 1: CRS_0004 path:/dev/oracleasm/disks/VOL2
NOTE: cache opening disk 5 of grp 1: CRS_0005 path:/dev/oracleasm/disks/VOL3
2023-08-08T16:00:15.552478+08:00
NOTE: Attempting voting file refresh on diskgroup CRS
NOTE: Refresh completed on diskgroup CRS. Found 3 voting file(s).
NOTE: Voting file relocation is required in diskgroup CRS
NOTE: Attempting voting file relocation on diskgroup CRS
NOTE: Successful voting file relocation on diskgroup CRS
2023-08-08T16:00:15.616116+08:00
GMON querying group 1 at 23 for pid 27, osid 6491
2023-08-08T16:00:15.657735+08:00
SUCCESS: refreshed membership for 1/0xd4a01dae (CRS)
2023-08-08T16:00:15.658273+08:00
SUCCESS: alter diskgroup CRS add disk '/dev/oracleasm/disks/VOL1','/dev/oracleasm/disks/VOL2','/dev/oracleasm/disks/VOL3' drop disk 'CRS_0000','CRS_0001','CRS_0002'
2023-08-08T16:00:15.659229+08:00
NOTE: starting rebalance of group 1/0xd4a01dae (CRS) at power 1
NOTE: starting process ARBA
Starting background process ARBA
2023-08-08T16:00:15.706992+08:00
ARBA started with pid=46, OS id=48198 
NOTE: starting process ARB0
Starting background process ARB0
2023-08-08T16:00:15.729429+08:00
ARB0 started with pid=47, OS id=48200 
NOTE: assigning ARBA to group 1/0xd4a01dae (CRS) to compute estimates
NOTE: assigning ARB0 to group 1/0xd4a01dae (CRS) with 1 parallel I/O
2023-08-08T16:00:16.056816+08:00
NOTE: header on disk 0 advanced to format #2 using fcn 0.0
NOTE: F1X0 on disk 0 (fmt 2) relocated at fcn 0.149388: AU 10 -> AU 0
NOTE: header on disk 1 advanced to format #2 using fcn 0.0
NOTE: F1X0 on disk 1 (fmt 2) relocated at fcn 0.149388: AU 10 -> AU 0
NOTE: header on disk 2 advanced to format #2 using fcn 0.0
NOTE: F1X0 on disk 2 (fmt 2) relocated at fcn 0.149388: AU 10 -> AU 0
NOTE: header on disk 3 advanced to format #2 using fcn 0.0
NOTE: F1X0 on disk 3 (fmt 2) relocated at fcn 0.149388: AU 0 -> AU 10
NOTE: header on disk 4 advanced to format #2 using fcn 0.0
NOTE: F1X0 on disk 4 (fmt 2) relocated at fcn 0.149388: AU 0 -> AU 10
NOTE: header on disk 5 advanced to format #2 using fcn 0.0
NOTE: F1X0 on disk 5 (fmt 2) relocated at fcn 0.149388: AU 0 -> AU 10
NOTE: 08/08/23 16:00:15 CRS.F1X0 copy 1 relocating from 0:10 to 4:10 at FCN 0.149388
NOTE: 08/08/23 16:00:15 CRS.F1X0 copy 2 relocating from 1:10 to 3:10 at FCN 0.149388
NOTE: 08/08/23 16:00:15 CRS.F1X0 copy 3 relocating from 2:10 to 5:10 at FCN 0.149388
2023-08-08T16:00:18.760349+08:00
NOTE: Attempting voting file refresh on diskgroup CRS
NOTE: Refresh completed on diskgroup CRS. Found 3 voting file(s).
NOTE: Voting file relocation is required in diskgroup CRS
NOTE: Attempting voting file relocation on diskgroup CRS
NOTE: voting file allocation (replicated) on grp 1 disk CRS_0003
NOTE: voting file allocation on grp 1 disk CRS_0003
NOTE: voting file allocation (replicated) on grp 1 disk CRS_0004
NOTE: voting file allocation on grp 1 disk CRS_0004
NOTE: voting file allocation (replicated) on grp 1 disk CRS_0005
NOTE: voting file allocation on grp 1 disk CRS_0005
NOTE: voting file deletion (replicated) on grp 1 disk CRS_0000
NOTE: voting file deletion on grp 1 disk CRS_0000
NOTE: voting file deletion (replicated) on grp 1 disk CRS_0001
NOTE: voting file deletion on grp 1 disk CRS_0001
NOTE: voting file deletion (replicated) on grp 1 disk CRS_0002
NOTE: voting file deletion on grp 1 disk CRS_0002
NOTE: Successful voting file relocation on diskgroup CRS
2023-08-08T16:00:21.170262+08:00
NOTE: stopping process ARB0
NOTE: stopping process ARBA
NOTE: Starting expel slave for group 1/0xd4a01dae (CRS)
2023-08-08T16:00:21.201736+08:00
NOTE: GroupBlock outside rolling migration privileged region
NOTE: requesting all-instance membership refresh for group=1
2023-08-08T16:00:21.304682+08:00
GMON updating for reconfiguration, group 1 at 24 for pid 49, osid 48289
2023-08-08T16:00:21.308250+08:00
NOTE: group 1 PST updated.
SUCCESS: grp 1 disk CRS_0000 emptied
SUCCESS: grp 1 disk CRS_0001 emptied
SUCCESS: grp 1 disk CRS_0002 emptied
NOTE: process _x000_+asm1 (48289) initiating offline of disk 0.4042321254 (CRS_0000) with mask 0x7e in group 1 (CRS) without client assisting
NOTE: process _x000_+asm1 (48289) initiating offline of disk 1.4042321255 (CRS_0001) with mask 0x7e in group 1 (CRS) without client assisting
NOTE: process _x000_+asm1 (48289) initiating offline of disk 2.4042321253 (CRS_0002) with mask 0x7e in group 1 (CRS) without client assisting
NOTE: initiating PST update: grp 1 (CRS), dsk = 0/0xf0f0ed66, mask = 0x6a, op = clear mandatory
NOTE: initiating PST update: grp 1 (CRS), dsk = 1/0xf0f0ed67, mask = 0x6a, op = clear mandatory
NOTE: initiating PST update: grp 1 (CRS), dsk = 2/0xf0f0ed65, mask = 0x6a, op = clear mandatory
2023-08-08T16:00:21.335368+08:00
GMON updating disk modes for group 1 at 25 for pid 49, osid 48289
NOTE: group CRS: updated PST location: disks 0003 0004 0005 
2023-08-08T16:00:21.342612+08:00
NOTE: PST update grp = 1 completed successfully 
NOTE: initiating PST update: grp 1 (CRS), dsk = 0/0xf0f0ed66, mask = 0x7e, op = clear mandatory
NOTE: initiating PST update: grp 1 (CRS), dsk = 1/0xf0f0ed67, mask = 0x7e, op = clear mandatory
NOTE: initiating PST update: grp 1 (CRS), dsk = 2/0xf0f0ed65, mask = 0x7e, op = clear mandatory
2023-08-08T16:00:21.343143+08:00
GMON updating disk modes for group 1 at 26 for pid 49, osid 48289
2023-08-08T16:00:21.345868+08:00
NOTE: cache closing disk 0 of grp 1: CRS_0000
2023-08-08T16:00:21.349547+08:00
NOTE: cache closing disk 1 of grp 1: CRS_0001
2023-08-08T16:00:21.349885+08:00
NOTE: cache closing disk 2 of grp 1: CRS_0002
2023-08-08T16:00:21.351878+08:00
NOTE: PST update grp = 1 completed successfully 
NOTE: Expelling disk 0 (CRS_0000) from grp 1 (CRS)
NOTE: Expelling disk 1 (CRS_0001) from grp 1 (CRS)
NOTE: Expelling disk 2 (CRS_0002) from grp 1 (CRS)
2023-08-08T16:00:21.354440+08:00
GMON updating for reconfiguration, group 1 at 27 for pid 49, osid 48289
2023-08-08T16:00:21.354951+08:00
NOTE: cache closing disk 0 of grp 1: (not open) CRS_0000
2023-08-08T16:00:21.355051+08:00
NOTE: cache closing disk 1 of grp 1: (not open) CRS_0001
2023-08-08T16:00:21.355145+08:00
NOTE: cache closing disk 2 of grp 1: (not open) CRS_0002
2023-08-08T16:00:21.357594+08:00
NOTE: group 1 PST updated.
NOTE: grp 1 disk 0 expelled from the PST.
NOTE: grp 1 disk 1 expelled from the PST.
NOTE: grp 1 disk 2 expelled from the PST.
NOTE: Erasing header on /dev/oracleasm/disks/VOL01
NOTE: Erasing header on /dev/oracleasm/disks/VOL02
NOTE: Erasing header on /dev/oracleasm/disks/VOL03
2023-08-08T16:00:21.625721+08:00
NOTE: membership refresh pending for group 1/0xd4a01dae (CRS)
NOTE: Attempting voting file refresh on diskgroup CRS
NOTE: Refresh completed on diskgroup CRS. Found 3 voting file(s).
NOTE: Voting file relocation is required in diskgroup CRS
2023-08-08T16:00:21.687195+08:00
GMON querying group 1 at 28 for pid 27, osid 6491
GMON querying group 1 at 29 for pid 27, osid 6491
2023-08-08T16:00:21.708706+08:00
NOTE: Disk CRS_0000 in mode 0x0 marked for de-assignment
NOTE: Disk CRS_0001 in mode 0x0 marked for de-assignment
NOTE: Disk CRS_0002 in mode 0x0 marked for de-assignment
SUCCESS: refreshed membership for 1/0xd4a01dae (CRS)
2023-08-08T16:00:22.580833+08:00
SUCCESS: rebalance completed for group 1/0xd4a01dae (CRS)
NOTE: Attempting voting file refresh on diskgroup CRS
NOTE: Refresh completed on diskgroup CRS. Found 3 voting file(s).
NOTE: Voting file relocation is required in diskgroup CRS
NOTE: Attempting voting file relocation on diskgroup CRS
NOTE: Successful voting file relocation on diskgroup CRS

4.2 剔除数据盘日志信息

2023-08-08T16:15:21.792257+08:00
NOTE: requesting all-instance membership refresh for group=2
2023-08-08T16:15:21.820517+08:00
NOTE: membership refresh pending for group 2/0xd4c01daf (DG)
2023-08-08T16:15:21.822852+08:00
GMON querying group 2 at 30 for pid 27, osid 6491
2023-08-08T16:15:21.844719+08:00
SUCCESS: refreshed membership for 2/0xd4c01daf (DG)
2023-08-08T16:15:21.845335+08:00
SUCCESS: alter diskgroup DG drop disk DG_0000 rebalance power 11 nowait

2023-08-08T16:15:23.305285+08:00
NOTE: Attempting voting file refresh on diskgroup DG
NOTE: Refresh completed on diskgroup DG. No voting file found.
2023-08-08T16:15:23.328193+08:00
NOTE: starting rebalance of group 2/0xd4c01daf (DG) at power 11
NOTE: starting process ARBA
Starting background process ARBA
2023-08-08T16:15:23.377695+08:00
ARBA started with pid=39, OS id=10549 
NOTE: starting process ARB0
Starting background process ARB0
2023-08-08T16:15:23.403147+08:00
ARB0 started with pid=47, OS id=10551 
NOTE: assigning ARBA to group 2/0xd4c01daf (DG) to compute estimates
NOTE: assigning ARB0 to group 2/0xd4c01daf (DG) with 11 parallel I/Os
2023-08-08T16:15:23.603553+08:00
NOTE: F1X0 on disk 0 (fmt 2) relocated at fcn 0.14556462: AU 10 -> AU 0
NOTE: header on disk 4 advanced to format #2 using fcn 0.0
NOTE: F1X0 on disk 4 (fmt 2) relocated at fcn 0.14556462: AU 0 -> AU 209615
NOTE: 08/08/23 16:15:23 DG.F1X0 copy 1 relocating from 0:10 to 4:209615 at FCN 0.14556462