ASM normal冗余USABLE_FILE_MB为负值后，失败组损坏之后是否还能mount
在之前的测试中USABLE_FILE_MB均有充足的空间，本次将会在USABLE_FILE_MB为负值的情况下，测试asm磁盘的mirror是否正常，磁盘组是否可以mount，数据有效性。
测试环境：Oracle 11.2.0.4 RAC
场景一、3个失败组 每个失败组1个磁盘，大小1g
1、创建3个失败组，每个失败组1个磁盘，大小1g
create diskgroup test normal redundancy
failgroup fg1 disk '/dev/asm_1g_1' size 1g
failgroup fg2 disk '/dev/asm_1g_2' size 1g
failgroup fg3 disk '/dev/asm_1g_3' size 1g
ATTRIBUTE 'au_size'='4M';
SQL> select name,state,total_mb/1024 total_gb,free_mb,required_mirror_free_mb,usable_file_mb from v$asm_diskgroup where name='TEST';
NAME STATE TOTAL_GB FREE_MB REQUIRED_MIRROR_FREE_MB USABLE_FILE_MB
---------- ----------- ---------- ---------- ----------------------- --------------
TEST MOUNTED 3 2856 1024 916
2、创建表空间，数据文件撑满asm磁盘组
create tablespace test datafile '+test' size 1400m;
create table test(id number) tablespace test;
insert into test values(1);
commit;
select * from test;
SQL> select name,state,total_mb/1024 total_gb,free_mb,required_mirror_free_mb,usable_file_mb from v$asm_diskgroup where name='TEST';
NAME STATE TOTAL_GB FREE_MB REQUIRED_MIRROR_FREE_MB USABLE_FILE_MB
---------- ----------- ---------- ---------- ----------------------- --------------
TEST MOUNTED 3 2856 1024 916
撑满asm磁盘组后，USABLE_FILE_MB为负值
NAME STATE TOTAL_GB FREE_MB REQUIRED_MIRROR_FREE_MB USABLE_FILE_MB
---------- ----------- ---------- ---------- ----------------------- --------------
TEST MOUNTED 3 36 1024 -494
3、第一次dd其中一个失败组磁盘
dd if=/dev/zero of=/dev/asm_1g_1 bs=1024 count=1
SQL> select GROUP_NUMBER,DISK_NUMBER,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,NAME,FAILGROUP,PATH,FAILGROUP_TYPE from v$asm_disk where name like 'TEST%' order by path;
GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU MODE_ST STATE NAME FAILG PATH FAILGRO
------------ ----------- ------- ------------ ------- -------- ---------- ----- -------------------- -------
4 0 CACHED CANDIDATE ONLINE NORMAL TEST_0000 FG1 /dev/asm_1g_1 REGULAR
4 1 CACHED MEMBER ONLINE NORMAL TEST_0001 FG2 /dev/asm_1g_2 REGULAR
4 2 CACHED MEMBER ONLINE NORMAL TEST_0002 FG3 /dev/asm_1g_3 REGULAR
4、手动卸载asm磁盘组
SQL> alter diskgroup test dismount force;
5、清除buffer
SQL> alter system flush buffer_cache;
System altered.
ERROR at line 1:
ORA-03113: end-of-file on communication channel
Process ID: 35670
Session ID: 1 Serial number: 5
数据库已宕机
6、强制mount
SQL> alter diskgroup test mount force;
Diskgroup altered.
磁盘组可以挂载
SQL> select GROUP_NUMBER,DISK_NUMBER,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,NAME,FAILGROUP,PATH,FAILGROUP_TYPE from v$asm_disk order by path;
GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU MODE_ST STATE NAME FAILG PATH FAILGRO
------------ ----------- ------- ------------ ------- -------- ---------- ----- -------------------- -------
0 8 CLOSED CANDIDATE ONLINE NORMAL /dev/asm_1g_1 REGULAR
4 1 CACHED MEMBER ONLINE NORMAL TEST_0001 FG2 /dev/asm_1g_2 REGULAR
4 2 CACHED MEMBER ONLINE NORMAL TEST_0002 FG3 /dev/asm_1g_3 REGULAR
4 0 MISSING UNKNOWN OFFLINE FORCING _DROPPED_0000_TEST FG1 REGULAR
mount之后，发现只有2个失败组fg2,fg3存活，fg1离线了
SQL> select name,state,total_mb/1024 total_gb,free_mb,required_mirror_free_mb,usable_file_mb from v$asm_diskgroup where name='TEST';
NAME STATE TOTAL_GB FREE_MB REQUIRED_MIRROR_FREE_MB USABLE_FILE_MB
---------- ----------- ---------- ---------- ----------------------- --------------
TEST MOUNTED 2 0 0 0
7、重启数据库验证数据
SQL> startup force;
SQL> select * from test;
ID
----------
1
数据验证无误
8、第二次dd第二个失败组磁盘
SQL> select GROUP_NUMBER,DISK_NUMBER,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,NAME,FAILGROUP,PATH,REPAIR_TIMER from v$asm_disk where name like 'TEST%' order by path;
GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU MODE_ST STATE NAME FAILG PATH REPAIR_TIMER
------------ ----------- ------- ------------ ------- -------- -------------------- ----- -------------------- ------------
4 1 CACHED MEMBER ONLINE NORMAL TEST_0001 FG2 /dev/asm_1g_2 0
4 2 CACHED MEMBER ONLINE NORMAL TEST_0002 FG3 /dev/asm_1g_3 0
dd if=/dev/zero of=/dev/asm_1g_2 bs=1024 count=1 fg2
SQL> select GROUP_NUMBER,DISK_NUMBER,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,NAME,FAILGROUP,PATH,FAILGROUP_TYPE from v$asm_disk where name like 'TEST%' order by path;
GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU MODE_ST STATE NAME FAILG PATH REPAIR_TIMER
------------ ----------- ------- ------------ ------- -------- -------------------- ----- -------------------- ------------
4 1 CACHED CANDIDATE ONLINE NORMAL TEST_0001 FG2 /dev/asm_1g_2 0
4 2 CACHED MEMBER ONLINE NORMAL TEST_0002 FG3 /dev/asm_1g_3 0
4、手动卸载asm磁盘组
SQL> alter diskgroup test dismount force;
5、清除buffer
SQL> alter system flush buffer_cache;
数据库已宕机
6、强制mount
SQL> alter diskgroup test mount force;
ERROR at line 1:
ORA-03113: end-of-file on communication channel
Process ID: 33409
Session ID: 36 Serial number: 107
强制mount的时候，asm实例异常
ASM告警日志：
SQL> ALTER DISKGROUP TEST MOUNT /* asm agent *//* {0:1:64} */
NOTE: Disk in mode 0x1 marked for de-assignment
NOTE: Disk in mode 0x7f marked for de-assignment
NOTE: Disk TEST_0002 in mode 0x7f marked for de-assignment
ERROR: diskgroup TEST was not mounted
ORA-15032: not all alterations performed
ORA-15040: diskgroup is incomplete
ORA-15042: ASM disk "1" is missing from group number "4"
ORA-15042: ASM disk "0" is missing from group number "4"
ERROR: ALTER DISKGROUP TEST MOUNT /* asm agent *//* {0:1:64} */
ASM Health Checker found 1 new failures
查看asm告警日志，test磁盘组已无法强制mount
场景二、4个失败组 每个失败组1个磁盘，大小1g
1、创建4个失败组，每个失败组1个磁盘，大小1g
create diskgroup test normal redundancy
failgroup fg1 disk '/dev/asm_1g_1' size 1g
failgroup fg2 disk '/dev/asm_1g_2' size 1g
failgroup fg3 disk '/dev/asm_1g_3' size 1g
failgroup fg4 disk '/dev/asm_1g_4' size 1g
ATTRIBUTE 'au_size'='4M';
SQL> select name,state,total_mb/1024 total_gb,free_mb,required_mirror_free_mb,usable_file_mb from v$asm_diskgroup where name='TEST';
NAME STATE TOTAL_GB FREE_MB REQUIRED_MIRROR_FREE_MB USABLE_FILE_MB
---------- ----------- ---------- ---------- ----------------------- --------------
TEST MOUNTED 4 3872 1024 1424
2、创建表空间，数据文件撑满asm磁盘组
create tablespace test datafile '+test' size 1800m;
create table test(id number) tablespace test;
insert into test values(1);
commit;
select * from test;
撑满asm磁盘组后，USABLE_FILE_MB为负值
NAME STATE TOTAL_GB FREE_MB REQUIRED_MIRROR_FREE_MB USABLE_FILE_MB
---------- ----------- ---------- ---------- ----------------------- --------------
TEST MOUNTED 4 252 1024 -386
3、第一次dd其中一个失败组磁盘
dd if=/dev/zero of=/dev/asm_1g_1 bs=1024 count=1
4、手动卸载asm磁盘组
SQL> alter diskgroup test dismount force;
5、清除buffer
SQL> alter system flush buffer_cache;
System altered.
ERROR at line 1:
ORA-03113: end-of-file on communication channel
Process ID: 35670
Session ID: 1 Serial number: 5
数据库已宕机
6、第一次强制mount
SQL> alter diskgroup test mount force;
Diskgroup altered.
磁盘组可以挂载
SQL> select GROUP_NUMBER,DISK_NUMBER,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,NAME,FAILGROUP,PATH,FAILGROUP_TYPE from v$asm_disk order by path;
GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU MODE_ST STATE NAME FAILGROUP PATH REPAIR_TIMER
------------ ----------- ------- ------------ ------- -------- ---------- ---------- -------------------- ------------
4 1 CACHED MEMBER ONLINE NORMAL TEST_0001 FG2 /dev/asm_1g_2 0
4 2 CACHED MEMBER ONLINE NORMAL TEST_0002 FG3 /dev/asm_1g_3 0
4 3 CACHED MEMBER ONLINE NORMAL TEST_0003 FG4 /dev/asm_1g_4 0
SQL> select name,state,total_mb/1024 total_gb,free_mb,required_mirror_free_mb,usable_file_mb from v$asm_diskgroup where name='TEST';
NAME STATE TOTAL_GB FREE_MB REQUIRED_MIRROR_FREE_MB USABLE_FILE_MB
---------- ----------- ---------- ---------- ----------------------- --------------
TEST MOUNTED 3 0 1024 -512
7、重启数据库验证数据
SQL> startup force;
SQL> select * from test;
ID
----------
1
第一次强制mount后，数据验证无误
8、第二次dd第二个失败组磁盘
SQL> select GROUP_NUMBER,DISK_NUMBER,MOUNT_STATUS,HEADER_STATUS,MODE_STATUS,STATE,NAME,FAILGROUP,PATH,REPAIR_TIMER from v$asm_disk where name like 'TEST%' order by path;
GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU MODE_ST STATE NAME FAILG PATH REPAIR_TIMER
------------ ----------- ------- ------------ ------- -------- -------------------- ----- -------------------- ------------
4 1 CACHED MEMBER ONLINE NORMAL TEST_0001 FG2 /dev/asm_1g_2 0
4 2 CACHED MEMBER ONLINE NORMAL TEST_0002 FG3 /dev/asm_1g_3 0
4 3 CACHED MEMBER ONLINE NORMAL TEST_0003 FG4 /dev/asm_1g_4 0
目前存活失败组fg2，fg3，fg4
开始破坏fg2失败组
dd if=/dev/zero of=/dev/asm_1g_2 bs=1024 count=1 fg2
4、手动卸载asm磁盘组
SQL> alter diskgroup test dismount force;
5、清除buffer
SQL> alter system flush buffer_cache;
数据库已宕机
6、第二次强制mount
SQL> alter diskgroup test mount force;
ERROR at line 1:
ORA-03113: end-of-file on communication channel
Process ID: 33409
Session ID: 36 Serial number: 107
强制mount的时候，asm实例异常
ASM告警日志：
ORA-00600: internal error code, arguments: [kfdp_isLivePath:03], [0], [2], [], [], [], [], [], [], [], [], []
GMON (ospid: 48820): terminating the instance due to error 493
System state dump requested by (instance=1, osid=48820 (GMON)), summary=[abnormal instance termination].
System State dumped to trace file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_diag_48792_20190628142108.trc
Dumping diagnostic data in directory=[cdmp_20190628142108], requested by (instance=1, osid=48820 (GMON)), summary=[abnormal instance termination].
Instance terminated by GMON, pid = 48820
NOTE: cache registered group TEST number=4 incarn=0x36254d44
NOTE: cache began mount (first) of group TEST number=4 incarn=0x36254d44
ASM Health Checker found 1 new failures
NOTE: Assigning number (4,2) to disk (/dev/asm_1g_3)
NOTE: Assigning number (4,3) to disk (/dev/asm_1g_4)
2019-06-28 14:21:41.751000 +08:00
NOTE: GMON heartbeating for grp 4
GMON querying group 4 at 22 for pid 28, osid 49516
NOTE: Assigning number (4,0) to disk ()
NOTE: Assigning number (4,1) to disk ()
GMON querying group 4 at 23 for pid 28, osid 49516
NOTE: cache closing disk 0 of grp 4: (not open)
ERROR: diskgroup TEST was not mounted
ORA-15032: not all alterations performed
ORA-15040: diskgroup is incomplete
ORA-15042: ASM disk "1" is missing from group number "4"
ORA-15042: ASM disk "0" is missing from group number "4"
ERROR: ALTER DISKGROUP TEST MOUNT /* asm agent *//* {0:1:6} */
ASM Health Checker found 1 new failures
SQL> ALTER DISKGROUP TEST MOUNT /* asm agent *//* {0:1:6} */
NOTE: cache registered group TEST number=4 incarn=0x5db54d4b
NOTE: cache began mount (first) of group TEST number=4 incarn=0x5db54d4b
ASM Health Checker found 1 new failures
NOTE: Assigning number (4,2) to disk (/dev/asm_1g_3)
NOTE: Assigning number (4,3) to disk (/dev/asm_1g_4)
查看asm告警日志，test磁盘组,总共4个asm成员，只找到了/dev/asm_1g_3，/dev/asm_1g_4，磁盘1，磁盘2找不到了，test磁盘组已无法强制mount
结论：在USABLE_FILE_MB为负值，asm磁盘组只能容忍第一次失败组磁盘失效，随后的asm磁盘组只要有其中一个失败组磁盘损坏，整个asm磁盘组就无法重新mount(即使mount force）。
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
