ASM丢失disk header导致ORA-15032、ORA-15040、ORA-15042 Diskgroup无法mount

SQL> select * from v$version;
BANNER
——————————————————————————–
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 – 64bit Production
PL/SQL Release 11.2.0.3.0 – Production
CORE 11.2.0.3.0 Production
TNS for Linux: Version 11.2.0.3.0 – Production
NLSRTL Version 11.2.0.3.0 – Production
SQL> alter diskgroup datadg mount;
alter diskgroup datadg mount
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15040: diskgroup is incomplete
ORA-15042: ASM disk “5” is missing from group number “1”
ERROR: alter diskgroup datadg mount
Wed Mar 13 07:42:03 2013
SQL> alter diskgroup datadg mount 
NOTE: cache registered group DATADG number=1 incarn=0xccb845cd
NOTE: cache began mount (first) of group DATADG number=1 incarn=0xccb845cd
NOTE: Assigning number (1,2) to disk (/dev/asm-diskg)
NOTE: Assigning number (1,1) to disk (/dev/asm-diskf)
NOTE: Assigning number (1,0) to disk (/dev/asm-diske)
Wed Mar 13 16:42:09 2013
NOTE: GMON heartbeating for grp 1
GMON querying group 1 at 20 for pid 27, osid 5439
NOTE: Assigning number (1,5) to disk ()
GMON querying group 1 at 21 for pid 27, osid 5439
NOTE: cache dismounting (clean) group 1/0xCCB845CD (DATADG) 
NOTE: messaging CKPT to quiesce pins Unix process pid: 5439, image: oracle@vmac1 (TNS V1-V3)
NOTE: dbwr not being msg’d to dismount
NOTE: lgwr not being msg’d to dismount
NOTE: cache dismounted group 1/0xCCB845CD (DATADG) 
NOTE: cache ending mount (fail) of group DATADG number=1 incarn=0xccb845cd
NOTE: cache deleting context for group DATADG 1/0xccb845cd
GMON dismounting group 1 at 22 for pid 27, osid 5439
NOTE: Disk in mode 0x8 marked for de-assignment
NOTE: Disk in mode 0x8 marked for de-assignment
NOTE: Disk in mode 0x8 marked for de-assignment
NOTE: Disk in mode 0x8 marked for de-assignment
ERROR: diskgroup DATADG was not mounted
ORA-15032: not all alterations performed
ORA-15040: diskgroup is incomplete
ORA-15042: ASM disk “5” is missing from group number “1” 
ERROR: alter diskgroup datadg mount
Wed Mar 13 16:42:10 2013
ASM Health Checker found 1 new failures
 
[grid@vmac1 ~]$ kfed read /dev/asm-diskh
kfbh.endian: 0 ; 0x000: 0x00
kfbh.hard: 0 ; 0x001: 0x00
kfbh.type: 0 ; 0x002: KFBTYP_INVALID
kfbh.datfmt: 0 ; 0x003: 0x00
kfbh.block.blk: 0 ; 0x004: blk=0
kfbh.block.obj: 0 ; 0x008: file=0
kfbh.check: 0 ; 0x00c: 0x00000000
kfbh.fcn.base: 0 ; 0x010: 0x00000000
kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
kfbh.spare1: 0 ; 0x018: 0x00000000
kfbh.spare2: 0 ; 0x01c: 0x00000000
7FA1DA233400 00000000 00000000 00000000 00000000 [................]
Repeat 255 times
KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]
col path for a20
set linesize 200 pagesize 1400
select path,header_status,state from v$asm_disk;
PATH HEADER_STATUS STATE
——————– ———————————— ————————
/dev/asm-diskh CANDIDATE NORMAL
/dev/asm-diskg MEMBER NORMAL
/dev/asm-diskf MEMBER NORMAL
/dev/asm-diske MEMBER NORMAL
/dev/asm-diskc MEMBER NORMAL
/dev/asm-diskd MEMBER NORMAL
/dev/asm-diskb MEMBER NORMAL
7 rows selected.
[grid@vmac1 ~]$ kfed repair /dev/asm-diskh
KFED-00320: Invalid block num1 = [0], num2 = [1], error = [endian_kfbh]
 
[grid@vmac1 ~]$ kfed repair /dev/asm-diskh ausz=1048576
KFED-00320: Invalid block num1 = [0], num2 = [1], error = [endian_kfbh]
 
关闭ASM实例
 
 
优先备份 问题ASM的header
dd if=<bad disk> of=<file> bs=4096 count=1
 
 
3. 检查现有磁盘找出拥有file 1 block 1的
[grid@vmac1 ~]$ kfed read /dev/asm-diske |grep f1b1
kfdhdb.f1b1locn: 2 ; 0x0d4: 0x00000002
[grid@vmac1 ~]$ 
[grid@vmac1 ~]$ kfed read /dev/asm-diskf |grep f1b1 
kfdhdb.f1b1locn: 0 ; 0x0d4: 0x00000000
[grid@vmac1 ~]$ kfed read /dev/asm-diskg |grep f1b1 
kfdhdb.f1b1locn: 0 ; 0x0d4: 0x00000000
 
这里asm-diske上出现了f1b1非零值，则其拥有file 1 block 1，可以通过检查第二个au的类型是否是KFBTYP_LISTHEAD来确认
[grid@vmac1 ~]$ kfed read /dev/asm-diske aun=2|grep kfbh.type
kfbh.type: 5 ; 0x002: KFBTYP_LISTHEAD
 
 
若丢失的磁盘包含了”file 1 block 1 F1B1″则扫描 该磁盘上所有的 AU直到找到KFBTYP_LISTHEAD ， 如果找不到LISTHEAD 那么别无选择只能重建diskgroup。
 
从版本11.1.0.7开始(10g是从10.2.0.5开始，所以尽量别用10.2.0.5之前的版本上的ASM)，当每一个I/O写提交向ASM disk header(AU 0 blocknum 0)，都会复制到 AU 1中，最后第二个块。基于不同的AU size，该块的位置不同
Allocation Unit Size Block Number on AU 1
1048576 254
4194304 1022
8388608 2046
16777216 4094
 
首先使用kfed 验证该位置是否有正确的disk header，否则手动找到合适的header。 表1如参考：
[grid@vmac1 ~]$ kfed read /dev/asm-diske ausz=1048576 aun=1 blkn=254|less 
kfbh.endian: 1 ; 0x000: 0x01
kfbh.hard: 130 ; 0x001: 0x82
kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD
kfbh.datfmt: 1 ; 0x003: 0x01
kfbh.block.blk: 254 ; 0x004: blk=254
kfbh.block.obj: 2147483648 ; 0x008: disk=0
kfbh.check: 2086475720 ; 0x00c: 0x7c5d17c8
kfbh.fcn.base: 31322 ; 0x010: 0x00007a5a
kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
kfbh.spare1: 0 ; 0x018: