KFED.PL for diagnosing - ORA-15063 ORA-15042 ORA-15020 ORA-15033 (Doc ID 1346190.1)

In this Document

Purpose

Requirements

Configuring

Instructions

Script

Sample Output

References

---

APPLIES TO:

Oracle Database - Enterprise Edition - Version 10.2.0.1 to 11.2.0.3 [Release 10.2 to 11.2]
Information in this document applies to any platform.

PURPOSE

kfed is a useful tool which allows to analyze ASM disk header information when ASM diskgroups can not be mounted. As the number of ASM devices for a diskgroup increase, sometimes it's very hard to anlayze all devices without eye-balling each kfed output for all ASM devices, which is time consuming and error prone. kfed.pl is a simple perl script which displays gist information such digkgroup, diskname etc in one line for each ASM device from the kfed output.


kfed.pl is particulary useful for diskgroup mount issue due to the following errors :

- ORA-15063 : ASM discovered an insufficient number of disks for diskgroup \"%s\"
- ORA-15042 : ASM disk \"%s\" is missing from group number \"%s\" 
- ORA-15020 : discovered duplicate ASM disk \"%s\"
- ORA-15033 : disk '%s' belongs to diskgroup \"%s\""
- v$asm_disk shows disks with "MISSING", "IGNORED".
- v$asm_disk shows disks with "PROVISIONED status while the diskgrop is mounted.

REQUIREMENTS

kfed output for all ASM devices

1-1. Build "kfed" executable if it does not exist.  kfed executable comes with installation from 11.1 onwards. 
$cd $ORACLE_HOME/rdbms/lib
$make -f ins_rdbms.mk ikfed

1-2. As root, 
- cd <asm home>/bin and create a shell file called kfed.sh using the following script 
- And change the permission as executable - ex) chmod 755 kfed.sh
- /dev/mapper/* on 3rd line should be replaced with ASM devices path on the server before running kfed.sh 

kfed.sh

#! /bin/sh
rm /tmp/kfed_DH.out /tmp/kfed_FS.out /tmp/kfed_BK.out /tmp/kfed_FD.out /tmp/kfed_DD.out
for i in `ls /dev/mapper/*`
do
echo $i >> /tmp/kfed_DH.out
./kfed read $i >> /tmp/kfed_DH.out
echo $i >> /tmp/kfed_FS.out
./kfed read $i blkn=1 >> /tmp/kfed_FS.out
echo $i >> /tmp/kfed_BK.out
./kfed read $i aun=1 blkn=254 >> /tmp/kfed_BK.out
echo $i >> /tmp/kfed_FD.out
./kfed read $i aun=2 blkn=1 >> /tmp/kfed_FD.out
echo $i >> /tmp/kfed_DD.out
./kfed read $i aun=2 blkn=2 >> /tmp/kfed_DD.out
done

- Run kfed.sh in <asm home>/bin

# chmod u+x kfed.sh
# kfed.sh

- This will generate kfed output files in /tmp. 
. /tmp/kfed_DH.out, /tmp/kfed_FS.out, /tmp/kfed_BK.out, /tmp/kfed_FD.out and /tmp/kfed_DD.out

CONFIGURING

Perl interpreter should be installed on the server - perl executable comes with Unix or Linux as default.

$ which perl
/usr/bin/perl

If perl executable is not found /usr/bin directory, the one in $ORACLE_HOME/perl/bin can be used.
$ export PATH=$ORACLE_HOME/perl/bin:$PATH

INSTRUCTIONS

Save the perl script at end of this note as "kfed.pl" and change the permission for execution.

# chmod u+x kfed.pl
# kfed.pl kfed_DH.out | sort -t ' ' -k 2.3 > kfed.out

CAUTION

This sample code is provided for educational purposes only, and is not supported by Oracle Support. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.

SCRIPT

 

#!/usr/bin/perl

# kfed.pl
# Input kfed_DH.out or kfed_BK.out
# Usage : kfed.pl kfed_DH.out | sort -t ' ' -k 2.3

# 2012/05/07 kyle Heo Added Pattern matching for all components
# 2012/05/06 kyle Heo Added Pattern matching for crestmphi/crestmplo & apply pattern capturing for asmlabel
# 2011/10/11 kyle Heo Added Voting Files information
# 2011/08/07 kyle Heo Added "N/A" for no ASMLabel
# 2011/07/24 kyle Heo Added Disk Size & Creation time
# 2011/06/03 kyle Heo Added Add ASMlabel
# 2011/03/26 kyle Heo dos2unix filename - To fix \r\n issue

use warnings;
use strict;

#Device Name String
#my $deviceM = '^\.';
my $deviceM = '^/dev/';
#my $deviceM = '^/u02/';
#my $deviceM = '^ora_';
#my $deviceM = "^asm";
#my $deviceM = "^\/fra";

my $device ="NOASMDEVICE" ;
my $dskname ;
my $dsknum ;
my $grpname ;
my $fgname ;
my $hdrsts;
my $grptyp;
my $f1b1locn;
my $asmlabel;
my $dsksize;
my $vf=0;
my ($crestmphi, $crestmplo, $c_secs, $c_mins, $c_hour, $c_days, $c_mnth, $c_year ) ;
my ($secs, $mins, $hour, $days, $mnth, $year );

my %tmp;
my @fields ;
my $num_args;

$num_args = $#ARGV + 1;
die "Usage: kfed.pl kfed_DH.out | sort -t ' ' -k 2.3 | more " if ($num_args != 1);

#Print Header
printf "%-30s %-15s %-8s %-15s %-15s %-15s %-10s %-8s %-2s %-14s\n" ,
"1.Device" ,"2.GroupName", "3.DN#(Size)", "4.DiskName", "5.ASMLabel", "6.FailGroup" ,
"7.H_Staus" , "8.Rdancy", "F1(V)" , "Creation Time" ;

while (<>) {

# To remove \r\n from dos format
s/^\s+//; s/\s+$// ;

if ( $_ =~ /$deviceM/ ) {
if ( $device =~ /NOASMDEVICE/ ) {
$device = $_ ;
next;
}
elsif ( $device && $grpname )
{
printf "%-30s %-15s %04d(%s) %-15s %-15s %-15s %-10s %-8s %-2s(%3s) %s%s \n" ,
$device, $grpname, $dsknum, $dsksize, $dskname, $asmlabel, $fgname ,
$hdrsts , $grptyp, $f1b1locn,$vf, $crestmphi, $crestmplo ;

$device = $_ ;
$grpname="";
next ;
} else {
printf "%-30s %-15s \n" , $device , "ZZZ - NO ASM DEVICE" ;

$device = $_ ;
next;
}

}

if ( $_ =~ /dskname:\s*(\w+)/ ) { $dskname = $1; }
elsif ( $_ =~ /dsknum:\s*(\w+)/ ) { $dsknum = $1; }
elsif ( $_ =~ /grpname:\s*(\w+)/ ) { $grpname = $1; }
elsif ( $_ =~ /fgname:\s*(\w+)/ ) { $fgname = $1; }
elsif ( $_ =~ /f1b1locn:\s*(\w+)/) { $f1b1locn = $1; }
elsif ( $_ =~ /dsksize:\s*(\w+)/ ) { $dsksize = $1; }
elsif ( $_ =~ /vfstart:\s*(\w+)/ ) { $vf = $1; }
elsif ( $_ =~ /hdrsts:(.*KFDHDR_)(\w+$)/ ) { $hdrsts = $2; }
elsif ( $_ =~ /grptyp:(.*KFDGTP_)(\w+$)/ ) { $grptyp = $2; }
elsif ( $_ =~ /provstr:\s*(\w+)/ ) {
$asmlabel = $1;
if ( $asmlabel eq "ORCLDISK" ) { $asmlabel = "N/A" }
else { $asmlabel =~ s/ORCLDISK//; };

}
elsif ( $_ =~ /crestmp\.hi/ ) {
@fields = split /:/, $_ ;
%tmp = $fields[2] =~ /(\w+)=(\w+)/g;

$c_year = sprintf( "%d", hex( $tmp{YEAR} ) );
$c_mnth = sprintf( "%02d", hex( $tmp{MNTH} ) );
$c_days = sprintf( "%02d", hex( $tmp{DAYS} ) );
$c_hour = sprintf( "%02d", hex( $tmp{HOUR} ) );

$crestmphi = sprintf ( "%s/%s/%s %s", $c_year, $c_mnth, $c_days, $c_hour );
}

elsif ( $_ =~ /crestmp\.lo/ ) {
@fields = split /:/, $_ ;
%tmp = $fields[2] =~ /(\w+)=(\w+)/g;

$c_mins = sprintf( "%02d", hex($tmp{MINS}) );
$c_secs = sprintf( "%02d", hex($tmp{SECS}) );

$crestmplo = sprintf ( ":%s:%s", $c_mins, $c_secs);
}
}

if ( $device && $grpname )
{
printf "%-30s %-15s %04d(%s) %-15s %-15s %-15s %-10s %-8s %-2s(%3s) %s%s \n" ,
$device, $grpname, $dsknum, $dsksize, $dskname, $asmlabel, $fgname ,
$hdrsts , $grptyp, $f1b1locn,$vf, $crestmphi, $crestmplo ;
}

 

SAMPLE OUTPUT

$ kfed.pl /tmp/kfed_DH.out | sort - t ' ' -k 2.3  > /tmp/kfed.out

- Check kfed.out if there is any missing disks from the diskgroup in question and work with System administrator team to find them.
- Check kfed.out if there is any duplicated disks in terms of disk number from the diskgroup in question, see note - 1299866.1.
- Or upload /tmp/kfed.out for Oracle support's review.

1.Device 2.GroupName 3.DN#(Size) 4.DiskName 5.ASMLabel 6.FailGroup 7.H_Staus 8.Rdancy F1(V) Creation Time
/u02/oradata/asm_disk10 DATA 0000(647) DATA_0000 N/A F2 FORMER NORMAL 0 ( 0) 2012/03/01 15:21:18
/u02/oradata/asm_disk11 DATA 0001(2000) DATA_0001 N/A F1 MEMBER NORMAL 731( 0) 2011/06/14 20:48:55
/u02/oradata/asm_disk12 DATA 0002(2000) DATA_0002 N/A F1 MEMBER NORMAL 0 ( 0) 2011/06/14 20:48:55
/u02/oradata/asm_disk13 DATA 0003(2000) DATA_0003 N/A F1 MEMBER NORMAL 0 ( 0) 2011/06/14 20:48:55
/u02/oradata/asm_disk14 DATA 0004(2000) DATA_0004 N/A F1 MEMBER NORMAL 0 ( 0) 2011/06/14 20:48:55
/u02/oradata/asm_disk18 DATA 0005(718) DATA_0005 N/A F2 FORMER NORMAL 0 ( 0) 2012/03/01 15:21:18
/u02/oradata/asm_disk35 DATA 0006(2000) DATA_0006 N/A F2 MEMBER NORMAL 0 ( 0) 2012/03/01 15:36:36
/u02/oradata/asm_disk36 DATA 0007(2000) DATA_0007 N/A F2 MEMBER NORMAL 0 ( 0) 2012/03/01 15:36:36
/u02/oradata/asm_disk29 DATA 0010(2000) DATA_0010 N/A F1 MEMBER NORMAL 0 ( 0) 2012/02/08 21:30:20
/u02/oradata/createAsmDisk1.sh ZZZ - NO ASM DEVICE
/u02/oradata/asm_test ZZZ - NO ASM DEVICE
/u02/oradata/afiedt.buf ZZZ - NO ASM DEVICE
/u02/oradata/tmp-OCR_NEW.ocr ZZZ - NO ASM DEVICE
/u02/oradata/createAsmDisk.sh ZZZ - NO ASM DEVICE

REFERENCES

NOTE:1299866.1  - V$ASM_DISK view shows some disk header status as IGNORED and group number as "0".
NOTE:946213.1  - How To Add Back An ASM Disk or Failgroup (Normal or High Redundancy) After A Transient Failure Occurred Or When The DISK_REPAIR_TIME Attribute Expired (10.1 to 12.1)?