关于ORACLE10gR2 ASM（raw实现和asmlib实现）使用GX2000透明代理测试报告

测试环境：

存储：AS1600（IP 192.168.21.241)

GX2000：(IP 192.168.21.119)

主机： Red Hat Enterprise Linux Server release 5.9  64bit

rac1： (IP 192.168.21.111)

rac2： (IP 192.168.21.112)

/etc/hosts

127.0.0.1 localhost.localdomain localhost 

192.168.21.111 rac1.mycorpdomain.com rac1 
192.168.21.113 rac1-vip.mycorpdomain.com rac1-vip 
10.10.10.111 rac1-priv.mycorpdomain.com rac1-priv 
192.168.21.112 rac2.mycorpdomain.com rac2 
192.168.21.114 rac2-vip.mycorpdomain.com rac2-vip 
10.10.10.112 rac2-priv.mycorpdomain.com rac2-priv

数据库：ORACLE   10.2.0.1.0 64bit

数据库名1：orcl（使用raw实现的ASM)

实例名：orcl1 orcl2

数据库名2：erp (使用asmlib实现的ASM)

实例名：erp1 erp2

OCR:

[oracle@rac1 ~]$ ocrcheck 
Status of Oracle Cluster Registry is as follows : 
Version : 2 
Total space (kbytes) : 10485672 
Used space (kbytes) : 4608 
Available space (kbytes) : 10481064 
ID : 136126823 
Device/File Name : /dev/raw/raw1 
Device/File integrity check succeeded 
Device/File Name : /dev/raw/raw2 
Device/File integrity check succeeded 
Cluster registry integrity check succeeded

VOTEDISK：

[oracle@rac1 ~]$ crsctl query css votedisk 
0. 0 /dev/raw/raw4 
located 1 votedisk(s).

一.使用raw实现ASM

/dev/raw/raw6 

/dev/raw/raw5

/dev/raw/raw4 
/dev/raw/raw3

/dev/raw/raw2 

/dev/raw/raw1

由于该方法实现的ASM，对于ASM来讲只识别到/dev/raw/raw*,

SQL> select state,path from v$asm_disk; 

STATE PATH 
-------- ---------------------------------------- 
NORMAL /dev/oracleasm/disks/DISK1 
NORMAL /dev/raw/raw4 
NORMAL /dev/raw/raw3 
NORMAL /dev/raw/raw1 
NORMAL /dev/raw/raw2 
NORMAL /dev/oracleasm/disks/DISK2 
NORMAL /dev/raw/raw6 
NORMAL /dev/raw/raw5

裸设备通过/etc/sysconfig/ rawdevices进行绑定

/dev/raw/raw1 /dev/mapper/ocr1 
/dev/raw/raw2 /dev/mapper/ocr2 
/dev/raw/raw3 /dev/mapper/vote1 
/dev/raw/raw4 /dev/mapper/vote2 
/dev/raw/raw5 /dev/mapper/data1 
/dev/raw/raw6 /dev/mapper/data2

所以使用GX2000透明代理后，只需要修改/etc/multipath.conf

multipaths { 
multipath { 
wwid 3600d0231000ea4bb646f09714bb0284c 
# wwid 360030d9070f79f021451a5a4e038f958 
alias ocr1 
} 
multipath { 
wwid 3600d0231000ea4bb170a69512a3c5052 
# wwid 360030d909d1ca102e3c2f7d14c5989bd 
alias ocr2 
} 

multipath { 
wwid 3600d0231000ea4bb6c9fecd7501e4cca 
# wwid 360030d90eef7a0021c0044e17f05c652 
alias vote1 
} 
multipath { 
wwid 3600d0231000ea4bb3b54cba111ba2274 
# wwid 360030d90a830a0024957311492e73cfe 
alias vote2 
} 
multipath { 
wwid 3600d0231000ea4bb3d9c1ecb1463c7c6 
# wwid 360030d902d41a102103b5eb2839dc5bc 
alias data1 
} 
multipath { 
wwid 3600d0231000ea4bb1dc053411e51debb 
# wwid 360030d903a61a1028a0a8d6fe8e62fd8 
alias data2 
} 
multipath { 
wwid 3600d0231000ea4bb10787116127e8722 
alias mpath8 
} 
multipath{ 
wwid 3600d0231000ea4bb06e3fcdb1bf2a783 
# wwid 360030d907ef32503e170127d6cb87908 
alias mpath7 
}

注释之前的wwid，使用GX2000透明代理过来的wwid进行绑定即可。

注意：过程中一定要注意裸设备的权限

[oracle@rac1 ~]$ ls -al /dev/raw/ 
total 0 
drwxr-xr-x 2 root root 160 Feb 13 16:56 . 
drwxr-xr-x 14 root root 4900 Feb 13 16:56 .. 
crw-rw---- 1 oracle oinstall 162, 1 Feb 13 18:56 raw1 
crw-rw---- 1 oracle oinstall 162, 2 Feb 13 18:56 raw2 
crw-rw---- 1 oracle oinstall 162, 3 Feb 13 16:56 raw3 
crw-rw---- 1 oracle oinstall 162, 4 Feb 13 18:56 raw4 
crw-rw---- 1 oracle oinstall 162, 5 Feb 13 18:56 raw5 
crw-rw---- 1 oracle oinstall 162, 6 Feb 13 18:56 raw6 
[oracle@rac1 ~]$ 

一般都是通过udev进行权限绑定的

[oracle@rac1 ~]$ cat /etc/udev/rules.d/99-raw.rules 
KERNEL=="raw[1-6]*", OWNER="oracle", GROUP="oinstall", MODE="660"

二.使用ASMlib实现的ASM

[root@rac1 ~]# oracleasm listdisks 
DISK1 
DISK2

查看DISK*对应的LUN

[root@rac1 ~]# oracleasm querydisk -d DISK1 

Disk "DISK1" is a valid ASM disk on device /dev/sdh[8,112] 
[root@rac1 ~]# oracleasm querydisk -d DISK2 
Disk "DISK2" is a valid ASM disk on device /dev/sdi[8,128]

通过/dev/sdh和/dev/sdi确认multipath

[root@rac1 mpath]# multipath -v3|grep sdi

...

sdi: ownership set to mpath8

...

由于该方法实现的ASM,在实现ASM的时候

oracleasm   createdisk  DISK1 /dev/mapper/mpath7

oracleasm craetedisk DISK2 /dev/mapper/mpath8

所以最终一定要确认 /dev/mapper/mpath7和 /dev/mapper/mpath8是对应的

multipaths { 
multipath { 
wwid 3600d0231000ea4bb646f09714bb0284c 
# wwid 360030d9070f79f021451a5a4e038f958 
alias ocr1 
} 
multipath { 
wwid 3600d0231000ea4bb170a69512a3c5052 
# wwid 360030d909d1ca102e3c2f7d14c5989bd 
alias ocr2 
} 

multipath { 
wwid 3600d0231000ea4bb6c9fecd7501e4cca 
# wwid 360030d90eef7a0021c0044e17f05c652 
alias vote1 
} 
multipath { 
wwid 3600d0231000ea4bb3b54cba111ba2274 
# wwid 360030d90a830a0024957311492e73cfe 
alias vote2 
} 
multipath { 
wwid 3600d0231000ea4bb3d9c1ecb1463c7c6 
# wwid 360030d902d41a102103b5eb2839dc5bc 
alias data1 
} 
multipath { 
wwid 3600d0231000ea4bb1dc053411e51debb 
# wwid 360030d903a61a1028a0a8d6fe8e62fd8 
alias data2 
} 
multipath { 
wwid 3600d0231000ea4bb10787116127e8722 
alias mpath8 
} 
multipath{ 
wwid 3600d0231000ea4bb06e3fcdb1bf2a783 
# wwid 360030d907ef32503e170127d6cb87908 
alias mpath7 
}

由于oracleasm在创建过程中已经绑定了具体的权限，后期使用中无需担心权限问题。