oracle udev 多路径,Suse 11下多路径及udev配置

最近给客户基于SuSe 11 SP3下多路径部署Oracle 10g RAC。SuSe 11下用10g，也算一朵奇葩，连篇文档都比较难找，谁叫Oracle太贵呢。下面主要是描述了在该环境下如何去配置多路径。由于10g下的ocr与votingdisk不能直接存放到asm磁盘，所以依旧要使用raw设备方式来保存。下文供大家参考。

一、查看当前scsi设备及获取设备wwid#使用 cat /proc/partitions或者/sbin/sfdisk命令查看当前系统中的scsi设备suse11a:~ # more /proc/partitions

major minor  #blocks  name

8        0  877264896 sda

8        1     200812 sda1

8        2  104856255 sda2

8        3  104856255 sda3

8        4  667340100 sda4

8       16 2097152000 sdb

8       17 2097150976 sdb1

.....

8      240    1048576 sdp

65        0    1048576 sdq

.....

65       48 2097152000 sdt

65       49 2097150976 sdt1

253        0 2097152000 dm-0

253        1 2097152000 dm-1

253        2 1751121920 dm-2

253        3    1048576 dm-3

253        4    1048576 dm-4

253        5 1751120896 dm-5

253        6 2097150976 dm-6

253        7 2097150976 dm-7

#获取多路径设备的wwid，使用下面的命令来获取当前系统中多路径设备的wwidfor i in `cat /proc/partitions | awk {'print $4'} |grep sd`; do echo "### $i: `/lib/udev/scsi_id --whitelist  --replace-whitespace /dev/$i`"; done

### sda: 3600605b007030d301a192f2d13c017a0

### sda1: 3600605b007030d301a192f2d13c017a0

.....................

### sdt: 360022a11000cb6b10a902db200000000

### sdt1: 360022a11000cb6b10a902db200000000

#获取后，过滤本地磁盘及重复的wwid

二、配置mulitpatch.conf文件#配置mulitpatch.conf文件的主要目的是用于使用有意义，用户自定义的设备名称，持久化多路径设备

#这个步骤是与配置单路径设备最大的区别

#下面是已经配置过后的multipath.conf文件suse11a:~ # cp /etc/multipath.conf /etc/multipath.conf.bak

suse11a:~ # vi /etc/multipath.conf

suse11a:~ # grep -v ^# /etc/multipath.conf

defaults {

user_friendly_names yes

}

blacklist {

devnode ^hd[a-z]

}

multipaths {

multipath {

wwid                    36200bc710099b9190ba6fc350000000a

alias                   mpatha

}

multipath {

wwid                    36200bc710099b9190ba7109b0000000b

alias                   mpathb

}

multipath {

wwid                    36200bc710099b9190ab1155300000009

alias                   mpathc

}

multipath {

wwid                    360022a11000cb6b10a902db200000000

alias                   mpathd

}

multipath {

wwid                    360022a11000cb6b10a9116d000000002

alias                   mpathe

}

}

三、校验多路径设备#使用下面的命令用于多路径配置生效suse11a:~ # service multipathd stop

suse11a:~ # service multipathd start

#下面校验多路径设备suse11a:~ # multipath -ll | grep mpath

mpathe (360022a11000cb6b10a9116d000000002) dm-1 HUAWEI  ,S5500T

mpathd (360022a11000cb6b10a902db200000000) dm-0 HUAWEI  ,S5500T

mpathc (36200bc710099b9190ab1155300000009) dm-2 HUAWEI  ,S2600T

mpathb (36200bc710099b9190ba7109b0000000b) dm-4 HUAWEI  ,S2600T

mpatha (36200bc710099b9190ba6fc350000000a) dm-3 HUAWEI  ,S2600T

#下面是设备的详细信息suse11a:~ # multipath -ll

mpathe (360022a11000cb6b10a9116d000000002) dm-1 HUAWEI  ,S5500T

size=2.0T features='0' hwhandler='0' wp=rw

|-+- policy='service-time 0' prio=1 status=active

| `- 5:0:0:1 sdc 8:32  active ready running

|-+- policy='service-time 0' prio=1 status=enabled

| `- 5:0:3:1 sdk 8:160 active ready running

|-+- policy='service-time 0' prio=1 status=enabled

| `- 6:0:2:1 sds 65:32 active ready running

`-+- policy='service-time 0' prio=1 status=enabled

`- 6:0:3:1 sdu 65:64 active ready running

mpathd (360022a11000cb6b10a902db200000000) dm-0 HUAWEI  ,S5500T

size=2.0T features='0' hwhandler='0' wp=rw

|-+- policy='service-time 0' prio=1 status=active

| `- 5:0:0:0 sdb 8:16  active ready running

|-+- policy='service-time 0' prio=1 status=enabled

| `- 5:0:3:0 sdj 8:144 active ready running

|-+- policy='service-time 0' prio=1 status=enabled

| `- 6:0:2:0 sdr 65:16 active ready running

`-+- policy='service-time 0' prio=1 status=enabled

`- 6:0:3:0 sdt 65:48 active ready running

.........

suse11a:~ # dmsetup ls |sort

mpatha  (253:3)

mpathb  (253:4)

mpathc  (253:2)

mpathc_part1    (253:5)

mpathd  (253:0)

mpathd_part1    (253:6)

mpathe  (253:1)

mpathe_part1    (253:7)

#以下命令均可以用于校验多路径设备# ll /dev/disk/by-id/

# ls -l /dev/dm-*

# ll /dev/mapper/

#/dev/dm-*下面的为动态名，即系统重启后会发生变化，此部分用于os 内部使用。

#/dev/mapper/下面查看到的内容为持久化名称，也即是我们在mulitpatch.conf文件中定义的。

四、配置RAW#在Oracle 10g R2 rac安装的过程中，OUI不能够校验共享的快涉笔，因此需要使用raw。Oracle 11g R1后可以直接使用多路径设备。

#使用udev来管理raw，编辑raw文件，如下suse11a:~ # cat /etc/raw | grep -v \#

raw1:mapper/mpatha

raw2:mapper/mpathb

raw3:mapper/mpathc

raw4:mapper/mpathd

raw5:mapper/mpathe

#重启raw以及配置开机启动rawsuse11a:~ # rcraw start

suse11a:~ # chkconfig raw on

suse11a:~ # raw -qa

/dev/raw/raw1:  bound to major 253, minor 3

/dev/raw/raw2:  bound to major 253, minor 4

/dev/raw/raw3:  bound to major 253, minor 2

/dev/raw/raw4:  bound to major 253, minor 0

/dev/raw/raw5:  bound to major 253, minor 1

#测试raw的可访问性# dd if=/dev/zero of=/dev/raw/raw1 bs=1024 count=100

100+0 records in

100+0 records out

102400 bytes (102 kB) copied, 0.762352 seconds, 134 kB/s

#配置权限，该权限用于oracle用户访问裸设备# Author : Leshami

# Blog   :http://blog.csdn.net/leshami

suse11a:~ # more /etc/udev/rules.d/99-raw.rules

KERNEL=="raw[1-2]*", GROUP="oinstall", MODE="664"

KERNEL=="raw[3-5]*", OWNER="oracle", GROUP="oinstall", MODE="664"

五、安装clusterware#安装clusterware时，关于ocr和votingdisk，选择/dev/raw/raw1，/dev/raw/raw2，如果有配置冗余，此处应该为多个裸设备。

#校验安转好clusterware后裸设备的使用情况suse11a:oracle:ora10g1 > ocrcheck

Status of Oracle Cluster Registry is as follows :

Version                  :          2

Total space (kbytes)     :    1048296

Used space (kbytes)      :       4592

Available space (kbytes) :    1043704

ID                       :  892902908

Device/File Name         : /dev/raw/raw1

Device/File integrity check succeeded

Device/File not configured

Cluster registry integrity check succeeded

suse11a:oracle:ora10g1 > crsctl query  css votedisk

0.     0    /dev/raw/raw2

located 1 votedisk(s).

#对于ASM磁盘可以通过dbca来创建，直接使用/dev/raw/raw*以及/dev/mapper/*。当然suse 11支持asmlib，也可以通过asmlib方式来访问asm磁盘。