多路径（multi-path）安装测试实例

1.确保安装以下的包：

device-mapper            

device-mapper-multipath 

 

[root@nticket1~]# rpm -qa "*device*"

device-mapper-multipath-0.4.9-56.0.3.el5

device-mapper-multipath-libs-0.4.9-56.0.3.el5

device-mapper-1.02.67-2.el5

device-mapper-event-1.02.67-2.el5

device-mapper-1.02.67-2.el5

 

2.编辑配置文件/etc/multipath.conf

要创建一个multipath.conf的配置文件，该文件在安装后不会自动创建。可以使用一个模板创建一个multipath.conf的文件。

 

[root@nticket1 ~]# cp /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf.synthetic  /etc/multipath.conf

 

defaults

{

       user_friendly_names       yes

       max_fds                   max

       queue_without_daemon      no

       flush_on_last_del         yes

}

 

devices {

       device {

               vendor                  "HUAWEI "

               product                 "S2300E"

               path_grouping_policy    multibus

               getuid_callout          "/sbin/scsi_id -g -u -s/block/%n"

               path_checker            directio

               path_selector           "round-robin 0"

               hardware_handler        "0"

               failback                15

               rr_weight               priorities

               no_path_retry           queue

               rr_min_io               100

               product_blacklist       LUNZ

       }

}

 

 

3.加载并启动multipath模块和服务

# modprobe dm_multipath

# lsmod |grep dm_multipath

dm_multipath           19157  0

dm_mod                 81987  3 dm_multipath,dm_mirror,dm_log

 

# service multipathd restart

# chkconfig --level 345 multipathd on

# chkconfig --list | grep multipathd

multipathd      0:关闭  1:关闭 2:关闭  3:启用  4:启用 5:启用  6:关闭

# multipath -v3  //装配多路径设备

 

4.检查multipath聚合后的设备名，以及设备对应的链路情况。

[root@nticket1~]# multipath -ll //显示当前多路径拓扑结构

Dec 20 16:31:21| multipath.conf line 73, invalid keyword: {

mpath2 (360022a11000659d858abb41000000001)dm-1 HUAWEI,S2300E

size=200G features='0' hwhandler='0' wp=rw

|-+-policy='round-robin 0' prio=1 status=active

| `- 1:0:0:1 sdc8:32  active ready running

|-+-policy='round-robin 0' prio=1 status=enabled

| `- 1:0:1:1 sde8:64  active ready running

|-+-policy='round-robin 0' prio=1 status=enabled

| `- 2:0:0:1 sdg8:96  active ready running

`-+-policy='round-robin 0' prio=1 status=enabled

  `- 2:0:1:1 sdi 8:128 active ready running

mpath1 (360022a11000659d858abb3f200000000) dm-0 HUAWEI,S2300E

size=200G features='0' hwhandler='0' wp=rw

|-+-policy='round-robin 0' prio=1 status=active

| `- 1:0:0:0 sdb8:16  active ready running

|-+-policy='round-robin 0' prio=1 status=enabled

| `- 1:0:1:0 sdd8:48  active ready running

|-+- policy='round-robin0' prio=1 status=enabled

| `- 2:0:0:0 sdf8:80  active ready running

`-+-policy='round-robin 0' prio=1 status=enabled

  `- 2:0:1:0 sdh 8:112 active ready running

 

以上的结果可以看出，每个磁盘设备对应的是4条链路。每个设备都有两个multipath设备名，分别是mpath[x]和dm-[x]，mpath位于/dev/mpath目录下，dm-[x]位于/dev目录下。

 

[root@nticket1 dev]# ls dm*

dm-0  dm-1

[root@nticket1 mpath]# ll

总计 0

lrwxrwxrwx 1 root root 7 12-20 15:01 360022a11000659d858abb3f200000000 -> ../dm-0

lrwxrwxrwx 1 root root 7 12-20 15:01 360022a11000659d858abb41000000001 -> ../dm-1

 

配置正确的话就会在/dev/mapper/目录下多出mpath1、mpath2等之类设备

[root@nticket1 ~]# cd /dev/mapper/

[root@nticket1 mapper]# ll

总计 0

crw------- 1 root root  10, 236 12-2010:32 control

brw-rw---- 1 root disk 252,   0 12-2015:01 mpath1

brw-rw---- 1 root disk 252,   1 12-2015:01 mpath2

 

用fdisk -l命令也可以看到多路径软件创建的磁盘，如下面中的/dev/dm-[0-1]

[root@nticket1 mapper]# fdisk -l

 

Disk /dev/sda: 1127.4 GB, 1127428915200 bytes

255 heads, 63 sectors/track, 137068 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

   Device Boot      Start         End      Blocks  Id  System

/dev/sda1   *           1          13      104391  83  Linux

/dev/sda2              14      130694 1049695132+  83  Linux

/dev/sda3          130695      137068   51199155   82  Linux swap / Solaris

 

Disk /dev/sdb: 214.7 GB, 214748364800 bytes

255 heads, 63 sectors/track, 26108 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

Disk /dev/sdb doesn't contain a valid partition table

......

......

Disk /dev/sdi: 214.7 GB, 214748364800 bytes

255 heads, 63 sectors/track, 26108 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

Disk /dev/sdi doesn't contain a valid partition table

 

Disk /dev/dm-0: 214.7 GB, 214748364800 bytes

255 heads, 63 sectors/track, 26108 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

Disk /dev/dm-0 doesn't contain a valid partition table

 

Disk /dev/dm-1: 214.7 GB, 214748364800 bytes

255 heads, 63 sectors/track, 26108 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

 

Disk /dev/dm-1 doesn't contain a valid partition table

 

5.multipath磁盘的基本操作

要对多路径软件生成的磁盘进行操作直接操作/dev/mapper/目录下的磁盘就行,

在对多路径软件生成的磁盘进行分区之前最好运行一下pvcreate命令:

# pvcreate /dev/mapper/mpath1

# fdisk /dev/mapper/mpath1

 

用fdisk对多路径软件生成的磁盘进行分区保存时会有一个报错,此报错不用理会，

这时所生成的磁盘分区没有马上添加到/dev/mapper目录下。

[root@nticket1 mapper]# ls

control  mpath1  mpath2

 

此时需要重启IPSAN或者FCSAN的驱动，或重启服务器，完成后会看到分区盘。

[root@nticket1 mapper]# ls

control  mpath1  mpath1p1 mpath2  mpath1p2

 

上面是一个简单的multipath配置，分区完成后就可以对mpath1p1、mpath2p1进行使用了，如格式化挂载到文件系统，或提供ASM使用。如下：

# mkfs.ext3 /dev/mapper/mpath0p1 #对mpath1p1分区格式化成ext3文件系统
# mount /dev/mapper/mpath0p1 /ipsan/ #挂载mpath1p1分区

[root@racdb1 mapper]# mkfs.ext3 /dev/mapper/mpath8
[root@racdb1 mapper]# mkdir /test1
[root@racdb1 mapper]# mount /dev/mapper/mpath8 /test1
[root@testdb ~]# vi /etc/fstab
/dev/mapper/mpath8  /test1  ext3 defaults 0 0 

6.路径切换测试

最后可以通过拔掉存储和SAN光纤交换机之间的光纤，或交换机和服务机之间的光纤来测试路径切换，使用iostat查看测试。

 

相关命令和参数详细说明参考下面的几篇文章：

http://wenku.baidu.com/link?url=lUZB5ffPOLNYGbLo4Kv-5c4D7NaUP8U7vlA6NO0A92NRfVqwBsT1Lg1TTBL84TtnABNDJoCyPWExxDQlBUedmcKu6x2uajB1ih2m2Qbj6CW

http://blog.itpub.net/23135684/viewspace-745789

http://blog.csdn.net/lichangzai/article/details/17501035