Multipath and Device Mapper

Reference

http://en.wikipedia.org/wiki/Multipath_I/O

http://en.wikipedia.org/wiki/Device-Mapper

http://www.linuxsymposium.org/2005/linuxsymposium_procv1.pdf

http://sources.redhat.com/lvm2/wiki/MultipathUsageGuide *** 比较详细的手册

什么是multipath

In computer storage, multipath I/O is a fault-tolerance and performance enhancement technique whereby there is more than one physical path between the CPU in a computer system and its mass storage devices through the buses, controllers, switches, and bridge devices connecting them.

主要用于 容错 和 性能提升。

Device Mapper

在linux中，用device mapper实现 multipath。但device mapper本身，不仅仅只是multipath。

下面是个简单的介绍。

Device mapper is a block subsystem that provides layering mechanism for block devices. One can write a device mapper to provide a specific functionality on top of a block device.

Currently the following functional layers are available:

concatenation
mirror
striping
encryption
flaky
delay
multipath

Multiple device mapper modules can be stacked to get the combined functionality.

我的这篇文章讲了一点DM的内容

http://blog.csdn.net/richardysteven/article/details/7937249

Device Mapper- Multipath包含哪些东西

DM-MP consists of 4 components:

DM MP kernel module - Kernel module that is responsible for making the multipathing decisions in normal and failure situations.

内核模块，io重定向

multipath command - User space tool that allows the user with initial configuration, listing and deletion of multipathed devices.

用户程序，查看/配置MP设备。

multipathd daemon - User space daemon that constantly monitors the paths. It marks a path as failed when it finds the path faulty and if all the paths in a priority group are faulty then it switches to the next enable priority group. It keeps checking the failed path, once the failed path comes alive, based on the failback policy, it can activate the path. It provides an CLI to monitor/manage individual paths. It automatically creates device mapper entries when new devices comes into existence.

监听链路失败信息，进行链路切换。

kpartx - User space command that creates device mapper entries for all the partitions in a multipathed disk/LUN. When the multipath command is invoked, this command automatically gets invoked. For DOS based partitions this command need to be run manually.

根据设备分区表建立device mapper设备。

multipah 命令的使用

那先从用户层的命令开始看，看看一般怎么使用。

显示multipath的topology

# multipath -l

mpatha (350000393c8205324) dm-5 IBM,MBF2600RC
size=559G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=0 status=active
| `- 0:0:5:0 sdb 8:16  active undef running
`-+- policy='round-robin 0' prio=0 status=enabled
  `- 1:0:5:0 sdh 8:112 active undef running

mpatha    用户定义的multipath的名字

dm-5        sysfs中multipath的名字

IBM,MBFXX  设备厂商 型号

policy      各个通路的策略

prio          优先级

一个multipath设备 可以有多个 multipath group, 一个 multipath group 可以 有多个 path.

Path

Connection from the server through a HBA to a specific LUN. Without DM-MP, each path would appear as a separate device.

Path Group

Paths are grouped into a path groups. At any point of time only path group will be active.Path selector decides which path in the path group gets to send the next I/O. I/O will be sent only to the active path.

Path States

This refers to the physical state of a path. A path can be in one of the following states:

1. ready: Path is up and can handle I/O requests.

2. faulty: Path is down and cannot handle I/O requests.

3. ghost: Path is a passive path. This state is shown in the passive path inActive/Passive mode.

4. shaky: Path is up, but temporarily not available for I/O requests.

DM Path States

This refers to the DM module(kernel)'s view of the path's state. It can be in one of the two states:

1. active: Last I/O sent to this path successfully completed. Analogous toready path state.

2. failed: Last I/O to this path failed. Analogous to faulty path state.

Path Group State

Path Groups can be in one of the following three states:

1. active: I/O will be sent to the multipath device will be sent to this path group. Only one path group will be in this state.

2. enabled: If none of the paths in the active path group is in the ready state, I/O will be sent these path groups. There can be one or more path groups in this state.

3. disabled: In none of the paths in the active path group and enabled path group is in the ready state. I/O will be sent to these path groups. There can be one or more path groups in this state. This state is available only for certain storage devices.

配置文件 /etc/multipath.conf

This file has 5 sections:

1. System level defaults ("defaults"): Where the user can specify system level default override.

2. Black listed devices ("blacklist"): User can specify the list of devices they do not want to be under the control of DM-Multipath. These devices will be excluded.

3. Black list exceptions ("blacklist_exceptions"): Specific devices to be treated as multipath candidates even if they exist in the blacklist.

4. Storage controller specific settings ("devices"): User specified configuration settings will be applied to devices with specified "Vendor" and "Product" information.

5. Device specific settings ("multipaths"): User can fine tune configuration settings for individual LUNs.