Linux 软raid auto-detect

一、初探

Boot time assembly of RAID arrays
---------------------------------

Tools that manage md devices can be found at
   http://www.kernel.org/pub/linux/utils/raid/


You can boot with your md device with the following kernel command
lines:

for old raid arrays without persistent superblocks::

  md=<md device no.>,<raid level>,<chunk size factor>,<fault level>,dev0,dev1,...,devn

for raid arrays with persistent superblocks::

  md=<md device no.>,dev0,dev1,...,devn

or, to assemble a partitionable array::

  md=d<md device no.>,dev0,dev1,...,devn

``md device no.``
+++++++++++++++++

The number of the md device

================= =========
``md device no.`` device
================= =========
              0		md0
	      1		md1
	      2		md2
	      3		md3
	      4		md4
================= =========

``raid level``
++++++++++++++

level of the RAID array

=============== =============
``raid level``  level
=============== =============
-1		linear mode
0		striped mode
=============== =============

other modes are only supported with persistent super blocks

``chunk size factor``
+++++++++++++++++++++

(raid-0 and raid-1 only)

Set  the chunk size as 4k << n.

``fault level``
+++++++++++++++

Totally ignored

``dev0`` to ``devn``
++++++++++++++++++++

e.g. ``/dev/hda1``, ``/dev/hdc1``, ``/dev/sda1``, ``/dev/sdb1``

A possible loadlin line (Harald Hoyer <HarryH@Royal.Net>)  looks like this::

	e:\loadlin\loadlin e:\zimage root=/dev/md0 md=0,0,4,0,/dev/hdb2,/dev/hdc3 ro


Boot time autodetection of RAID arrays
--------------------------------------

When md is compiled into the kernel (not as module), partitions(分区) of
type 0xfd are scanned and automatically assembled into RAID arrays.
This autodetection may be suppressed(压制) with the kernel parameter
``raid=noautodetect``.  As of kernel 2.6.9, only drives with a type 0
superblock can be autodetected and run at boot time.

The kernel parameter ``raid=partitionable`` (or ``raid=part``) means
that all auto-detected arrays are assembled as partitionable(可分区的).

Boot time assembly of degraded/dirty arrays
-------------------------------------------

If a raid5 or raid6 array is both dirty and degraded, it could have
undetectable data corruption.  This is because the fact that it is
``dirty`` means that the parity cannot be trusted, and the fact that it
is degraded means that some datablocks are missing and cannot reliably
be reconstructed (due to no parity).

For this reason, md will normally refuse to start such an array.  This
requires the sysadmin to take action to explicitly start the array
despite possible corruption.  This is normally done with::

   mdadm --assemble --force ....

This option is not really available if the array has the root
filesystem on it.  In order to support this booting from such an
array, md supports a module parameter ``start_dirty_degraded`` which,
when set to 1, bypassed the checks and will allows dirty degraded
arrays to be started.

So, to boot with a root filesystem of a dirty degraded raid 5 or 6, use::

   md-mod.start_dirty_degraded=1

Documentation/admin-guide/md.rst

从上述描述来看，只有分区类型为0xfd且超级块类型为0.0的分区才能被自动探测并组装成raid，那如何修改分区type？

答：fdisk命令可以修改分区Id。

curtis # fdisk /dev/sdb

Command (m for help): m
Command action
   a   toggle a bootable flag
   b   edit bsd disklabel
   c   toggle the dos compatibility flag
   d   delete a partition
   l   list known partition types
   m   print this menu
   n   add a new partition
   o   create a new empty DOS partition table
   p   print the partition table
   q   quit without saving changes
   s   create a new empty Sun disklabel
   t   change a partition's system id   <-- 这个选项。
   u   change display/entry units
   v   verify the partition table
   w   write table to disk and exit
   x   extra functionality (experts only)

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

fdisk默认的磁盘ID是什么呢？

root@raspberrypi:/home/curtis# fdisk -l /dev/sdb
Disk /dev/sdb: 14.84 GiB, 15938355200 bytes, 31129600 sectors
Disk model: Flash Disk
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xcad4ebea

Device     Boot   Start     End Sectors Size Id Type
/dev/sdb1          2048 2099199 2097152   1G 83 Linux
/dev/sdb2       2099200 4196351 2097152   1G 83 Linux
/dev/sdb3       4196352 8390655 4194304   2G 83 Linux

答：Linux默认的Id为83。

fdisk 创建的逻辑分区lsblk查看分区大小为1K？这是为什么？

sdc      8:32   1  58.6G  0 disk
├─sdc1   8:33   1     1G  0 part
├─sdc2   8:34   1     1G  0 part
├─sdc3   8:35   1     2G  0 part
└─sdc4   8:36   1     1K  0 part

由于历史原因，各个操作系统为共存的需要而约定俗成，一个物理磁盘最多有4个主分区。
比如第一块物理磁盘，它一般会有启动分区部分用来包含系统。其余的可以作其他用途。
它可能最多包含四个主分区：
第一区：主分区（被设为活动者，启动时，此分区被引导
第二区：主分区
第三区：主分区
第四区：主分区
也可能是这样：
第一区： 主分区
第二区： 扩展分区
而扩展分区 又分为 ： 逻辑分区1 ， 逻辑分区2，逻辑分区3 ，逻辑分区4…
如果仅仅是为了存放数据，也可能是这样：
拓展分区：逻辑分区1 ， 逻辑分区2，逻辑分区3 ，逻辑分区4…

第一块物理硬盘中，主分区是必须的，因为要引导系统嘛，这是没有办法的事。
对第二块物理硬盘， 可以有几种方法去使用：

• 主分区 + 扩展分区（里面分成逻辑分区1，逻辑分区2…）
  此方式下，此处主分区似乎有点不伦不类。当然也是可以当作数据区使用。
• 仅有一个 主分区，拿来直接当数据区用也是可以的。
• 完全是扩展分区。（里面分成逻辑分区1，逻辑分区2…）
  要注意： 仅有扩展分区，不进一步划分逻辑分区，是不能拿来做文件系统的，也就是为什么看到分区大小为1K的原因。

https://www.cnblogs.com/gaojian/archive/2012/10/19/2730958.html

也就是说创建拓展分区之后还需要创建逻辑分区，才能被正常使用，lsblk看到的信息只是一个标记而已。

root@raspberrypi:/home/curtis# fdisk /dev/sdc

Welcome to fdisk (util-linux 2.36.1).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): n
All primary partitions are in use.
Adding logical partition 5
First sector (8392704-122909695, default 8392704):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (8392704-122909695, default 122909695): +3G

Created a new partition 5 of type 'Linux' and of size 3 GiB.

Command (m for help): n
All primary partitions are in use.
Adding logical partition 6
First sector (14686208-122909695, default 14686208):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (14686208-122909695, default 122909695): +2G

Created a new partition 6 of type 'Linux' and of size 2 GiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

root@raspberrypi:/home/curtis# lsblk /dev/sdc
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sdc      8:32   1 58.6G  0 disk
├─sdc1   8:33   1    1G  0 part
├─sdc2   8:34   1    1G  0 part
├─sdc3   8:35   1    2G  0 part
├─sdc4   8:36   1    1K  0 part
├─sdc5   8:37   1    3G  0 part
└─sdc6   8:38   1    2G  0 part

磁盘layout类似于：sdc: sdc1 sdc2 sdc3 sdc4 < sdc5 sdc6 >

分完区之后，如果一个个删除分区比较麻烦，有没有一键刷掉所有分区的命令？
wipefs --all /dev/sdx。

DESCRIPTION
       wipefs  can  erase  filesystem, raid or partition-table signatures (magic strings) from the specified device to make
       the signatures invisible for libblkid.  wipefs does not erase the filesystem itself nor any other data from the  de‐
       vice.

       When  used without any options, wipefs lists all visible filesystems and the offsets of their basic signatures.  The
       default output is subject to change.  So whenever possible, you should avoid using default outputs in your  scripts.
       Always  explicitly  define  expected columns by using --output columns-list in environments where a stable output is
       required.

       wipefs calls the BLKRRPART ioctl when it has erased a partition-table signature  to  inform  the  kernel  about  the
       change.  The  ioctl  is called as the last step and when all specified signatures from all specified devices are al‐
       ready erased.  This feature can be used to wipe content on partitions devices as well as partition table on  a  disk
       device, for example by wipefs -a /dev/sdc1 /dev/sdc2 /dev/sdc.

       Note  that  some filesystems and some partition tables store more magic strings on the device (e.g., FAT, ZFS, GPT).
       The wipefs command (since v2.31) lists all the offset where a magic strings have been detected.

       When option -a is used, all magic strings that are visible for libblkid are erased. In this case  the  wipefs  scans
       the device again after each modification (erase) until no magic string is found.

       Note  that  by default wipefs does not erase nested partition tables on non-whole disk devices.  For this the option
       --force is required.

使用下面命令组建的软raid，内核启动时为什么Scanned设备为0？

mdadm --verbose --create /dev/md0 --raid-devices 2 --level 1 /dev/sdb1 /dev/sdc1 --bitmap internal --metadata=0.9

[   12.547690] md: Waiting for all devices to be available before autodetect
[   12.547839] md: If you don't use raid, use raid=noautodetect
[   12.556435] md: Autodetecting RAID arrays.
[   12.556623] auto detect devices list is empty
[   12.556815] md: Scanned 0 and added 0 devices.
[   12.557332] md: autorun ...
[   12.557575] md: ... autorun DONE.

答：Linux内核会扫描所有磁盘，查找分区为0xfd类型，如果不包含分区类型为0xfd的分区，0xfd类型分区链表为空。

// init/do_mounts_md.c

|-  prepare_namespace(void);
	|- md_run_setup(void);
		|- autodetect_raid(void);

在init/do_mounts_md.c定义了raid自动检测的全局变量。

#ifdef CONFIG_MD_AUTODETECT
static int __initdata raid_noautodetect;
#else
static int __initdata raid_noautodetect=1;
#endif

如果CONFIG_MD_AUTODETECT=y选项在编译时被打开，raid_noautodetect被初始化为0，表示需要自动探测软raid。

static int __init raid_setup(char *str)
{
	int len, pos;

	len = strlen(str) + 1;
	pos = 0;

	while (pos < len) {
		char *comma = strchr(str+pos, ',');
		int wlen;
		if (comma)
			wlen = (comma-str)-pos;
		else	wlen = (len-1)-pos;

		if (!strncmp(str, "noautodetect", wlen))
			raid_noautodetect = 1;
		if (!strncmp(str, "autodetect", wlen))
			raid_noautodetect = 0;
		if (strncmp(str, "partitionable", wlen)==0)
			raid_autopart = 1;
		if (strncmp(str, "part", wlen)==0)
			raid_autopart = 1;
		pos += wlen+1;
	}
	return 1;
}

查一查man手册，里面也有一段自动detect的描述，mdadm命令也可以通过ioctl的方式出发自动扫描流程。

       --auto-detect
         Request that the kernel starts any auto-detected arrays.  This can only work if md is compiled into the kernel — not if it is a module. Arrays can  be  auto-detected  by the kernel if all the components are in primary MS-DOS partitions with partition type FD, and all use v0.90 metadata. In-kernel autodetect is not recommended for new installations.  Using mdadm to detect and assemble arrays — possibly in an initrd — is  substan‐tially(实质上) more flexible and should be preferred.

二、软raid auto-detect前置条件

1、内核需要将auto-detect内核配置选项打开（CONFIG_MD_AUTODETECT=y）。

2、软raid构建时需要指定metadata的版本为0.9。

3、磁盘分区的类型为0xfd（Linux raid autodetect），可以用fdisk -l查看分区类型。

内核日志探测：

[   10.832687] md: Autodetecting RAID arrays.
[   10.841583] md: Scanned 2 and added 2 devices.
[   10.842251] md: autorun ...
[   10.842819] md: considering sdc1 ...
[   10.843163] md:  adding sdc1 ...
[   10.843163] md:  adding sdb1 ...
[   10.844230] md: created md0
[   10.844906] md: bind<sdb1>
[   10.846130] md: bind<sdc1>
[   10.846956] md: running: <sdc1><sdb1>
[   10.857141] md/raid1:md0: active with 2 out of 2 mirrors
[   10.871776] md0: detected capacity change from 0 to 1073676288
[   10.873053] md: ... autorun DONE.

三、关闭auto-detect

raid_setup函数会根据内核启动参数来设定raid自动探测相关参数，也就是说即使在编译内核时将对应的配置选项打开，也可以通过内核参数将该能力关闭。

需要在内核启动参数添加：

raid=noautodetect

四、通过添加内核参数组建软raid

在组建好软raid之后，需要在内核启动参数添加：

md=0,/dev/vdb,/dev/vdc