linux之磁盘阵列实战
一、磁盘阵列概述
磁盘阵列(Redundant Arrays of Independent Disks,RAID),有“独立磁盘构成的具有冗余能力的阵列”之意。是防止硬盘突然损坏导致数据丢失的一种冗余备份机制。
磁盘阵列有三种形式:外接式磁盘阵列柜、内接式磁盘阵列卡,利用软件来仿真。
二、常见磁盘阵列RAID类型:
1、RAID 0
RAID 0(也称条带模式)是组建磁盘阵列中最简单的一种形式,只需要2块以上的硬盘即可,成本低,可以提高整个磁盘的性能和吞吐量。RAID 0没有提供冗余或错误修复能力,但实现成本是最低的。如下图,如数据量为10G,第一块硬盘存储5G,第二块硬盘存储剩下的5G。当其中一块硬盘损坏时,数据就损坏了。生产环境不建议使用。
2、RAID 1
RAID 1(也称磁盘镜像),是把一个磁盘的数据镜像到另一个磁盘上,也就是说数据在写入一块磁盘的同时,会在另一块闲置的磁盘上生成镜像文件。磁盘使用率为50%,其中一块磁盘损坏,不会影响数据,如下图。以四块80GB容量的硬盘来讲,可利用的磁盘空间仅为160GB。RAID 1多用在保存关键性的重要数据的场合。
3、RAID 5
RAID 5是分布式奇偶校验的独立磁盘结构。如下图,它的奇偶校验码存在于所有磁盘上,其中的p0代表第0带区的奇偶校验值,至少需要三块硬盘,第一块磁盘和第二块磁盘做一个异或运算得到一个校验位,每两块磁盘做一次异或运算,这样在一块磁盘损坏的时候,根据其余两块磁盘就可以推算出损坏的部分,从而保证了数据的完整性。磁盘使用率为(n-1)/ n ,工作中使用最多的就是raid 5的模式。
4、RAID 10
Raid 10是一个Raid 0与Raid1的组合体,它是利用奇偶校验实现条带集镜像,所以它继承了Raid0的快速和Raid1的安全,最少需要4块硬盘。如下图
三、各RAID类型实战
注:通过以上类型描述,以下均采用软RAID实现。创建软RAID命令为mdadm
mdadm命令常用参数如下:
-C或--creat:建立一个新阵列
-A:激活磁盘阵列
-D或--detail:打印阵列设备的详细内容
-s或--scan:扫描配置文件或/proc/mdstat得到阵列缺失信息
-f:将设备状态定为故障
-a或--add:添加设备到阵列
-v或--verbose:显示详细信息
-r:移除设备
-l或--level=:设定磁盘阵列的级别
-n或--raid-devices=:设定阵列成员(分区/磁盘)的数量
-x或--space-devices=:设定阵列中备用磁盘数量
-c或--chunk=:设定阵列的块chunk大小,单位为KB。chunk(块),RAID的存储数据时每个数据段大小为4KB、64KB
-G或--grow:改变阵型大小或形态
1、RAID 0
示例要求:创建RAID 0;格式化并且永久挂载到指定目录。
示例环境:准备2块磁盘,/dev/sdb /dev/sdc,每一个硬盘上分出一个5G的分区,在/mnt目录下创建RAID0目录(mkdir -pv /mnt/RAID0,供挂载使用)
1)查看当前硬盘信息
[root@hc03 ~]# fdisk -l Disk /dev/sda: 42.9 GB, 42949672960 bytes 255 heads, 63 sectors/track, 5221 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000ecb54 Device Boot Start End Blocks Id System /dev/sda1 * 1 103 819200 83 Linux Partition 1 does not end on cylinder boundary. /dev/sda2 103 3927 30720000 8e Linux LVM Disk /dev/sdb: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdc: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000
2)为sdb硬盘分区,创建新主分区sdb1
[root@hc03 ~]# fdisk /dev/sdb Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0x19ab7785. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').
3)输入n回车新建分区,接着再输入p回车新建主分区
Command (m for help): n Command action e extended p primary partition (1-4) p
4)此处要求选择分区号在1-4间,输入1回车
Partition number (1-4): 1 First cylinder (1-1305, default 1): Using default value 1
5)设置分区分区大小为5G,输入+5G后回车
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): +5G
6)输入p查看分区是否成功,输入p回车,如下图:显示分区成功
Command (m for help): p Disk /dev/sdb: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x19ab7785 Device Boot Start End Blocks Id System /dev/sdb1 1 654 5253223+ 83 Linux
7)最后输入w回车,进行保存退出
Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
8)按上述方法,为硬盘sdc创建5G大小sdc1分区
Command (m for help): p Disk /dev/sdc: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x356d6b05 Device Boot Start End Blocks Id System /dev/sdc1 1 654 5253223+ 83 Linux
9)使用mdadm命令把sdb/sdc硬盘分别sdb1、sdc1分区创建为RAID 0
[root@hc03 ~]# mdadm -C -v /dev/md0 -l 0 -n 2 /dev/sdb1 /dev/sdc1 mdadm: chunk size defaults to 512K mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md0 started. 注释:-C创建新磁盘阵列,-l 0 为RAID类型为0,2为硬盘个数。
10)查看阵列信息,命令mdadm -Ds,以及创建raid 0后/dev/md0硬盘分区大小信息
[root@hc03 ~]# mdadm -Ds ARRAY /dev/md0 metadata=1.2 name=hc03:0 UUID=b524bc38:194dae34:41d4adec:24b3a69e [root@hc03 ~]# fdisk -l /dev/md0 Disk /dev/md0: 10.8 GB, 10750001152 bytes 2 heads, 4 sectors/track, 2624512 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 524288 bytes / 1048576 bytes Disk identifier: 0x00000000
11)为/dev/md0新生成硬盘分区(与上述sdb或sdc分区相同)
[root@hc03 ~]# fdisk /dev/md0 Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0xe58709bd. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u'). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-2624512, default 257): Using default value 257 Last cylinder, +cylinders or +size{K,M,G} (257-2624512, default 2624512): Using default value 2624512 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
12)查看创建后的分区信息
[root@hc03 ~]# fdisk -l /dev/md* last_lba(): I don't know how to handle files with mode 40755 Disk /dev/md0: 10.8 GB, 10750001152 bytes 2 heads, 4 sectors/track, 2624512 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 524288 bytes / 1048576 bytes Disk identifier: 0xe58709bd Device Boot Start End Blocks Id System /dev/md0p1 257 2624512 10497024 83 Linux Disk /dev/md0p1: 10.7 GB, 10748952576 bytes 2 heads, 4 sectors/track, 2624256 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 524288 bytes / 1048576 bytes Disk identifier: 0x00000000
13)格式化/dev/md0p1
[root@hc03 ~]# mkfs.ext4 /dev/md0p1 mke2fs 1.41.12 (17-May-2010) Filesystem label= #文件系统标签 OS type: Linux #操作系统 Block size=4096 (log=2) #块大小 Fragment size=4096 (log=2) Stride=128 blocks, Stripe width=256 blocks 657072 inodes, 2624256 blocks 131212 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=2688548864 81 block groups 32768 blocks per group, 32768 fragments per group 8112 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 Writing inode tables: done #写入inode表 Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 29 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override.
14)设置开机启动自动挂载/dev/md0p1至/dev/RAID0目录
[root@hc03 ~]# mount /dev/md0p1 /mnt/RAID0 [root@hc03 ~]# mount -l /dev/mapper/my03-LogVol01 on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sda1 on /boot type ext4 (rw) /dev/mapper/my03-LogVol02 on /usr type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) /dev/md0p1 on /mnt/RAID0 type ext4 (rw) #挂载成功 [root@hc03 ~]# cat /etc/fstab #加入开机启动自动挂载 # # /etc/fstab # Created by anaconda on Sat Oct 31 03:45:15 2015 # # Accessible filesystems, by reference, are maintained under '/dev/disk' # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/my03-LogVol01 / ext4 defaults 1 1 UUID=c06d0732-24b8-4dfa-baba-f5ed54bea494 /boot ext4 defaults 1 2 /dev/mapper/my03-LogVol02 /usr ext4 defaults 1 2 /dev/mapper/my03-LogVol00 swap swap defaults 0 0 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 /dev/md0p1 /mnt/RAID0 ext4 defaults 0 0
2、RAID 1
示例要求:创建RAID 1;开机自动挂载至/mnt/RAID1并准备一个热备盘,模拟RAID 1中一个硬盘故障,并移除阵列。
示例环境:准备3块磁盘,/dev/sdb、/dev/sdc、/dev/sdd,每一个硬盘上分出一个2G的分区,分区以此/dev/sdb2、/dev/sdc2、/dev/sdd1
1)查看当前硬盘信息
[root@hc03 ~]# fdisk -l Disk /dev/sda: 42.9 GB, 42949672960 bytes 255 heads, 63 sectors/track, 5221 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x000ecb54 Device Boot Start End Blocks Id System /dev/sda1 * 1 103 819200 83 Linux Partition 1 does not end on cylinder boundary. /dev/sda2 103 3927 30720000 8e Linux LVM Disk /dev/sdb: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x19ab7785 Device Boot Start End Blocks Id System /dev/sdb1 1 654 5253223+ 83 Linux Disk /dev/sdc: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x356d6b05 Device Boot Start End Blocks Id System /dev/sdc1 1 654 5253223+ 83 Linux Disk /dev/sdd: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000
2)为sdb硬盘分区,创建新主分区sdb2(创建方法同上)
[root@hc03 ~]# fdisk /dev/sdb WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u'). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 2 First cylinder (655-1305, default 655): Using default value 655 Last cylinder, +cylinders or +size{K,M,G} (655-1305, default 1305): +2G Command (m for help): P^H Disk /dev/sdb: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x19ab7785 Device Boot Start End Blocks Id System /dev/sdb1 1 654 5253223+ 83 Linux /dev/sdb2 655 916 2104515 83 Linux Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 16: Device or resource busy. The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8) Syncing disks.
注:建议重启,因为sdb1被作为RAID0并且挂载
3)按上述方法,为硬盘sdc创建2G大小sdc2分区,硬盘sdd创建2G大小sdd1分区
[root@hc03 ~]# fdisk -l /dev/sdc Disk /dev/sdc: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x356d6b05 Device Boot Start End Blocks Id System /dev/sdc1 1 654 5253223+ 83 Linux /dev/sdc2 655 916 2104515 83 Linux [root@hc03 ~]# fdisk /dev/sdd #创建sdd1分区 Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0x96e37a1b. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u'). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-1305, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): +2G Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@hc03 ~]# fdisk -l /dev/sdd Disk /dev/sdd: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x96e37a1b Device Boot Start End Blocks Id System /dev/sdd1 1 262 2104483+ 83 Linux
4)使用mdadm命令把sdb/sdc硬盘分别sdb2、sdc2分区创建为RAID 2,sdd1为热备盘
[root@hc03 ~]# mdadm -C -v /dev/md1 -l 1 -n 2 -x 1 /dev/sdb2 /dev/sdc2 /dev/sdd1 mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90 mdadm: size set to 2102400K Continue creating array? Continue creating array? (y/n) y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started. 注释:-C创建新磁盘阵列,-l 1 为RAID类型为1,2为硬盘个数,-x 1 为热备盘
5)查看阵列信息,命令mdadm -Ds,以及创建raid 0后/dev/md0硬盘分区大小信息
[root@hc03 ~]# mdadm -Ds ARRAY /dev/md/hc03:0 metadata=1.2 name=hc03:0 UUID=b524bc38:194dae34:41d4adec:24b3a69e ARRAY /dev/md1 metadata=1.2 spares=1 name=hc03:1 UUID=ac6a9ef0:abadb228:f9c02876:1fad13c3
6)为/dev/md1新生成硬盘分区(与上述sdb或sdc分区相同)
[root@hc03 ~]# fdisk /dev/md0 [root@hc03 ~]# fdisk /dev/md1 Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0x34b7f63d. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u'). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-525600, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-525600, default 525600): Using default value 525600 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
7)查看创建后的分区信息
[root@hc03 ~]# fdisk -l /dev/md1* Disk /dev/md1: 2152 MB, 2152857600 bytes 2 heads, 4 sectors/track, 525600 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x34b7f63d Device Boot Start End Blocks Id System /dev/md1p1 1 525600 2102398 83 Linux Disk /dev/md127: 10.8 GB, 10750001152 bytes 2 heads, 4 sectors/track, 2624512 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 524288 bytes / 1048576 bytes Disk identifier: 0xe58709bd Device Boot Start End Blocks Id System /dev/md127p1 257 2624512 10497024 83 Linux Disk /dev/md127p1: 10.7 GB, 10748952576 bytes 2 heads, 4 sectors/track, 2624256 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 524288 bytes / 1048576 bytes Disk identifier: 0x00000000 Disk /dev/md1p1: 2152 MB, 2152855552 bytes 2 heads, 4 sectors/track, 525599 cylinders Units = cylinders of 8 * 512 = 4096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000
8)格式化/dev/md1p1
[root@hc03 ~]# mkfs.ext4 /dev/md1p1 mke2fs 1.41.12 (17-May-2010) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 131648 inodes, 525599 blocks 26279 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=541065216 17 block groups 32768 blocks per group, 32768 fragments per group 7744 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912 Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 24 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override.
9)设置开机启动自动挂载/dev/md1p1至/mnt/RAID1目录
[root@hc03 ~]# mkdir -pv /mnt/RAID1 mkdir: created directory `/mnt/RAID1' [root@hc03 ~]# mount /dev/md1p1 /mnt/RAID1 [root@hc03 ~]# mount -l /dev/mapper/my03-LogVol01 on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sda1 on /boot type ext4 (rw) /dev/mapper/my03-LogVol02 on /usr type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) /dev/md1p1 on /mnt/RAID1 type ext4 (rw) #挂载成功 [root@hc03 ~]# cat /etc/fstab # # /etc/fstab # Created by anaconda on Sat Oct 31 03:45:15 2015 # # Accessible filesystems, by reference, are maintained under '/dev/disk' # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/my03-LogVol01 / ext4 defaults 1 1 UUID=c06d0732-24b8-4dfa-baba-f5ed54bea494 /boot ext4 defaults 1 2 /dev/mapper/my03-LogVol02 /usr ext4 defaults 1 2 /dev/mapper/my03-LogVol00 swap swap defaults 0 0 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 /dev/md0p1 /mnt/RAID0 ext4 defaults 0 0 /dev/md1p1 /mnt/RAID1 ext4 defaults 0 0
10)使用/proc/mdstat查看阵列信息
[root@hc03 ~]# cat /proc/mdstat Personalities : [raid0] [raid1] md1 : active raid1 sdd1[2](S) sdc2[1] sdb2[0] 2102400 blocks super 1.2 [2/2] [UU] md127 : active raid0 sdb1[0] sdc1[1] 10498048 blocks super 1.2 512k chunks unused devices: <none> 注释: #sdd1[2](s):其中s表示备用盘 #[2/2]表示raid 1中的两块磁盘都是正常的 #active表示此阵列正常读写
11)生成mdadm.conf配置文件
[root@hc03 ~]# mdadm -Ds > /etc/mdadm.conf [root@hc03 ~]# cat /etc/mdadm.conf ARRAY /dev/md/hc03:0 metadata=1.2 name=hc03:0 UUID=b524bc38:194dae34:41d4adec:24b3a69e ARRAY /dev/md1 metadata=1.2 spares=1 name=hc03:1 UUID=ac6a9ef0:abadb228:f9c02876:1fad13c3 注释: #spares=1 存在一个热备盘
12)模拟/dev/sdc2为故障盘,重启一个终端动态刷新/proc/mdstat配置信息,命令:watch -n 1 cat /proc/mdstat
[root@hc03 ~]# mdadm -f /dev/md1 /dev/sdc2 #设定sdc2为故障 mdadm: set /dev/sdc2 faulty in /dev/md1 [root@hc03 ~]# cat /proc/mdstat Personalities : [raid0] [raid1] md1 : active raid1 sdd1[2] sdc2[1](F) sdb2[0] 2102400 blocks super 1.2 [2/2] [UU] md127 : active raid0 sdb1[0] sdc1[1] 10498048 blocks super 1.2 512k chunks unused devices: <none> 注释: sdc2[1](F) 为故障 sdd1[2]并且已经接替故障盘
13)移除故障盘sdc2,并且更新/etc/mdadm.conf配置文件
[root@hc03 ~]# mdadm -r /dev/md1 /dev/sdc2 mdadm: hot removed /dev/sdc2 from /dev/md1 [root@hc03 ~]# cat /proc/mdstat Personalities : [raid0] [raid1] md1 : active raid1 sdd1[2] sdb2[0] 2102400 blocks super 1.2 [2/2] [UU] md127 : active raid0 sdb1[0] sdc1[1] 10498048 blocks super 1.2 512k chunks unused devices: <none> [root@hc03 ~]# mdadm -Ds > /etc/mdadm.conf
3、RAID 5
示例要求:创建RAID 5;开机自动挂载至/mnt/RAID5并准备一个热备盘,模拟RAID 5中一个硬盘故障,并移除阵列,同时再扩展硬盘。
示例环境:准备5块磁盘,由于资源比较充裕,直接添加新的盘/dev/sdc、/dev/sdd、/dev/sde、/dev/sdf、/dev/sdg,每一个硬盘上分出一个2G的分区,分区以此/dev/sdc1、/dev/sdd1、/dev/sde1、/dev/sdf1、/dev/sdg1
1)创建以上新分区方法,请参照上述相关操作完成,这里不再阐述!
Device Boot Start End Blocks Id System /dev/sdc1 1 262 2104483+ 83 Linux Device Boot Start End Blocks Id System /dev/sdd1 1 262 2104483+ 83 Linux Device Boot Start End Blocks Id System /dev/sde1 1 262 2104483+ 83 Linux Device Boot Start End Blocks Id System /dev/sdf1 1 262 2104483+ 83 Linux Device Boot Start End Blocks Id System /dev/sdg1 1 262 2104483+ 83 Linux
2)使用mdadm命令把sdc/sdd/sde/sdf硬盘分别sdc1/sdd1/sde1分区创建为RAID 5,sdf1为热备盘
[root@lvs ~]# mdadm -C -v /dev/md5 -l 5 -n 3 -x 1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 mdadm: layout defaults to left-symmetric mdadm: layout defaults to left-symmetric mdadm: chunk size defaults to 512K mdadm: size set to 2102272K mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md5 started.
注释:-C创建新磁盘阵列,-l 5 为RAID类型为5,3为硬盘个数,-x 1 为热备盘
3)查看阵列信息,命令mdadm -Ds,以及创建raid 0后/dev/md0硬盘分区大小信息,并生成配置文件/etc/mdadm.conf
[root@lvs ~]# mdadm -Ds > /etc/mdadm.conf ARRAY /dev/md5 metadata=1.2 spares=1 name=lvs:5 UUID=8a62a5ad:9b1fd7ae:09b5f7e2:c0756e62
4)使用/proc/mdstat查看阵列信息
[root@lvs ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md5 : active raid5 sde1[4] sdf1[3](S) sdd1[1] sdc1[0] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] 注释: #sdf1[3](s):其中s表示备用盘 #[3/3]表示raid 1中的两块磁盘都是正常的 #active表示此阵列正常读写
5)模拟/dev/sde1为故障盘,重启一个终端动态刷新/proc/mdstat配置信息,命令:watch -n 1 cat /proc/mdstat
[root@hc03 ~]# mdadm -f /dev/md5 /dev/sde1 #设定sde1为故障 [root@lvs ~]# mdadm -f /dev/md5 /dev/sde1 mdadm: set /dev/sde1 faulty in /dev/md5 [root@lvs ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md5 : active raid5 sde1[4](F) sdf1[3] sdd1[1] sdc1[0] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] unused devices: <none> 注释: sde1[4](F) 为故障 sdf1[3]并且已经接替故障盘 6)移除故障盘sde1,更新/etc/mdadm.conf配置文件
[root@lvs ~]# mdadm -r /dev/md5 /dev/sde1 mdadm: hot removed /dev/sde1 from /dev/md5 [root@lvs ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md5 : active raid5 sdf1[3] sdd1[1] sdc1[0] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] unused devices: <none> [root@lvs ~]# mdadm -Ds > /etc/mdadm.conf
7)md5磁盘分区格式化,并开机自动挂载至/mnt/RAID5目录
[root@lvs ~]# mkdir -pv /mnt/RAID5 #创建/mnt/RAID5目录 mkdir: 已创建目录 "/mnt/RAID5" [root@lvs ~]# fdisk /dev/md5 #磁盘分区 Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0xc24c42ef. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u'). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-1051136, default 257): Using default value 257 Last cylinder, +cylinders or +size{K,M,G} (257-1051136, default 1051136): Using default value 1051136 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@lvs ~]# mkfs.ext4 /dev/md5p1 #格式化分区 mke2fs 1.41.12 (17-May-2010) 文件系统标签= 操作系统:Linux 块大小=4096 (log=2) 分块大小=4096 (log=2) Stride=128 blocks, Stripe width=256 blocks 262944 inodes, 1050880 blocks 52544 blocks (5.00%) reserved for the super user 第一个数据块=0 Maximum filesystem blocks=1077936128 33 block groups 32768 blocks per group, 32768 fragments per group 7968 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 正在写入inode表: 完成 Creating journal (32768 blocks): 完成 Writing superblocks and filesystem accounting information: 完成 This filesystem will be automatically checked every 39 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. [root@lvs ~]# mount /dev/md5p1 /mnt/RAID5 #挂载 [root@lvs ~]# mount -l #查看挂载 /dev/mapper/vg_lvs-lv_root on / type ext4 (rw) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/sda1 on /boot type ext4 (rw) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) /dev/md5p1 on /mnt/RAID5 type ext4 (rw) #正常挂载 [root@lvs ~]# cat /etc/fstab #添加至开机启动挂载 # # /etc/fstab # Created by anaconda on Mon Nov 2 18:20:45 2015 # # Accessible filesystems, by reference, are maintained under '/dev/disk' # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/mapper/vg_lvs-lv_root / ext4 defaults 1 1 UUID=146bb43b-343e-4f5a-90c6-4056c695bfa1 /boot ext4 defaults 1 2 /dev/mapper/vg_lvs-lv_swap swap swap defaults 0 0 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 /dev/md5p1 /mnt/RAID5 ext4 defaults 0 0
8)操作停止和激活阵列,使用mdadm -S 或 mdadm -As(注意:挂载中的阵列不允许停止,需要卸载挂载)
如:停止上面阵列RAID5
[root@lvs ~]# mdadm -S /dev/md5 mdadm: Cannot get exclusive access to /dev/md5:Perhaps a running process, mounted filesystem or active volume group? [root@lvs ~]# umount /dev/md5p1 /mnt/RAID5 umount: /mnt/RAID5: not mounted [root@lvs ~]# mdadm -S /dev/md5 mdadm: stopped /dev/md5 注:mdadm -Ss 为停止所有阵列
激活停止的RAID5,需要重新挂载
激活raid 5阵列:
[root@lvs ~]# mdadm -As mdadm: /dev/md/5 has been started with 3 drives. mdadm: Found some drive for an array that is already active: /dev/md/5 mdadm: giving up. [root@lvs ~]# mount /dev/md/5p1 /mnt/RAID5 #重新挂载
9)上面模拟一个硬盘故障,如何向阵列中添加一个磁盘,这里以另外一个磁盘/dev/sdg1为例(注意:挂载中的队列不允许,添加磁盘)
[root@lvs ~]# umount /dev/md/5p1 /mnt/RAID5 #卸载挂载 [root@lvs ~]# mdadm -a /dev/md5 /dev/sdg1 mdadm: added /dev/sdg1 [root@lvs ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md5 : active raid5 sdg1[4](S) sdc1[0] sdf1[3] sdd1[1] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] unused devices: <none> 注: sdg1[4](S) #说明sdg1已经是备用盘了 [root@lvs ~]# mdadm -G /dev/md5 -n 4 #添加至md5 [root@lvs ~]# cat /proc/mdstat Personalities : [raid6] [raid5] [raid4] md5 : active raid5 sdg1[4] sdc1[0] sdf1[3] sdd1[1] 4204544 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU] [=======>.............] reshape = 36.0% (757580/2102272) finish=0.2min speed=108225K/sec unused devices: <none> 注: active raid5 sdg1[4] sdc1[0] sdf1[3] sdd1[1] #当前活动的已经有4个盘了 [root@lvs ~]# mdadm -Ds > /etc/mdadm.conf #刷新配置 [root@lvs ~]# mdadm -Ds ARRAY /dev/md/5 metadata=1.2 name=lvs:5 UUID=8a62a5ad:9b1fd7ae:09b5f7e2:c0756e62
4、RAID 10
示例要求:创建RAID 10;开机自动挂载至/mnt/RAID10。
示例环境:准备4块磁盘,直接添加新的盘/dev/sdb、/dev/sdc、/dev/sdd、/dev/sde,每一个硬盘上分出一个5G的分区,分区以此/dev/sdb1、/dev/sdc1、/dev/sdd1、/dev/sde1。
1)创建以上新分区方法,请参照上述相关操作完成,这里不再阐述!
Device Boot Start End Blocks Id System /dev/sdb1 1 652 5237158+ 83 Linux Device Boot Start End Blocks Id System /dev/sdc1 1 652 5237158+ 83 Linux Device Boot Start End Blocks Id System /dev/sdd1 1 652 5237158+ 83 Linux Device Boot Start End Blocks Id System /dev/sde1 1 652 5237158+ 83 Linux
2)使用mdadm命令把sdb/sdc和/sdd/sde硬盘分别sdb1/sdc1和/sdd1/sde1分区分别两个创建为RAID 1
[root@node1 ~]# mdadm -C -v /dev/md00 -l 1 -n 2 /dev/sdb1 /dev/sdc1 mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90 mdadm: size set to 5233024K Continue creating array? Continue creating array? (y/n) y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md00 started. [root@node1 ~]# mdadm -C -v /dev/md01 -l 1 -n 2 /dev/sdd1 /dev/sde1 mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90 mdadm: size set to 5233024K Continue creating array? Continue creating array? (y/n) y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md01 started. 注释:-C创建新磁盘阵列,-l 1 为RAID类型为1,2为硬盘个数
3)把上面创建新/dev/md00和/dev/md01创建为raid 0
[root@node1 ~]# mdadm -C -v /dev/md10 -l 0 -n 2 /dev/md0 /dev/md1 mdadm: chunk size defaults to 512K mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md10 started.
注意:上面创建用新的磁盘/dev/md00、/dev/md01用做raid 1 之后,名称变为/dev/md0和/dev/md1(可用fdisk -l 查看)
4)查看阵列信息,命令mdadm -Ds或/proc/mdstat,并生成配置文件/etc/mdadm.conf
[root@node1 ~]# cat /proc/mdstat Personalities : [raid1] [raid0] md10 : active raid0 md1[1] md0[0] 10457088 blocks super 1.2 512k chunks md1 : active raid1 sde1[1] sdd1[0] 5233024 blocks super 1.2 [2/2] [UU] md0 : active raid1 sdc1[1] sdb1[0] 5233024 blocks super 1.2 [2/2] [UU] unused devices: <none> 注释: #sdc1[1] sdb1[0]和sde1[1] sdd1[0]:为RAID 1 #md1[1] 和 md0[0]:为RAID 0 [root@lvs ~]# mdadm -Ds > /etc/mdadm.conf ARRAY /dev/md0 metadata=1.2 name=node1:00 UUID=dc9713a3:487f0925:0b8dcd0f:d13aec68 ARRAY /dev/md1 metadata=1.2 name=node1:01 UUID=edce41b0:6bb6b1cc:de7233fb:41cc27fb ARRAY /dev/md10 metadata=1.2 name=node1:10 UUID=58b21c66:8a8d9b01:c3900b13:6d2d5292
5)md10创建后进行磁盘分区格式化,并开机自动挂载至目录(这里不再演示,请自行参照上述操作)
5、如何删除RAID 阵列,这里上面的raid 10(md10)为例子
1)取消挂载 umount /dev/md10p1(如/dev/md10有挂载)
2)停止raid 10设备
[root@node1 ~]# mdadm -S /dev/md10 mdadm: stopped /dev/md10
3)删除/etc/mdadm.conf文件
[root@node1 ~]# rm -rf /etc/mdadm.conf
4)清除磁盘/dev/md0和/dev/md1的raid标识
[root@node1 ~]# mdadm --misc --zero-superblock /dev/md0 [root@node1 ~]# mdadm --misc --zero-superblock /dev/md1 注释: --misc #修改MD相关设备 --zero-superblock #擦除设备中MD的超级块
[root@node1 ~]# mdadm -Ds #查看 ARRAY /dev/md0 metadata=1.2 name=node1:00 UUID=dc9713a3:487f0925:0b8dcd0f:d13aec68 ARRAY /dev/md1 metadata=1.2 name=node1:01 UUID=edce41b0:6bb6b1cc:de7233fb:41cc27fb [root@node1 ~]# cat /proc/mdstat #查看当前阵列 Personalities : [raid1] [raid0] md1 : active raid1 sde1[1] sdd1[0] 5233024 blocks super 1.2 [2/2] [UU] md0 : active raid1 sdc1[1] sdb1[0] 5233024 blocks super 1.2 [2/2] [UU] unused devices: <none>
转载于:https://blog.51cto.com/daisywei/1879561