Linux RAID 和 LVM 配置小实验[1]



最终结构：

 

操作步骤：

注：步骤中所有命令均改为粗体黄色，注意的地方用绿色标出

 

磁盘分区：

首先，在机器上挂载 4 块 10GB 的磁盘（这里用 Azure 虚拟机来模拟环境，系统使用 CentOS 6.7）：

[root@DanCentOS67 daniel]# fdisk -l   ……   Disk /dev/sdc: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000   Disk /dev/sdd: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000   Disk /dev/sde: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000   Disk /dev/sdf: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000

 

接着分别将每个磁盘分为2个分区，以 /dev/sdc 为例：

使用 fdisk 工具划出第一个分区 /dev/sdc1，大小约为 5GB（dev/sdc2 同理，Partition number 要改为 2，扇区的起止编号也不同，sdc2扇区编号是 654-1305）：

[root@DanCentOS67 daniel]# fdisk /dev/sdc   WARNING: DOS-compatible mode is deprecated. It's strongly recommended to          switch off the mode (command 'c') and change display units to          sectors (command 'u').   Command (m for help): n Command action    e   extended    p   primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-1305, default 1): 1 Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): 653   Command (m for help): t Selected partition 1 Hex code (type L to list codes): fd Changed system type of partition 1 to fd (Linux raid autodetect)   Command (m for help): w The partition table has been altered!   Calling ioctl() to re-read partition table.   WARNING: Re-reading the partition table failed with error 16: Device or resource busy. The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8) Syncing disks.

 

使用 fdisk-l 查看分区结果：

[root@DanCentOS67 daniel]# fdisk -l   ……   Disk /dev/sdc: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xbd293e5b      Device Boot      Start         End      Blocks   Id  System /dev/sdc1               1         653     5245191   fd  Linux raid autodetect /dev/sdc2             654        1305     5237190   fd  Linux raid autodetect

 

对剩下的3个磁盘（/dev/sdd，/dev/sde，/dev/sdf）进行同样的处理，处理后使用 fdisk -l 查看是否全部完成分区格式化：

[root@DanCentOS67 daniel]# fdisk -l   ……   Disk /dev/sdc: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xbd293e5b      Device Boot      Start         End      Blocks   Id  System /dev/sdc1               1         653     5245191   fd  Linux raid autodetect /dev/sdc2             654        1305     5237190   fd  Linux raid autodetect   Disk /dev/sdd: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x13c704fe      Device Boot      Start         End      Blocks   Id  System /dev/sdd1               1         653     5245191   fd  Linux raid autodetect /dev/sdd2             654        1305     5237190   fd  Linux raid autodetect   Disk /dev/sde: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x3273d383      Device Boot      Start         End      Blocks   Id  System /dev/sde1               1         653     5245191   fd  Linux raid autodetect /dev/sde2             654        1305     5237190   fd  Linux raid autodetect   Disk /dev/sdf: 10.7 GB, 10737418240 bytes 255 heads, 63 sectors/track, 1305 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x6283a8a2      Device Boot      Start         End      Blocks   Id  System /dev/sdf1               1         653     5245191   fd  Linux raid autodetect /dev/sdf2             654        1305     5237190   fd  Linux raid autodetect

到这里，就完成了倒数第二层结构。

 

RAID 5的配置：

这里采用 RAID5，在提高IO性能的同时保证数据安全。

首先加载raid5内核模块：

[root@DanCentOS67 daniel]# modprobe raid5

 

接着将 /dev/sdc1，/dev/sdc2，/dev/sdd1 合并为 /dev/md127：

[root@DanCentOS67 daniel]# mdadm --create /dev/md127 --level=5 --raid-devices=3 /dev/sd[cd]1 /dev/sdc2 mdadm: /dev/sdc1 appears to contain an ext2fs file system        size=5245188K  mtime=Thu Jan  1 00:00:00 1970 mdadm: /dev/sdd1 appears to contain an ext2fs file system        size=5245188K  mtime=Thu Jan  1 00:00:00 1970 mdadm: /dev/sdc2 appears to contain an ext2fs file system        size=5237188K  mtime=Thu Jan  1 00:00:00 1970 Continue creating array? y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md127 started.

 

查看 /dev/md127 的状态：

[root@DanCentOS67 daniel]# mdadm --misc --detail /dev/md127 /dev/md127:         Version : 1.2   Creation Time : Thu Mar  9 12:37:43 2017      Raid Level : raid5      Array Size : 10465280 (9.98 GiB 10.72 GB)   Used Dev Size : 5232640 (4.99 GiB 5.36 GB)    Raid Devices : 3   Total Devices : 3     Persistence : Superblock is persistent       Update Time : Thu Mar  9 12:41:44 2017           State : clean  Active Devices : 3 Working Devices : 3  Failed Devices : 0   Spare Devices : 0            Layout : left-symmetric      Chunk Size : 512K              Name : DanCentOS67:127  (local to host DanCentOS67)            UUID : 1a86a744:b01bd085:ee05717d:8ba86e4a          Events : 18       Number   Major   Minor   RaidDevice State        0       8       33        0      active sync   /dev/sdc1        1       8       49        1      active sync   /dev/sdd1        3       8       34        2      active sync   /dev/sdc2

可以看到实际可用大小变为10GB左右，即 5GB*(3-1) = 10GB，具体原理可以参考RAID5的实现原理。

 

同理，将 /dev/sdd2，/dev/sde1，/dev/sde2 合并为  /dev/md126：

[root@DanCentOS67 daniel]# mdadm --create /dev/md126 --level=5 --raid-devices=3 /dev/sd[de]2 /dev/sde1

 

将 /dev/sdf1 和 /dev/sdf2 合并为 /dev/md125：

[root@DanCentOS67 daniel]# mdadm --create /dev/md125 --level=5 --raid-device=2 /dev/sdf1 /dev/sdf2

到这里RAID 5已经全部做完。

 

LVM配置：

首先，将 /dev/md127，/dev/md126，/dev/md125 创建为 PhysicalVolume(PV)：

[root@DanCentOS67 daniel]# pvcreate /dev/md127 /dev/md126 /dev/md125   Physical volume "/dev/md127" successfully created   Physical volume "/dev/md126" successfully created   Physical volume "/dev/md125" successfully created

 

扫描 Physical Volume 的改动：

[root@DanCentOS67 daniel]# pvscan   PV /dev/md125         lvm2 [4.99 GiB]   PV /dev/md126         lvm2 [9.98 GiB]   PV /dev/md127         lvm2 [9.98 GiB]   Total: 3 [24.95 GiB] / in use: 0 [0   ] / in no VG: 3 [24.95 GiB]

 

查看每个 Volume 的详细信息：

[root@DanCentOS67 daniel]# pvdisplay   "/dev/md125" is a new physical volume of "4.99 GiB"   --- NEW Physical volume ---   PV Name               /dev/md125   VG Name                 PV Size               4.99 GiB   Allocatable           NO   PE Size               0     Total PE              0   Free PE               0   Allocated PE          0   PV UUID               JWf4ZE-FOCZ-mQfh-3cIA-sbcg-19S6-x28jMy      "/dev/md126" is a new physical volume of "9.98 GiB"   --- NEW Physical volume ---   PV Name               /dev/md126   VG Name                 PV Size               9.98 GiB   Allocatable           NO   PE Size               0     Total PE              0   Free PE               0   Allocated PE          0   PV UUID               DVxViy-miUM-6q7r-520o-SP2T-xBbX-xHj793      "/dev/md127" is a new physical volume of "9.98 GiB"   --- NEW Physical volume ---   PV Name               /dev/md127   VG Name                 PV Size               9.98 GiB   Allocatable           NO   PE Size               0     Total PE              0   Free PE               0   Allocated PE          0   PV UUID               URW7wO-pQLV-uXkP-oXEg-jOKc-fber-oenEj7

 

这里我们首先将 /dev/md127 和 /dev/md126 添加到 Volume Group 中：

[root@DanCentOS67 daniel]# vgcreate VolGroup1 /dev/md127 /dev/md126   Volume group "VolGroup1" successfully created

 

查看 Volume Group的使用情况：

[root@DanCentOS67 daniel]# vgdisplay   --- Volume group ---   VG Name               VolGroup1   System ID               Format                lvm2   Metadata Areas        2   Metadata Sequence No  1   VG Access             read/write   VG Status             resizable   MAX LV                0   Cur LV                0   Open LV               0   Max PV                0   Cur PV                2   Act PV                2   VG Size               19.95 GiB   PE Size               4.00 MiB   Total PE              5108   Alloc PE / Size       0 / 0     Free  PE / Size       5108 / 19.95 GiB   VG UUID               dsTlXU-D3Y5-Fqb3-wCOl-LHmj-7Qyf-hXU186

 

Volume Group 创建好后，我们在其上划分新的 Logical Volume：

[root@DanCentOS67 daniel]# lvcreate -l 2500 -n LogicalVol1 VolGroup1   Logical volume "LogicalVol1" created. [root@DanCentOS67 daniel]# lvcreate -l 2608 -n LogicalVol2 VolGroup1   Logical volume "LogicalVol2" created.

前面我们 Volume Group中一共有 5108 个 PE，每个 PE（Physical Extent）的默认大小是 4.00 MiB，一共 19.95 GiB。我们为 LogicalVol1划分了 2500 个 PE，即 10000 MiB(9.77 GiB)，为 LogicalVol2 划分了 2608 个 PE，即 10432MiB(10.19 Gib)。

 

查看 LogicalVolume 的划分情况：

[root@DanCentOS67 daniel]# lvdisplay   --- Logical volume ---   LV Path                /dev/VolGroup1/LogicalVol1   LV Name                LogicalVol1   VG Name                VolGroup1   LV UUID                8w2ZhQ-hTSr-Sg0z-OJRv-h9aq-coZw-zeYCtI   LV Write Access        read/write   LV Creation host, time DanCentOS67, 2017-03-09 13:37:56 +0000   LV Status              available   # open                 0   LV Size                9.77 GiB   Current LE             2500   Segments               1   Allocation             inherit   Read ahead sectors     auto   - currently set to     4096   Block device           253:0      --- Logical volume ---   LV Path                /dev/VolGroup1/LogicalVol2   LV Name                LogicalVol2   VG Name                VolGroup1   LV UUID                JdQsXc-OMwk-A6ZB-e3vJ-rLPv-hNek-3yNVVg   LV Write Access        read/write   LV Creation host, time DanCentOS67, 2017-03-09 13:38:08 +0000   LV Status              available   # open                 0   LV Size                10.19 GiB   Current LE             2608   Segments               2   Allocation             inherit   Read ahead sectors     auto   - currently set to     4096   Block device           253:1

 

fdisk -l 查看设备：

[root@DanCentOS67 daniel]# fdisk -l   ……   Disk /dev/mapper/VolGroup1-LogicalVol1: 10.5 GB, 10485760000 bytes 255 heads, 63 sectors/track, 1274 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 524288 bytes / 1048576 bytes Disk identifier: 0x00000000   Disk /dev/mapper/VolGroup1-LogicalVol2: 10.9 GB, 10938744832 bytes 255 heads, 63 sectors/track, 1329 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 524288 bytes / 1048576 bytes Disk identifier: 0x00000000

 

使用 mkfs.ext4 工具将两个 LogicalVolume 的文件系统格式化为 ext4：

[root@DanCentOS67 daniel]# mkfs.ext4 /dev/mapper/VolGroup1-LogicalVol1 mke2fs 1.41.12 (17-May-2010) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=128 blocks, Stripe width=256 blocks 640848 inodes, 2560000 blocks 128000 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=2621440000 79 block groups 32768 blocks per group, 32768 fragments per group 8112 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632   Writing inode tables: done                            Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done   This filesystem will be automatically checked every 39 mounts or 180 days, whichever comes first.  Use tune2fs -c or -i to override. [root@DanCentOS67 daniel]# mkfs.ext4 /dev/mapper/VolGroup1-LogicalVol2 ……

 

格式化后，将格式化得到的文件分区mount到目录中（如果要开机自动挂载可以修改 /etc/fstab）：

[root@DanCentOS67 daniel]# mkdir /mnt/LV1 [root@DanCentOS67 daniel]# mkdir /mnt/LV2 [root@DanCentOS67 daniel]# mount /dev/mapper/VolGroup1-LogicalVol1 /mnt/LV1 [root@DanCentOS67 daniel]# mount /dev/mapper/VolGroup1-LogicalVol2 /mnt/LV2

到这里，LVM部分已经配置完成。