This guide explains how to set up software RAID1 on an already running RHEL 4 system. The GRUB bootloader will be configured in such a way that the system will still be able to boot if one of the hard drives fails (no matter which one).
[@more@]Introduce my environment:
In my test environment I'm using a RHEL 4 U 5 system with two hard drives, /dev/sda and /dev/sdb which are identical in size. /dev/sdb is currently unused, and /dev/sda has the following partitions:
/dev/sda1: /boot partition, ext3;
/dev/sda2: swap;
/dev/sda3: / partition, ext3
[root@server1 ~]# df –h
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 4.3G 2.4G 1.8G 58% /
/dev/sda1 99M 12M 83M 13% /boot
tmpfs 185M 0 185M 0% /dev/shm
[root@server1 ~]# fdisk -l
Disk /dev/sda: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0007b217
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 77 514080 82 Linux swap / Solaris
/dev/sda3 78 652 4618687+ 83 Linux
Disk /dev/sdb: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000
Disk /dev/sdb doesn't contain a valid partition table
2 Preparing /dev/sdb
2.1 Copy the partition table from /dev/sda to /dev/sdb so that both disks have exactly the same layout:
[root@server1 ~]# sfdisk –d /dev/sda | sfdisk /dev/sdb
2.2 Next we must change the partition type of our three partitions on /dev/sdb to Linux raid autodetect:
[root@server1 ~]# fdisk /dev/sdb
Command (m for help): Partition number (1-4): Hex code (type L to list codes): Changed system type of partition 1 to fd (Linux raid autodetect)Command (m for help): Partition number (1-4): Hex code (type L to list codes): Changed system type of partition 2 to fd (Linux raid autodetect)
Command (m for help): Partition number (1-4): Hex code (type L to list codes): Changed system type of partition 3 to fd (Linux raid autodetect)
Command (m for help):
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@server1 ~]#
2.3 To make sure that there are no remains from previous RAID installations on /dev/sdb
[root@server1 ~]# mdadm --zero-superblock /dev/sdb1
[root@server1 ~]# mdadm --zero-superblock /dev/sdb2
[root@server1 ~]# mdadm --zero-superblock /dev/sdb3
3 Creating Our RAID Arrays
[root@server1 ~]# mdadm --create /dev/md0 --level=1 --raid-disks=2 missing /dev/sdb1
[root@server1 ~]# mdadm --create /dev/md1 --level=1 --raid-disks=2 missing /dev/sdb2
[root@server1 ~]# mdadm --create /dev/md2 --level=1 --raid-disks=2 missing /dev/sdb3
3.1 Because the system is currently running on /dev/sda, therefore we use the placeholder missing in the above three commands.
[root@server1 ~]# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md2 : active raid1 sdb3[1]
4618560 blocks [2/1] [_U]
md1 : active raid1 sdb2[1]
513984 blocks [2/1] [_U]
md0 : active raid1 sdb1[1]
104320 blocks [2/1] [_U]
unused devices:
[root@server1 ~]#
3.2 Next we create filesystems on our RAID arrays (ext3 on /dev/md0 and /dev/md2 and swap on /dev/md1):
[root@server1 ~]# mkfs.ext3 /dev/md0
[root@server1 ~]# mkswap /dev/md1
[root@server1 ~]# mkfs.ext3 /dev/md2
3.3 Next we create /etc/mdadm.conf as follows:
[root@server1 ~]# mdadm --detail --scan > /etc/mdadm.conf
[root@server1 ~]# cat /etc/mdadm.conf
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=2848a3f5:cd1c26b6:e762ed83:696752f9
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=8a004bac:92261691:227767de:4adf6592
ARRAY /dev/md2 level=raid1 num-devices=2 UUID=939f1c71:be9c10fd:d9e5f8c6:a46bcd49
4 Adjusting The System To RAID1
4.1 Now let's mount /dev/md0 and /dev/md2 (we don't need to mount the swap array /dev/md1):
[root@server1 ~]# mkdir /mnt/md0[root@server1 ~]# mkdir /mnt/md2[root@server1 ~]# mount /dev/md0 /mnt/md0[root@server1 ~]# mount /dev/md2 /mnt/md2
4.2 Now we copy the contents of /dev/sda1 and /dev/sda3 to /dev/md0 and /dev/md2 (which are mounted on /mnt/md0 and /mnt/md2):
[root@server1 ~]# cp -dpRx / /mnt/md2[root@server1 ~]# cd /boot[root@server1 ~]# cp -dpRx . /mnt/md0
复制文件建议放在修改完grub.conf之后进行,否则会不同步,后面会有解释
4.3 Next we modify /etc/fstab. Replace LABEL=/boot with /dev/md0, LABEL=SWAP-sda2 with /dev/md1, and LABEL=/ with /dev/md2 so that the file looks as follows:
[root@server1 ~]# cat /etc/fstab
/dev/md2 / ext3 defaults 1 1
/dev/md0 /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
/dev/md1 swap swap defaults 0 0
4.4 Next replace LABEL=/boot with /dev/md0 and LABEL=/ with /dev/md2 in /etc/mtab
[root@server1 ~]# cat /etc/mtab
/dev/md2 / ext3 rw 0 0
proc /proc proc rw 0 0
sysfs /sys sysfs rw 0 0
devpts /dev/pts devpts rw,gid=5,mode=620 0 0
/dev/md0 /boot ext3 rw 0 0
tmpfs /dev/shm tmpfs rw 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw 0 0
sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw 0 0
4.5 Now up to the GRUB boot loader. Open /boot/grub/grub.conf and add fallback=1 right after default=0, This makes that if the first kernel (counting starts with 0, so the first kernel is 0) fails to boot, kernel #2 will be booted.
[root@server1 ~]# vi /boot/grub/grub.conf
[...]
default=0
fallback=1
[...]
4.6 In the same file, go to the bottom where you should find some kernel stanzas. Copy the first of them and paste the stanza before the first existing stanza; replace root=LABEL=/ with root=/dev/md2 and root (hd0,0) with root (hd1,0).The whole file should look something like this:
[root@server1 ~]# cat /boot/grub/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/md2
# initrd /initrd-version.img
#boot=/dev/sda1
default=0
fallback=1
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux AS (2.6.9-42.ELsmp)
root (hd1,0)
kernel /vmlinuz-2.6.9-42.ELsmp ro root=/dev/md2 rhgb quiet
initrd /initrd-2.6.9-42.ELsmp.img
title Red Hat Enterprise Linux AS (2.6.9-42.ELsmp)
root (hd0,0)
kernel /vmlinuz-2.6.9-42.ELsmp ro root=LABEL=/ rhgb quiet
initrd /initrd-2.6.9-42.ELsmp.img
4.7 Next we adjust the ramdisk to the new situation:
[root@server1 ~]# mv /boot/initrd-`uname -r`.img /boot/initrd-`uname -r`.img_orig
[root@server1 ~]# mkinitrd /boot/initrd-`uname -r`.img `uname -r`
--这一步骤很重要,如果没有做的话用第二块磁盘启动时会提示“could not find filesystem '/dev/root'的错误消息5 Preparing GRUB
5.1 Install the GRUB bootloader on the second hard drive /dev/sdb:
[root@server1 ~]# grub
grub> root (hd0,0)
Filesystem type is ext2fs, partition type 0x83
grub> setup (hd0)
Checking if "/boot/grub/stage1" exists... no
Checking if "/grub/stage1" exists... yes
Checking if "/grub/stage2" exists... yes
Checking if "/grub/e2fs_stage1_5" exists... yes
Running "embed /grub/e2fs_stage1_5 (hd0)"... 16 sectors are embedded.
succeeded
Running "install /grub/stage1 (hd0) (hd0)1+16 p (hd0,0)/grub/stage2 /grub/grub.conf"... succeeded
Done.
grub> root (hd1,0)
Filesystem type is ext2fs, partition type 0xfd
grub> setup (hd1)
Checking if "/boot/grub/stage1" exists... no
Checking if "/grub/stage1" exists... yes
Checking if "/grub/stage2" exists... yes
Checking if "/grub/e2fs_stage1_5" exists... yes
Running "embed /grub/e2fs_stage1_5 (hd1)"... 16 sectors are embedded.
succeeded
Running "install /grub/stage1 (hd1) (hd1)1+16 p (hd1,0)/grub/stage2 /grub/grub.conf"... succeeded
Done.
grub> quit
[root@server1 ~]# reboot
Now, back on the normal shell, reboot the system and hope that it boots ok from our RAID arrays:
这里我在服务器上测试时出现了问题
grub.conf的设置是用root=/dev/md2来启动系统,启动后发现mount的磁盘还是/dev/sda1 /dev/sda2...,并不是/dev/md0 /dev/md1.
原因是我们在修改/etc/fstab /etc/mtab的之前是把之前的/etc/fstab /etc/mtab文件整个复制到/sdb磁盘的,所以在修改这些文件的时候并没有更新到sdb的磁盘。
在用sdb来启动的时候自然还是用的旧的/etc/fstab /etc/mtab来启动的,就会挂载/sda1 /sda2之类的磁盘,而不是/dev/md0 /dev/md1了。
6 Preparing /dev/sda
6.1 If all goes well, we should now find /dev/md0 and /dev/md2 in the output of df -h
root@server1 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md2 4.4G 2.4G 1.8G 58% /
/dev/md0 99M 15M 80M 16% /boot
tmpfs 185M 0 185M 0% /dev/shm
6.2 Now we must change the partition types of our three partitions on /dev/sda to Linux raid autodetect as well:
[root@server1 ~]# fdisk /dev/sda
Command (m for help): Partition number (1-4): Hex code (type L to list codes): Changed system type of partition 1 to fd (Linux raid autodetect)
Command (m for help): Partition number (1-4): Hex code (type L to list codes): Changed system type of partition 2 to fd (Linux raid autodetect)
Command (m for help): Partition number (1-4): Hex code (type L to list codes): Changed system type of partition 3 to fd (Linux raid autodetect)
Command (m for help): The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
6.3 Now we can add /dev/sda1, /dev/sda2, and /dev/sda3 to the respective RAID arrays:
[root@server1 ~]# mdadm --add /dev/md0 /dev/sda1
[root@server1 ~]# mdadm --add /dev/md1 /dev/sda2
[root@server1 ~]# mdadm --add /dev/md2 /dev/sda3
[root@server1 ~]# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid1 sda1[0] sdb1[1]
104320 blocks [2/2] [UU]
md1 : active raid1 sda2[0] sdb2[1]
513984 blocks [2/2] [UU]
md2 : active raid1 sda3[2] sdb3[1]
4618560 blocks [2/1] [_U]
[=====>...............] recovery = 29.9% (1384256/4618560) finish=2.3min speed=22626K/sec
unused devices:
6.4 Then adjust /etc/mdadm.conf to the new situation:
[root@server1 ~]# mdadm --detail --scan > /etc/mdadm.conf
7 Preparing GRUB again
7.1 Now we must modify /boot/grub/grub.conf again. Right now it is configured to boot from /dev/sdb (hd1,0), we still want the system to be able to boot from /dev/sdb in case /dev/sdb fails. Therefore we copy the first kernel stanza (which contains hd1), paste it below and replace hd1 with hd0. so that it looks as follows:
[root@server1 ~]# cat /boot/grub/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/md2
# initrd /initrd-version.img
#boot=/dev/sda1
default=0
fallback=1
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux AS (2.6.9-42.ELsmp)
root (hd1,0)
kernel /vmlinuz-2.6.9-42.ELsmp ro root=/dev/md2 rhgb quiet
initrd /initrd-2.6.9-42.ELsmp.img
title Red Hat Enterprise Linux AS (2.6.9-42.ELsmp)
root (hd0,0)
kernel /vmlinuz-2.6.9-42.ELsmp ro root=/dev/md2 rhgb quiet
initrd /initrd-2.6.9-42.ELsmp.img
7.2 Afterwards, update the ramdisk:
[root@server1 ~]#mv /boot/initrd-`uname -r`.img /boot/initrd-`uname -r`.img_orig2
[root@server1 ~]#mkinitrd /boot/initrd-`uname -r`.img `uname -r`
7.3 ... and reboot the system:
[root@server1 ~]# reboot
That's it – now I have successfully set up software RAID1 on running RHEL4 system!
8 Testing
8.1 Now let's simulate a hard drive failure. It doesn't matter if you select /dev/sda or /dev/sdb here. In this example I assume that /dev/sdb has failed.
To simulate the hard drive failure, you can either shut down the system and remove /dev/sdb from the system, or you (soft-)remove it like this:
[root@server1 ~]# mdadm --manage /dev/md0 --fail /dev/sdb1
[root@server1 ~]# mdadm --manage /dev/md1 --fail /dev/sdb2
[root@server1 ~]# mdadm --manage /dev/md2 --fail /dev/sdb3
[root@server1 ~]# mdadm --manage /dev/md0 --remove /dev/sdb1
[root@server1 ~]# mdadm --manage /dev/md1 --remove /dev/sdb2
[root@server1 ~]# mdadm --manage /dev/md2 --remove /dev/sdb3
After reboot system and put in a new /dev/sdb drive, file /proc/mdstat may be like below:
[root@server1 ~]# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid1 sda1[0]
104320 blocks [2/1] [U_]
md1 : active raid1 sda2[0]
513984 blocks [2/1] [U_]
md2 : active raid1 sda3[0]
4618560 blocks [2/1] [U_]
unused devices:
[root@server1 ~]#
Now we copy the partition table of /dev/sda to /dev/sdb:
[root@server1 ~]# sfdisk -d /dev/sda | sfdisk /dev/sdb
(If you get an error, you can try the --force option: sfdisk -d /dev/sda | sfdisk --force /dev/sdb)
Afterwards we remove any remains of a previous RAID array from /dev/sdb...
[root@server1 ~]# mdadm --zero-superblock /dev/sdb1
[root@server1 ~]# mdadm --zero-superblock /dev/sdb2
[root@server1 ~]# mdadm --zero-superblock /dev/sdb3
... and add /dev/sdb to the RAID array:
[root@server1 ~]# mdadm -a /dev/md0 /dev/sdb1
[root@server1 ~]# mdadm -a /dev/md1 /dev/sdb2
[root@server1 ~]# mdadm -a /dev/md2 /dev/sdb3
Wait until the synchronization has finished:
[root@server1 ~]# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid1 sdb1[1] sda1[0]
104320 blocks [2/2] [UU]
md1 : active raid1 sdb2[1] sda2[0]
513984 blocks [2/2] [UU]
md2 : active raid1 sdb3[1] sda3[0]
4618560 blocks [2/2] [UU]
unused devices:
Then run
[root@server1 ~]# Grub
and install the bootloader on both hard disks:
grub>root (hd0,0)
grub>setup (hd0)
grub>root (hd1,0)
grub>setup (hd1)
grub>quit
That's it. You've just replaced a failed hard drive in your RAID1 array.
来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/271283/viewspace-1029276/,如需转载,请注明出处,否则将追究法律责任。
转载于:http://blog.itpub.net/271283/viewspace-1029276/