DRBD是一种块设备,它使用主备模式,可以被用于高可用(HA)之中。类似于一个网络RAID-1功能,当将数据写入本地文件系统的同时还将会被发送到网络中另一台主机上。以相同的形式记录在一个文件系统中。本地(主节点)与远程主机(备节点)的数据可以保证实时同步。当本地系统出现故障时,远程主机上还会保留有一份相同的数据,可以继续使用。在高可用(HA)中使用DRBD功能,可以代替使用一个共享盘阵。因为数据同时存在于本地主机和远程主机上。切换时,远程主机只要使用它上面的那份备份数据,就可以继续进行服务了。
环境:两台主机,一台server1充当本地主机,另一台server2充当远程主机,并且两台主机都要关闭selinux和iptables,同时做好解析。
server1.example.com: 172.25.30.1
server2.example.com: 172.25.30.2
在server1和server2上分别安装下面的安装包:
yum install -y heartbeat-3.0.4-2.el6.x86_64.rpm heartbeat-libs-3.0.4-2.el6.x86_64.rpm
heartbeat-devel-3.0.4-2.el6.x86_64.rpm ldirectord-3.9.5-3.1.x86_64.rpm
[root@server1 ~]# cd /etc/ha.d
[root@server1 ha.d]# ls
harc rc.d README.config resource.d shellfuncs
[root@server1 ha.d]# rpm -q heartbeat -d
[root@server1 ha.d]# cp /usr/share/doc/heartbeat-3.0.4/authkeys .
[root@server1 ha.d]# cp /usr/share/doc/heartbeat-3.0.4/ha.cf .
[root@server1 ha.d]# cp /usr/share/doc/heartbeat-3.0.4/haresources .
[root@server1 ha.d]# ls
authkeys harc rc.d resource.d
ha.cf haresources README.config shellfuncs
[root@server1 ha.d]# vim ha.cf #在这里边添加修改如下参数
logfile /var/log/ha-log
keepalive 2
deadtime 30
Warntime 10
initdead 60
udpport 695
bcast eth0
node server1.example.com ##后面为主机名,写在前面的为主设备,这里server1 为主设备
node server2.example.com
ping 172.25.30.250
respawn hacluster /usr/lib64/heartbeat/ipfail
apiauth ipfail gid=haclient uid=hacluster
[root@server1 ha.d]# vim authkeys
auth 1
1 crc
[root@server1 ha.d]# ll authkeys
-rw-r--r-- 1 root root 643 7月 18 16:34 authkeys
[root@server1 ha.d]# chmod 600 authkeys
[root@server1 ha.d]# vim haresources
server1.example.com IPaddr::172.25.30.100/24/eth0 httpd ##这里的172.25.30.100为虚拟IP,即VIP。
在server1上将这三个配置文件传给server2,并在两端都开启heartbeat,并安装httpd.
[root@server1 ha.d]# scp ha.cf authkeys haresources 172.25.30.2:/etc/ha.d/
/etc/init.d/heartbeat start
chkconfig heartbeat off
tail -f /var/log/ha-log ##查看日志
yum install httpd -y
此时,关闭server1上的heartbeat
/etc/init.d/heartbeat stop
就可以在浏览器上访问到VIP:172.25.30.100的默认测试页。
再将heartbeat关闭就可以VIP被主设备抢占。
当分别在server1 server2的/var/www/html的默认测试界面分别写上server1 和server2时 ,用浏览器访问172.25.30.100,就会很明显的看到默认测试页的内容为server1,即VIP被server1抢占。
当关闭httpd时对其并不影响。只有主设备上的heartbeat被关闭或者坏掉后,则VIP开始重新选取主设备,即此时的server2为主设备。
当server1上的heartbeat再次被开启或者被恢复时,由于在配置文件中定义的是server1为主节点,所以此时,VIP又重新被server1 抢占回来。
DRBD的配置:
对DRBD进行编译并解决依赖性:
[root@server1 ~]# yum install -y rpm-build
[root@server1 mnt]# tar zxf drbd-8.4.2.tar.gz
[root@server1 mnt]# ls
drbd-8.4.2 heartbeat-devel-3.0.4-2.el6.x86_64.rpm
drbd-8.4.2.tar.gz heartbeat-libs-3.0.4-2.el6.x86_64.rpm
heartbeat-3.0.4-2.el6.x86_64.rpm ldirectord-3.9.5-3.1.x86_64.rpm
[root@server1 mnt]# cd drbd-8.4.2
[root@server1 drbd-8.4.2]# ./configure --enable-spec --with-km
[root@server1 drbd-8.4.2]# yum install -y gcc
[root@server1 drbd-8.4.2]# yum install flex -y
[root@server1 drbd-8.4.2]# ./configure --enable-spec --with-km
[root@server1 drbd-8.4.2]# rpmbuild -bb drbd.spec
error: File /root/rpmbuild/SOURCES/drbd-8.4.2.tar.gz: No such file or directory
[root@server1 drbd-8.4.2]# cp /mnt/drbd-8.4.2.tar.gz ~/rpmbuild/SOURCES/
[root@server1 drbd-8.4.2]# rpmbuild -bb drbd.spec
[root@server1 ~]# cd /root/rpmbuild/RPMS/x86_64/
[root@server1 x86_64]# rpm -ivh *
[root@server1 x86_64]# scp * 172.25.30.2:
在server2上也同样安转所有的安装包。
[root@server2 ~]# rpm -ivh *
分别在server1和server2上分别添加一个4G的硬盘。
[root@server1 drbd.d]# vim example.res
[root@server1 drbd.d]# scp example.res 172.25.30.2:/etc/drbd.d/
两边同时初始化并载入数据:
[root@server1 drbd.d]# drbdadm create-md example
[root@server1 ~]# cd drbd-8.4.2
[root@server1 drbd-8.4.2]# ls
autogen.sh COPYING drbd-km.spec.in preamble-rhel5
benchmark documentation drbd.spec preamble-sles10
ChangeLog drbd drbd.spec.in preamble-sles11
config.log drbd_config.h filelist-redhat README
config.status drbd-kernel.spec filelist-suse rpm-macro-fixes
configure drbd-kernel.spec.in Makefile.in scripts
configure.ac drbd-km.spec preamble user
[root@server1 drbd-8.4.2]# ll drbd.spec
-rw-r--r-- 1 root root 11802 7月 24 15:05 drbd.spec
[root@server1 drbd-8.4.2]# rpmbuild -bb drbd-km.spec
error: Failed build dependencies:
kernel-devel is needed by drbd-km-8.4.2-2.el6.x86_64
[root@server1 drbd-8.4.2]# yum install kernel-devel -y
[root@server1 drbd-8.4.2]# rpmbuild -bb drbd-km.spec
[root@server1 x86_64]# rpm -ivh drbd-km-2.6.32_431.el6.x86_64-8.4.2-2.el6.x86_64.rpm
[root@server1 x86_64]# scp drbd-km-2.6.32_431.el6.x86_64-8.4.2-2.el6.x86_64.rpm 172.25.30.2:
[root@server2 ~]# rpm -ivh drbd-km-2.6.32_431.el6.x86_64-8.4.2-2.el6.x86_64.rpm
Preparing... ########################################### [100%]
1:drbd-km-2.6.32_431.el6.########################################### [100%]
两边同时打开drbd
[root@server1 x86_64]# /etc/init.d/drbd start
[root@server2 ~]# /etc/init.d/drbd start
等待两边同步完成:
[root@server1 x86_64]# drbdadm primary example --force
[root@server1 x86_64]# cat /proc/drbd
version: 8.4.2 (api:1/proto:86-101)
GIT-hash: 7ad5f850d711223713d6dcadc3dd48860321070c build by root@server1.example.com, 2016-07-24 15:11:11
1: cs:Connected ro:Primary/Secondary ds:UpToDate/Diskless C r-----
ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:4194140
[root@server2 ~]# cat /proc/drbd
version: 8.4.2 (api:1/proto:86-101)
GIT-hash: 7ad5f850d711223713d6dcadc3dd48860321070c build by root@server1.example.com, 2016-07-24 15:11:11
1: cs:Connected ro:Secondary/Primary ds:Diskless/UpToDate C r-----
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
将/dev/drbd1格式化为ext4形式
[root@server1 x86_64]# ll /dev/vda
brw-rw---- 1 root disk 252, 0 7月 24 15:19 /dev/vda
[root@server1 x86_64]# ll /dev/drbd1
brw-rw---- 1 root disk 147, 1 7月 24 15:21 /dev/drbd1
[root@server1 x86_64]# mkfs.ext4 /dev/drbd1
将/dev/drbd1挂载到/mnt下,测试其是否何以相互同步数据:
[root@server1 x86_64]# mount /dev/drbd1 /mnt/
[root@server1 mnt]# ls
lost+found
[root@server1 mnt]# cp /etc/passwd .
[root@server1 mnt]# ls
lost+found passwd
[root@server1 mnt]# cd
[root@server1 ~]# umount /mnt
[root@server1 ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup-lv_root 6926264 1321388 5253032 21% /
tmpfs 510200 0 510200 0% /dev/shm
/dev/sda1 495844 33450 436794 8% /boot
[root@server2 ~]# mount /dev/drbd1 /mnt
mount: you must specify the filesystem type
[root@server1 ~]# cat /proc/drbd
version: 8.4.2 (api:1/proto:86-101)
GIT-hash: 7ad5f850d711223713d6dcadc3dd48860321070c build by root@server1.example.com, 2016-07-24 15:11:11
1: cs:Connected ro:Primary/Secondary ds:UpToDate/Diskless C r-----
ns:0 nr:0 dw:132472 dr:1389 al:42 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:4194140
将server1.example.com变为secondary example
[root@server1 ~]# drbdadm secondary example
[root@server1 ~]# cat /proc/drbd
version: 8.4.2 (api:1/proto:86-101)
GIT-hash: 7ad5f850d711223713d6dcadc3dd48860321070c build by root@server1.example.com, 2016-07-24 15:11:11
1: cs:Connected ro:Secondary/Secondary ds:UpToDate/Diskless C r-----
ns:0 nr:0 dw:132472 dr:1389 al:42 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:4194140
[root@server2 ~]# drbdadm primary example
[root@server2 ~]# cat /proc/drbd
version: 8.4.2 (api:1/proto:86-101)
GIT-hash: 7ad5f850d711223713d6dcadc3dd48860321070c build by root@server1.example.com, 2016-07-24 15:11:11
1: cs:Connected ro:Primary/Secondary ds:Diskless/UpToDate C r-----
ns:0 nr:664 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
[root@server2 ~]# mount /dev/drbd1 /mnt
[root@server2 ~]# cd /mnt
[root@server2 mnt]# ls
lost+found passwd
[root@server2 mnt]# cp /etc/fstab .
[root@server2 mnt]# cd
[root@server2 ~]# umount /mnt
[root@server2 ~]# drbdadm secondary example
[root@server1 ~]# drbdadm primary example
[root@server1 ~]# cd /mnt
[root@server1 mnt]# ls
drbd-8.4.2 heartbeat-devel-3.0.4-2.el6.x86_64.rpm
drbd-8.4.2.tar.gz heartbeat-libs-3.0.4-2.el6.x86_64.rpm
heartbeat-3.0.4-2.el6.x86_64.rpm ldirectord-3.9.5-3.1.x86_64.rpm
[root@server1 mnt]# mount /dev/drbd1 /mnt
[root@server1 mnt]# cd
[root@server1 ~]# cd /mnt
[root@server1 mnt]# ls
fstab lost+found passwd
[root@server1 mnt]# cd
[root@server1 ~]# umount /mnt
[root@server1 ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup-lv_root 6926264 1321388 5253032 21% /
tmpfs 510200 0 510200 0% /dev/shm
/dev/sda1 495844 33450 436794 8% /boot
[root@server1 ~]# mount /dev/vda /mnt
mount: unknown filesystem type 'drbd'
[root@server1 ~]# mount /dev/vdb /mnt
mount: you must specify the filesystem type
[root@server1 ~]# ll /dev/drbd1
brw-rw---- 1 root disk 147, 1 7月 24 15:37 /dev/drbd1
[root@server1 ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup-lv_root 6926264 1321388 5253032 21% /
tmpfs 510200 0 510200 0% /dev/shm
/dev/sda1 495844 33450 436794 8% /boot
[root@server1 ~]# drbd-overview
1:example/0 Connected Primary/Secondary UpToDate/Diskless C r-----
[root@server1 ~]# drbdadm down example
[root@server1 ~]# drbd-overview
1:example/0 Unconfigured . . . .
[root@server1 ~]# drbdadm up example
[root@server1 ~]# drbd-overview
1:example/0 Connected Secondary/Secondary UpToDate/Diskless C r-----
[root@server1 ~]# drbdadm primary example
[root@server1 ~]# cd /mnt
[root@server1 ~]# mount /dev/drbd1 /mnt
[root@server1 ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup-lv_root 6926264 1321388 5253032 21% /
tmpfs 510200 0 510200 0% /dev/shm
/dev/sda1 495844 33450 436794 8% /boot
/dev/drbd1 4128284 73728 3844852 2% /mnt
破坏内核:
[root@server1 ~]# echo c > /proc/sysrq-trigger
不连网:
[root@server1 ~]# cd /etc/ha.d
[root@server1 ha.d]# ls
authkeys harc rc.d resource.d
ha.cf haresources README.config shellfuncs
[root@server1 ha.d]# vim haresources
[root@server1 ha.d]# scp haresources 172.25.30.2:/etc/ha.d
[root@server1 ~]# /etc/init.d/heartbeat start
Starting High-Availability services: INFO: Resource is stopped
Done.
[root@server2 ~]# /etc/init.d/heartbeat stop
Stopping High-Availability services: Done.
[root@server2 ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup-lv_root 6926264 1124200 5450220 18% /
tmpfs 510200 0 510200 0% /dev/shm
/dev/sda1 495844 33450 436794 8% /boot
[root@server2 ~]# drbd-overview
1:example/0 Connected Secondary/Primary Diskless/UpToDate C r-----
[root@server2 ~]# /etc/init.d/drbd start
[root@server1 ~]#cd
[root@server1 ~]# umount /mnt
[root@server1 ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup-lv_root 6926264 1319404 5255016 21% /
tmpfs 510200 0 510200 0% /dev/shm
/dev/sda1 495844 33450 436794 8% /boot
/dev/drbd1 4128284 73728 3844852 2% /var/www/html
[root@server1 ~]# ip addr show
[root@server1 ~]# cd /var/www/html
[root@server1 html]# ls
fstab lost+found passwd
[root@server1 html]# rm -fr *
[root@server1 html]# vim index.html
[root@server1 html]# curl localhost
www.westos.org
[root@server1 html]# cd
[root@server1 ~]# /etc/init.d/heartbeat stop
Stopping High-Availability services: Done.
[root@server2 ha.d]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup-lv_root 6926264 1124284 5450136 18% /
tmpfs 510200 0 510200 0% /dev/shm
/dev/sda1 495844 33450 436794 8% /boot
/dev/drbd1 4128284 73708 3844872 2% /var/www/html
[root@server2 ha.d]# ip addr show
[root@server2 ha.d]# curl 172.25.30.100
www.westos.org