Mysql HA-Install DRBD+HeartBeat+Mysql On Redhat 6.3

配置信息：

Primary：
Name:zbdba1
OS:redhat 6.3
IP:192.168.56.220
drbd:8.4.0
heartbeat:3.0.4

Standby:
Name:zbdba2
OS:Redhat 6.3
IP:192.168.56.221
drbd:8.4.0
heartbeat:3.0.4

主要分为如下步骤：
1、安装DRBD
2、安装Mysql
3、测试DRBD
4、安装Heartbeat
5、测试Heartbeat

1、安装DRBD
下载drbd：
wget http://oss.linbit.com/drbd/8.4/drbd-8.4.0.tar.gz


在zbdba1、zbdba2：
添加磁盘

Disk /dev/sdb: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

安装依赖包：

yum install gcc flex rpm-build kernel-devel docbook-style-xsl -y

建立rpm目录：

rpmbuild ~

解压并生成rpm包：

tar xvf drbd-8.4.0.tar.gz  尝试了8.4.3，版本太新 linux内核不兼容
./configure
make rpm 
make km-rpm
此时已经生成drbd的rpm包

进入安装：

cd /root/rpmbuild/RPMS/x86_64
yum install * -y

查看drbd模块：

modprobe drbd
lsmod |grep drbd

配置DRBD：

[root@zbdba1 etc]# cat drbd.conf
include "drbd.d/drbd.conf.example";

[root@zbdba1 etc]# cat drbd.d/drbd.conf.example
resource zbdba {
        options {
                on-no-data-accessible suspend-io;
        }

        net {
                cram-hmac-alg "sha1";
                shared-secret "secret_string";
        }

        # The disk section is possible on resource level and in each
        # volume section
        disk {
                # If you have a resonable RAID controller
                # with non volatile write cache (BBWC, flash)
                disk-flushes no;
                disk-barrier no;
                md-flushes no;
        }

        # volume sections on resource level, are inherited to all node
        # sections. Place it here if the backing devices have the same
        # device names on all your nodes.
#       volume 1 {
#               device minor 1;
#               disk /dev/sdb;
#               meta-disk internal;
#
#               disk {
#                       resync-after example/0;
#               }
#       }

        on zbdba1 {
                address 192.168.56.220:7780;

                volume 0 {
                       device minor 0;
                       disk /dev/sdb;
                       meta-disk internal;
                }
        }
        on zbdba2 {
                address 192.168.56.221:7780;

                volume 0 {
                       device minor 0;
                       disk /dev/sdb;
                       meta-disk internal;
                }
        }
}


drbdadm create-md zbdba

[root@zbdba1 etc]# service drbd start
[root@zbdba2 etc]# service drbd start

在zbdba1：

drbdsetup /dev/drbd0  primary --force
监控传输速率：
[root@Dbrd1 etc]# watch cat /proc/drbd
Every 2.0s: cat /proc/drbd                                                                                                                     Mon Jan 19 07:27:45 2015

version: 8.4.0 (api:1/proto:86-100)
GIT-hash: 28753f559ab51b549d16bcf487fe625d5919c49c build by root@Dbrd1, 2015-01-19 03:52:17
0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
    ns:6976912 nr:0 dw:639076 dr:6339457 al:169 bm:400 lo:0 pe:4 ua:1 ap:0 ep:1 wo:d oos:1820288       
        [==============>.....] sync'ed: 78.4% (1776/8188)M
        finish: 0:00:41 speed: 43,688 (29,716) K/sec
1: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r-----
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

其中：ro是角色信息：Primary/Secondary（代表这个是主节点）
                    Secondary/Primary（代表这个是副节点）

      ds是磁盘状态：UpToDate/Inconsistent（正在同步，数据还没有一致）
                    UpToDate/UpToDate    （同步完成，数据一致）
      ns是网络传输的数据包：以K为字节
    
      dw是磁盘写操作
      dr是磁盘读操作


挂载mysql目录：
mkdir /mysql
mkfs.ext4 /dev/drbd0
mount /dev/drbd0 /mysql

查看DRBD状态：

[root@zbdba1 ~]# service drbd status
drbd driver loaded OK; device status:
version: 8.4.0 (api:1/proto:86-100)
GIT-hash: 28753f559ab51b549d16bcf487fe625d5919c49c build by root@Dbrd1, 2015-01-19 03:52:17
m:res    cs         ro                 ds                 p  mounted  fstype
0:zbdba  Connected  Primary/Secondary  UpToDate/UpToDate  C  /mysql   ext4

[root@zbdba2 ~]# service drbd status
drbd driver loaded OK; device status:
version: 8.4.0 (api:1/proto:86-100)
GIT-hash: 28753f559ab51b549d16bcf487fe625d5919c49c build by root@Dbrd1, 2015-01-19 03:52:17
m:res    cs         ro                 ds                 p  mounted  fstype
0:zbdba  Connected  Secondary/Primary  UpToDate/UpToDate  C

2、安装Mysql：

tar -zxvf mysql-5.6.12-linux-glibc2.5-x86_64
mv mysql-5.6.12-linux-glibc2.5-x86_64 /mysql
添加用户
groupadd mysql
useradd mysql -g mysql
chown -R mysql.mysql mysql/


cp support-files/my-default.cnf /etc/my.cnf

在my.cnf中制定数据库基本目录：
[mysqld]
basedir =/mysql
datadir = /mysql/data

初始化数据库：
scripts/mysql_install_db --user=mysql


cp support-files/mysql.server /etc/init.d/mysql
chkconfig mysql on
service mysql start


修改环境变量：
vi /root/.bash_profile
PATH=$PATH:$HOME/bin:/mysql/bin

3、测试DRBD：

在zbdba1:
service mysql stop

umount /dev/drbd0

drbdadm secondary zbdba

scp /etc/my.cnf zbdba2:`pwd`
在zbdba2：
drbdadm primary zbdba
mount /dev/drbd0 /mysql

cp support-files/mysql.server /etc/init.d/mysql
chkconfig mysql on

启动mysql：
service mysql start
测试成功

4、安装Heartbeat:

在zbdba1、zbdba2：
cluster-glue-1.0.5-6.el6.x86_64
cluster-glue-libs-1.0.5-6.el6.x86_64
heartbeat-3.0.4-1.el6.x86_64
heartbeat-libs-3.0.4-1.el6.x86_64
perl-TimeDate-1.16-11.1.el6.noarch
resource-agents-3.9.2-21.el6.x86_6

yum install * -y

cp /usr/share/doc/heartbeat-3.0.4/ha.cf /etc/ha.d/ha.cf
cp /usr/share/doc/heartbeat-3.0.4/authkeys /etc/ha.d/authkeys
cp /usr/share/doc/heartbeat-3.0.4/haresources /etc/ha.d/haresources

配置文件内容如下：

[root@zbdba1 ha.d]# cat ha.cf
debugfile /var/log/ha-debug
logfile /var/log/ha-log
logfacility     local0
keepalive 2
deadtime 30
warntime 10
bcast   eth0            # Linux
#bcast  eth1 eth2       # Linux
#bcast  le0             # Solaris
#bcast  le1 le2         # Solaris
auto_failback on
node    zbdba1
node    zbdba2

[root@zbdba1 ha.d]# cat authkeys
auth 1
1 crc

[root@zbdba1 ha.d]# cat haresources
zbdba1 192.168.56.225/24/eth0 drbddisk::zbdba Filesystem::/dev/drbd0::/mysql::ext4 mysql

service heartbeat start

查看vip是否启动：
eth0:0    Link encap:Ethernet  HWaddr 08:00:27:EB:CE:6F 
          inet addr:192.168.56.225  Bcast:192.168.56.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1



tail -100f  /var/log/ha-log

配置成功

5、测试Heartbeat：

1、直接关闭hearbeat

[root@zbdba1 ha.d]# service heartbeat stop

监控zbdba1上ha-log日志：
Jan 20 20:43:56 zbdba1 heartbeat: [1517]: info: Heartbeat shutdown in progress. (1517)
Jan 20 20:43:56 zbdba1 heartbeat: [2672]: info: Giving up all HA resources.
ResourceManager(default)[2685]: 2015/01/20_20:43:56 info: Releasing resource group: zbdba1 192.168.56.225/24/eth0 drbddisk::zbdba Filesystem::/dev/drbd0::/mysql::ext4 mysql
ResourceManager(default)[2685]: 2015/01/20_20:43:56 info: Running /etc/init.d/mysql  stop
ResourceManager(default)[2685]: 2015/01/20_20:43:58 info: Running /etc/ha.d/resource.d/Filesystem /dev/drbd0 /mysql ext4 stop
Filesystem(Filesystem_/dev/drbd0)[2772]:        2015/01/20_20:43:58 INFO: Running stop for /dev/drbd0 on /mysql
Filesystem(Filesystem_/dev/drbd0)[2772]:        2015/01/20_20:43:58 INFO: Trying to unmount /mysql
Filesystem(Filesystem_/dev/drbd0)[2772]:        2015/01/20_20:43:58 INFO: unmounted /mysql successfully
/usr/lib/ocf/resource.d//heartbeat/Filesystem(Filesystem_/dev/drbd0)[2764]:     2015/01/20_20:43:58 INFO:  Success
ResourceManager(default)[2685]: 2015/01/20_20:43:58 info: Running /etc/ha.d/resource.d/drbddisk zbdba stop
ResourceManager(default)[2685]: 2015/01/20_20:43:58 info: Running /etc/ha.d/resource.d/IPaddr 192.168.56.225/24/eth0 stop
IPaddr(IPaddr_192.168.56.225)[2929]:    2015/01/20_20:43:58 INFO: ifconfig eth0:0 down
/usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.56.225)[2903]: 2015/01/20_20:43:58 INFO:  Success
Jan 20 20:43:58 zbdba1 heartbeat: [2672]: info: All HA resources relinquished.
Jan 20 20:43:59 zbdba1 heartbeat: [1517]: WARN: 1 lost packet(s) for [zbdba2] [835:837]
Jan 20 20:43:59 zbdba1 heartbeat: [1517]: info: No pkts missing from zbdba2!
Jan 20 20:44:00 zbdba1 heartbeat: [1517]: info: killing HBFIFO process 1526 with signal 15
Jan 20 20:44:00 zbdba1 heartbeat: [1517]: info: killing HBWRITE process 1527 with signal 15
Jan 20 20:44:00 zbdba1 heartbeat: [1517]: info: killing HBREAD process 1528 with signal 15
Jan 20 20:44:00 zbdba1 heartbeat: [1517]: info: Core process 1526 exited. 3 remaining
Jan 20 20:44:00 zbdba1 heartbeat: [1517]: info: Core process 1528 exited. 2 remaining
Jan 20 20:44:00 zbdba1 heartbeat: [1517]: info: Core process 1527 exited. 1 remaining
Jan 20 20:44:00 zbdba1 heartbeat: [1517]: info: zbdba1 Heartbeat shutdown complete.

监控zbdba2上的ha-log:
Jan 20 20:43:58 zbdba2 heartbeat: [1573]: info: Received shutdown notice from 'zbdba1'.
Jan 20 20:43:58 zbdba2 heartbeat: [1573]: info: Resources being acquired from zbdba1.
Jan 20 20:43:58 zbdba2 heartbeat: [1979]: info: acquire local HA resources (standby).
Jan 20 20:43:58 zbdba2 heartbeat: [1980]: info: No local resources [/usr/share/heartbeat/ResourceManager listkeys zbdba2] to acquire.
Jan 20 20:43:58 zbdba2 heartbeat: [1979]: info: local HA resource acquisition completed (standby).
Jan 20 20:43:58 zbdba2 heartbeat: [1573]: info: Standby resource acquisition done [all].
harc(default)[2005]:    2015/01/20_20:43:58 info: Running /etc/ha.d//rc.d/status status
mach_down(default)[2022]:       2015/01/20_20:43:58 info: Taking over resource group 192.168.56.225/24/eth0
ResourceManager(default)[2049]: 2015/01/20_20:43:58 info: Acquiring resource group: zbdba1 192.168.56.225/24/eth0 drbddisk::zbdba Filesystem::/dev/drbd0::/mysql::ext4 mysql
/usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.56.225)[2077]: 2015/01/20_20:43:58 INFO:  Resource is stopped
ResourceManager(default)[2049]: 2015/01/20_20:43:58 info: Running /etc/ha.d/resource.d/IPaddr 192.168.56.225/24/eth0 start
IPaddr(IPaddr_192.168.56.225)[2164]:    2015/01/20_20:43:59 INFO: Using calculated netmask for 192.168.56.225: 255.255.255.0
IPaddr(IPaddr_192.168.56.225)[2164]:    2015/01/20_20:43:59 INFO: eval ifconfig eth0:0 192.168.56.225 netmask 255.255.255.0 broadcast 192.168.56.255
/usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.56.225)[2138]: 2015/01/20_20:43:59 INFO:  Success
ResourceManager(default)[2049]: 2015/01/20_20:43:59 info: Running /etc/ha.d/resource.d/drbddisk zbdba start
/usr/lib/ocf/resource.d//heartbeat/Filesystem(Filesystem_/dev/drbd0)[2300]:     2015/01/20_20:43:59 INFO:  Resource is stopped
ResourceManager(default)[2049]: 2015/01/20_20:43:59 info: Running /etc/ha.d/resource.d/Filesystem /dev/drbd0 /mysql ext4 start
Filesystem(Filesystem_/dev/drbd0)[2379]:        2015/01/20_20:43:59 INFO: Running start for /dev/drbd0 on /mysql
/usr/lib/ocf/resource.d//heartbeat/Filesystem(Filesystem_/dev/drbd0)[2371]:     2015/01/20_20:43:59 INFO:  Success
ResourceManager(default)[2049]: 2015/01/20_20:43:59 info: Running /etc/init.d/mysql  start
mach_down(default)[2022]:       2015/01/20_20:44:03 info: /usr/share/heartbeat/mach_down: nice_failback: foreign resources acquired
mach_down(default)[2022]:       2015/01/20_20:44:03 info: mach_down takeover complete for node zbdba1.
Jan 20 20:44:03 zbdba2 heartbeat: [1573]: info: mach_down takeover complete.
Jan 20 20:44:30 zbdba2 heartbeat: [1573]: WARN: node zbdba1: is dead
Jan 20 20:44:30 zbdba2 heartbeat: [1573]: info: Dead node zbdba1 gave up resources.
Jan 20 20:44:30 zbdba2 heartbeat: [1573]: info: Link zbdba1:eth0 dead.

查看zbdba1的drbd角色：

[root@zbdba1 ha.d]# service drbd status
drbd driver loaded OK; device status:
version: 8.4.0 (api:1/proto:86-100)
GIT-hash: 28753f559ab51b549d16bcf487fe625d5919c49c build by root@Dbrd1, 2015-01-19 03:52:17
m:res    cs         ro                 ds                 p  mounted  fstype
0:zbdba  Connected  Secondary/Primary  UpToDate/UpToDate  C

查看zbdba2的drbd角色：

[root@zbdba2 ~]# service drbd status
drbd driver loaded OK; device status:
version: 8.4.0 (api:1/proto:86-100)
GIT-hash: 28753f559ab51b549d16bcf487fe625d5919c49c build by root@Dbrd1, 2015-01-19 03:52:17
m:res    cs         ro                 ds                 p  mounted  fstype
0:zbdba  Connected  Primary/Secondary  UpToDate/UpToDate  C  /mysql   ext4

查看zbdba2的VIP：

[root@zbdba2 ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr 08:00:27:EB:CE:6F 
          inet addr:192.168.56.221  Bcast:192.168.56.255  Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:feeb:ce6f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:906411 errors:0 dropped:0 overruns:0 frame:0
          TX packets:396042 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1364740285 (1.2 GiB)  TX bytes:26821136 (25.5 MiB)

eth0:0    Link encap:Ethernet  HWaddr 08:00:27:EB:CE:6F 
          inet addr:192.168.56.225  Bcast:192.168.56.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

查看zbdba2上mysql是否正常运行：

[root@zbdba2 ~]# ps -ef |grep mysql
root      2508     1  0 20:43 ?        00:00:00 /bin/sh /mysql/bin/mysqld_safe --datadir=/mysql/data --pid-file=/mysql/data/zbdba2.pid
mysql     2647  2508  0 20:44 ?        00:00:01 /mysql/bin/mysqld --basedir=/mysql --datadir=/mysql/data --plugin-dir=/mysql/lib/plugin --user=mysql --log-error=/mysql/data/zbdba2.err --pid-file=/mysql/data/zbdba2.pid
root      2734  1824  0 20:46 pts/0    00:00:00 grep mysql
[root@zbdba2 ~]# mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.12 MySQL Community Server (GPL)

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

哈哈，终于测试成功，heartbeat不会监控mysql服务是否正常、所以需要自己写脚本监控mysql的服务状态