实验环境：
由于新公司的服务器全部是Debian，以前接触全部是Centos，过程中遇见了很多坑，特此记录
192.168.3.143：ha.cluster.master1
192.168.3.148：ha.cluster.slave1
一、安装mysql
1.1、第一个坑，使用apt-get安装的mysql-5.5，服务启动正常，连接正常，但是服务停止不了，尝试过几种停止方法
1.1.1、/etc/init.d/mysql stop
1.1.2、service mysql stop
1.1.3、kill -9 pkill -9
全部无法停止，错误日志显示正常退出，但是进程依然在，可以使用客户端工具连接到服务器，花了一个上午依然没有解决，于是尝试编译安装mysql解决问题。。。。。。有哪位大仙遇到过类似问题，欢迎留言告知，谢谢
1.2、编译步骤和CentOS没什么差别，只不过依赖貌似不太一样，下面记录编译过程
1.2.1、编译安装
root@ha:/usr/local/src/mysql-5.5.38# apt-get install cmake g++ libncurses5-dev root@ha:/usr/local/src/mysql-5.5.38# cmake -DCMAKE_INSTALL_PREFIX=/usr/local/mysql \ -DMYSQL_UNIX_ADDR=/space/mysql/data/mysql.sock \ -DDEFAULT_CHARSET=utf8 -DDEFAULT_COLLATION=utf8_general_ci \ -DWITH_EXTRA_CHARSETS:STRING=utf8,gbk -DWITH_MYISAM_STORAGE_ENGINE=1 \ -DWITH_INNOBASE_STORAGE_ENGINE=1 -DWITH_MEMORY_STORAGE_ENGINE=1-DWITH_READLINE=1 \ -DENABLED_LOCAL_INFILE=1 -DMYSQL_DATADIR=/space/mysql/data -DMYSQL_USER=mysql -DMYSQL_TCP_PORT=3306 root@ha:/usr/local/src/mysql-5.5.38# make && make install
1.2.2、准备配置文件等
root@ha:/usr/local/mysql/support-files# cp my-large.cnf /etc/my.cnf root@ha:/usr/local/mysql/support-files# cp mysql.server /etc/init.d/mysqld root@ha:/usr/local/mysql/support-files# chmod +x /etc/init.d/mysqld root@ha:/usr/local/mysql/support-files# useradd mysql root@ha:/usr/local/mysql# chmod +x scripts/mysql_install_db root@ha:/usr/local/mysql# ./scripts/mysql_install_db --datadir=/space/mysql/data --basedir=/usr/local/mysql --user=mysql
1.2.3、第二个坑（服务启动不了，提示找不到mysql.service）
在Centos中，将mysql.server拷贝到init.d下，执行chkconfig就可以了，Debian中要做下面的操作，否则服务启动不了
root@ha:/usr/local/mysql# update-rc.d mysqld defaults
至此，mysql安装完成，经测试，启动停止终于正常了，哭了。。
二、配置drbd
2.1、安装
2.1.1、
root@ha:/usr/local/mysql# apt-get update root@ha:/usr/local/mysql# apt-get -y upgrade root@ha:/usr/local/mysql# apt-get install drbd-utils
2.1.2、查看系统是否正确加载了drbd模块
root@ha:/usr/local/mysql/support-files# lsmod |grep drbd drbd 318526 4 lru_cache 13032 1 drbd libcrc32c 12426 1 drbd
2.2、配置两台节点的ssh互信和hosts
2.2.1、master1节点
root@ha:/usr/local/mysql# ssh-keygen -t rsa -P root@ha:/usr/local/mysql# ssh-copy-id -i .ssh/id_rsa.pub root@192.168.3.148 root@ha:/usr/local/mysql/support-files# cat /etc/hosts 127.0.0.1 localhost ha.cluster.master1 192.168.2.56 ha.cluster.master1 192.168.3.143 ha.cluster.master1 192.168.3.148 ha.cluster.slave1
2.2.2、slave1节点
root@ha:~# ssh-keygen -t rsa -P root@ha:~# ssh-copy-id -i .ssh/id_rsa.pub root@192.168.3.148 root@ha:~# cat /etc/hosts 127.0.0.1 localhost ha.cluster.slave1 192.168.2.56 ha.cluster.slave1 192.168.3.143 ha.cluster.master1 192.168.3.148 ha.cluster.slave1
注意：debian如果需要改本机host名称，还需要同时更改/etc/hostname才能生效
drbd添加磁盘就不记录了，正常操作，格式化
2.3、配置drbd资源（两个节点配置文件一样，配置完毕拷贝到另一个节点上）
root@ha:/etc/drbd.d# vim global_common.conf
global {
usage-count no;
}
common {
net {
cram-hmac-alg "sha1";
shared-secret "mydrbd1fa2jg8";
}
syncer {
rate 200M;
}
}
resource r0 {
on ha.cluster.master1 {
device /dev/drbd0;
disk /dev/sdb1;
address 192.168.3.143:7789;
meta-disk internal;
}
on ha.cluster.slave1 {
device /dev/drbd0;
disk /dev/sdb1;
address 192.168.3.148:7789;
meta-disk internal;
}
}
root@ha:/etc/drbd.d# scp global_common.conf ha.cluster.slave1:/etc/drbd.d/
2.4、在两个节点上创建定义的drbd资源
root@ha:/etc/drbd.d# drbdadm create-md r0 --== Thank you for participating in the global usage survey ==-- The server's response is: Writing meta data... initializing activity log NOT initialized bitmap New drbd meta data block successfully created. root@ha:~# drbdadm create-md r0 --== Thank you for participating in the global usage survey ==-- The server's response is: Writing meta data... initializing activity log NOT initialized bitmap New drbd meta data block successfully created.
2.5、启动drbd服务并查看状态
root@ha:~# /etc/init.d/drbd start version: 8.4.3 (api:1/proto:86-101) srcversion: 1A9F77B1CA5FF92235C2213 0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r----- ns:21176 nr:90328 dw:91024 dr:27909 al:7 bm:10 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 root@ha:/etc/drbd.d# /etc/init.d/drbd start version: 8.4.3 (api:1/proto:86-101) srcversion: 1A9F77B1CA5FF92235C2213 0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r----- ns:21176 nr:90328 dw:91024 dr:27909 al:7 bm:10 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
现在两台服务器得drbd都处于从状态，并且数据不是同步的
2.6、在master1节点手动提升为主节点并挂载drbd分区
root@ha:/etc/drbd.d# drbdadm primary r0 root@ha:/etc/drbd.d# mount /dev/drbd0 /space/mysql/data/ root@ha:/etc/drbd.d# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) srcversion: 1A9F77B1CA5FF92235C2213 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- ns:44724 nr:696 dw:45420 dr:10531 al:16 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
2.7、重新安装mysql库使其运行在drbd分区上
2.7.1、创建测试表并提升从节点为主查看数据是否存在
root@ha:/usr/local/mysql# ./scripts/mysql_install_db --datadir=/space/mysql/data --basedir=/usr/local/mysql --user=mysql
root@ha:/usr/local/mysql# chmod -R mysql.mysql /space/mysql/data
root@ha:/usr/local/mysql# service mysqld start
root@ha:/usr/local/mysql# mysql -uroot
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.5.38-log Source distribution
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
mysql> create database jumpserver;
mysql> create table user(id int primary key auto_increment, name varchar(30), description varchar(500));
mysql> insert into jumpserver.user (name,description)value('gouyacai','administrator');
mysql> select * from jumpserver.user;
+----+---------------+---------------+
| id | name | description |
+----+---------------+---------------+
| 1 | gouyacai | administrator |
+----+---------------+---------------+
2 rows in set (0.01 sec)
mysql>
2.7.2、手动提升slave为主
1、主操作
root@ha:/usr/local/mysql# service mysqld stop root@ha:/usr/local/mysql# umount /space/mysql/data root@ha:/usr/local/mysql# drbdadm secondary r0 root@ha:/usr/local/mysql# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) srcversion: 1A9F77B1CA5FF92235C2213 0: cs:Connected ro:Secondary/Secondary ds:UpToDate/UpToDate C r----- ns:45208 nr:696 dw:45904 dr:12615 al:16 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
2、从操作
root@ha:~# drbdadm primary r0 root@ha:~# mount /dev/drbd0 /space/mysql/data/ root@ha:~# service mysqld start
2.7.3、登录从上的mysql查看数据
mysql> select * from jumpserver.user; +----+---------------+---------------+ | id | name | description | +----+---------------+---------------+ | 1 | gouyacai | administrator | +----+---------------+---------------+ 2 rows in set (0.00 sec) mysql>
可以看出，从上的drbd数据已经同步过来。至此，drbd配置完成。
三、配置HeartBeat实现故障自动切换
3.1、配置认证文件
root@ha:/etc/ha.d# cat authkeys # Authentication file. Must be mode 600 auth 3 3 md5 ha-cluster-memcache root@ha:/etc/ha.d# chmod 600 authkeys
3.2、配置时间同步
master和从都要配置
*/5 * * * * /sbin/ntpdate 192.168.2.57 &> /dev/null
3.3、heartbeat主配置文件
1、master
root@ha:/etc/ha.d# vim ha.cf logfacility local0 keepalive 2 deadtime 5 warntime 10 initdead 10 udpport 694 ucast eth0 192.168.3.148 # 使用单播方式，地址是对方节点的ip用作心跳检测 # ping 192.168.2.1 # 当对方节点ping不通时，就ping这个ip auto_failback off use_logd yes node ha.cluster.master1 node ha.cluster.slave1
2、slave
root@ha:/etc/ha.d# vim ha.cf logfacility local0 keepalive 2 deadtime 5 warntime 10 initdead 10 udpport 694 ucast eth0 192.168.3.143 auto_failback off use_logd yes node ha.cluster.master1 node ha.cluster.slave1
3.4、配置资源
1、主
root@ha:/etc/ha.d# vim haresources ha.cluster.master1 IPaddr::192.168.3.151/16/eth0:0 drbddisk::r0 Filesystem::/dev/drbd0::/space/mysql/data::ext3 mysqld
2、从
root@ha:/etc/ha.d# cat haresources #ha.cluster.master1 192.168.3.151/16/eth0:0 memcache ha.cluster.master1 IPaddr::192.168.3.151/16/eth0:0 drbddisk::r0 Filesystem::/dev/drbd0::/space/mysql/data::ext3 mysqld
注：该文件内IPaddr,Filesystem等脚本存放路径在/etc/ha.d/resource.d/下，也可以在该目录下存放其他服务启动脚本，如memcached，nginx等。将相同脚本名称添加到/etc/ha.d/haresources内容中，从而跟随heartbeat启动而启动该脚本。
IPaddr::192.168.3.151/16/eth0:0：用IPaddr脚本配置浮动VIP
drbddisk::r0：用drbddisk脚本实现DRBD主从节点资源组的挂载和卸载
Filesystem::/dev/drbd0::data::ext4：用Filesystem脚本实现磁盘挂载和卸载
3.5、取消mysqld和drbd的自动启动
因为主heartbeat启动的时候会挂载drbd文件系统以及启动mysql，切换的时候会将主上的mysql停止并卸载文件系统，从上会挂载文件系统，并启动mysql。
1、主
root@ha:~# sysv-rc-conf --level 2345 mysqld off root@ha:~# sysv-rc-conf --level 2345 drbd off
2、从
root@ha:~# sysv-rc-conf --level 2345 mysqld off root@ha:~# sysv-rc-conf --level 2345 drbd off
3.6、测试
重启master和slave两台节点，看是否heartbeat会自动挂载drbd设备并启动mysql
root@ha:~# ps -ef |grep heartbeat root 924 1 0 03:53 ? 00:00:00 heartbeat: master control process root 930 924 0 03:53 ? 00:00:00 heartbeat: FIFO reader root 931 924 0 03:53 ? 00:00:00 heartbeat: write: ucast eth0 root 932 924 0 03:53 ? 00:00:00 heartbeat: read: ucast eth0 root 2251 967 0 03:57 pts/0 00:00:00 grep heartbeat root@ha:~# root@ha:~# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) srcversion: 1A9F77B1CA5FF92235C2213 0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r----- ns:24868 nr:0 dw:292 dr:31269 al:7 bm:4 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 root@ha:~# ifconfig eth0 Link encap:Ethernet HWaddr 00:0c:29:be:10:c3 inet addr:192.168.3.143 Bcast:192.168.255.255 Mask:255.255.0.0 inet6 addr: fe80::20c:29ff:febe:10c3/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:4341 errors:0 dropped:5 overruns:0 frame:0 TX packets:18732 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:688318 (672.1 KiB) TX bytes:26807817 (25.5 MiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:155 errors:0 dropped:0 overruns:0 frame:0 TX packets:155 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:32655 (31.8 KiB) TX bytes:32655 (31.8 KiB) root@ha:~# ps -ef |grep mysql root 2276 967 0 03:58 pts/0 00:00:00 grep mysql
可以看到，master上drbd设备没有挂载，vip没有漂移上来，mysql未启动
下面查看heartbeat日志
Aug 14 03:29:46 ha /usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.3.151): [1566]: INFO: Resource is stopped Aug 14 03:29:46 ha ResourceManager(default)[1511]: [1581]: info: Running /etc/ha.d/resource.d/IPaddr 192.168.3.151/16/eth0:0 start Aug 14 03:29:46 ha IPaddr(IPaddr_192.168.3.151)[1606]: [1637]: INFO: Using calculated netmask for 192.168.3.151: 255.255.0.0 Aug 14 03:29:46 ha IPaddr(IPaddr_192.168.3.151)[1606]: [1659]: INFO: eval ifconfig eth0:0 192.168.3.151 netmask 255.255.0.0 broadcast 192.168.255.255 Aug 14 03:29:46 ha /usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.3.151): [1681]: INFO: Success Aug 14 03:29:46 ha ResourceManager(default)[1511]: [1710]: info: Running /etc/ha.d/resource.d/drbddisk r0 start Aug 14 03:29:46 ha kernel: [ 24.395989] block drbd0: role( Secondary -> Primary ) Aug 14 03:29:46 ha kernel: [ 24.396555] block drbd0: new current UUID FD9C41E5A4B2DC47:A077794C3E2A99EA:1F567BC15BEB6FB0:1F557BC15BEB6FB1 Aug 14 03:29:46 ha /usr/lib/ocf/resource.d//heartbeat/Filesystem(Filesystem_/dev/d: [1770]: INFO: Resource is stopped Aug 14 03:29:46 ha ResourceManager(default)[1511]: [1785]: info: Running /etc/ha.d/resource.d/Filesystem /dev/drbd0 /space/mysql/data ext3 start Aug 14 03:29:46 ha Filesystem(Filesystem_/dev/drbd0)[1792]: [1823]: INFO: Running start for /dev/drbd0 on /space/mysql/data Aug 14 03:29:46 ha kernel: [ 24.519309] EXT4-fs (drbd0): mounting ext3 file system using the ext4 subsystem Aug 14 03:29:46 ha /usr/lib/ocf/resource.d//heartbeat/Filesystem(Filesystem_/dev/d: [1842]: INFO: Success Aug 14 03:29:46 ha kernel: [ 24.539684] EXT4-fs (drbd0): mounted filesystem with ordered data mode. Opts: (null) Aug 14 03:29:46 ha ResourceManager(default)[1511]: [1872]: info: Running /etc/init.d/mysqld start Aug 14 03:29:48 ha ResourceManager(default)[1511]: [2207]: ERROR: Return code 1 from /etc/init.d/mysqld Aug 14 03:29:48 ha ResourceManager(default)[1511]: [2209]: CRIT: Giving up resources due to failure of mysqld Aug 14 03:29:48 ha ResourceManager(default)[1511]: [2211]: info: Releasing resource group: ha.cluster.master1 IPaddr::192.168.3.151/16/eth0:0 drbddisk::r0 Filesystem::/dev/drbd0::/space/mysql/data::ext3 mysqld Aug 14 03:29:48 ha ResourceManager(default)[1511]: [2222]: info: Running /etc/init.d/mysqld stop Aug 14 03:29:48 ha ResourceManager(default)[1511]: [2247]: info: Running /etc/ha.d/resource.d/Filesystem /dev/drbd0 /space/mysql/data ext3 stop Aug 14 03:29:48 ha Filesystem(Filesystem_/dev/drbd0)[2254]: [2285]: INFO: Running stop for /dev/drbd0 on /space/mysql/data Aug 14 03:29:48 ha Filesystem(Filesystem_/dev/drbd0)[2254]: [2300]: INFO: Trying to unmount /space/mysql/data Aug 14 03:29:48 ha Filesystem(Filesystem_/dev/drbd0)[2254]: [2308]: INFO: unmounted /space/mysql/data successfully Aug 14 03:29:48 ha /usr/lib/ocf/resource.d//heartbeat/Filesystem(Filesystem_/dev/d: [2315]: INFO: Success Aug 14 03:29:48 ha ResourceManager(default)[1511]: [2330]: info: Running /etc/ha.d/resource.d/drbddisk r0 stop Aug 14 03:29:48 ha kernel: [ 26.836087] block drbd0: role( Primary -> Secondary ) Aug 14 03:29:48 ha kernel: [ 26.836873] block drbd0: bitmap WRITE of 2 pages took 1 jiffies Aug 14 03:29:48 ha kernel: [ 26.836878] block drbd0: 248 KB (62 bits) marked out-of-sync by on disk bit-map. Aug 14 03:29:48 ha ResourceManager(default)[1511]: [2350]: info: Running /etc/ha.d/resource.d/IPaddr 192.168.3.151/16/eth0:0 stop Aug 14 03:29:48 ha IPaddr(IPaddr_192.168.3.151)[2375]: [2391]: INFO: ifconfig eth0:0 down Aug 14 03:29:48 ha /usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.3.151): [2395]: INFO: Success
可以看到上面的heartbeat日志显示启动mysql时发生了错误，导致drbd分区umont，vip释放等操作。于是手动启动mysql发现错误
180814 3:46:08 [ERROR] /usr/local/mysql/bin/mysqld: Can't create/write to file '/var/run/mysql/mysql.pid' (Errcode: 2) 180814 3:46:08 [ERROR] Can't start server: can't create PID file: No such file or directory 180814 03:46:08 mysqld_safe mysqld from pid file /var/run/mysql/mysql.pid ended
排查发现/var/run/下面没有mysql目录，于是手动创建并赋予了mysql属主，重启服务器测试，问题依旧。这里不明原因，有知道的大仙麻烦留言告知。。
解决办法：将my.cnf中的pid-file变量设置到/space/mysql/data/mysql.pid中解决
1、下面查看master的服务状态
Aug 14 03:54:07 ha ip-request-resp(default)[1521]: [1533]: received ip-request-resp IPaddr::192.168.3.151/16/eth0:0 OK yes Aug 14 03:54:07 ha ResourceManager(default)[1534]: [1545]: info: Acquiring resource group: ha.cluster.master1 IPaddr::192.168.3.151/16/eth0:0 drbddisk::r0 Filesystem::/dev/drbd0::/space/mysql/data::ext3 mysqld Aug 14 03:54:07 ha /usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.3.151): [1589]: INFO: Resource is stopped Aug 14 03:54:07 ha ResourceManager(default)[1534]: [1604]: info: Running /etc/ha.d/resource.d/IPaddr 192.168.3.151/16/eth0:0 start Aug 14 03:54:07 ha IPaddr(IPaddr_192.168.3.151)[1629]: [1660]: INFO: Using calculated netmask for 192.168.3.151: 255.255.0.0 Aug 14 03:54:07 ha IPaddr(IPaddr_192.168.3.151)[1629]: [1682]: INFO: eval ifconfig eth0:0 192.168.3.151 netmask 255.255.0.0 broadcast 192.168.255.255 Aug 14 03:54:07 ha /usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.3.151): [1709]: INFO: Success Aug 14 03:54:07 ha ResourceManager(default)[1534]: [1738]: info: Running /etc/ha.d/resource.d/drbddisk r0 start Aug 14 03:54:07 ha kernel: [ 42.027571] block drbd0: role( Secondary -> Primary ) Aug 14 03:54:07 ha heartbeat: [924]: info: remote resource transition completed. Aug 14 03:54:07 ha /usr/lib/ocf/resource.d//heartbeat/Filesystem(Filesystem_/dev/d: [1798]: INFO: Resource is stopped Aug 14 03:54:07 ha ResourceManager(default)[1534]: [1813]: info: Running /etc/ha.d/resource.d/Filesystem /dev/drbd0 /space/mysql/data ext3 start Aug 14 03:54:07 ha Filesystem(Filesystem_/dev/drbd0)[1820]: [1851]: INFO: Running start for /dev/drbd0 on /space/mysql/data Aug 14 03:54:07 ha kernel: [ 42.402139] EXT4-fs (drbd0): mounting ext3 file system using the ext4 subsystem Aug 14 03:54:07 ha kernel: [ 42.424843] EXT4-fs (drbd0): mounted filesystem with ordered data mode. Opts: (null) Aug 14 03:54:07 ha /usr/lib/ocf/resource.d//heartbeat/Filesystem(Filesystem_/dev/d: [1874]: INFO: Success Aug 14 03:54:07 ha ResourceManager(default)[1534]: [1904]: info: Running /etc/init.d/mysqld start Aug 14 03:55:01 ha CRON[2248]: (root) CMD (/sbin/ntpdate 192.168.2.57 &> /dev/null) Aug 14 04:00:01 ha CRON[2278]: (root) CMD (/sbin/ntpdate 192.168.2.57 &> /dev/null) root@ha:/space/mysql/data# ifconfig eth0 Link encap:Ethernet HWaddr 00:0c:29:be:10:c3 inet addr:192.168.3.143 Bcast:192.168.255.255 Mask:255.255.0.0 inet6 addr: fe80::20c:29ff:febe:10c3/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:17311 errors:0 dropped:12 overruns:0 frame:0 TX packets:20789 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2799246 (2.6 MiB) TX bytes:28640153 (27.3 MiB) eth0:0 Link encap:Ethernet HWaddr 00:0c:29:be:10:c3 inet addr:192.168.3.151 Bcast:192.168.255.255 Mask:255.255.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:161 errors:0 dropped:0 overruns:0 frame:0 TX packets:161 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:32967 (32.1 KiB) TX bytes:32967 (32.1 KiB) root@ha:/space/mysql/data# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) srcversion: 1A9F77B1CA5FF92235C2213 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- ns:25020 nr:0 dw:444 dr:31537 al:7 bm:4 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 root@ha:/space/mysql/data# ps -ef |grep mysq; root 1914 1 0 03:54 ? 00:00:00 /bin/sh /usr/local/mysql/bin/mysqld_safe --datadir=/space/mysql/data --pid-file=/space/mysql/data/mysql.pid mysql 2214 1914 0 03:54 ? 00:00:00 /usr/local/mysql/bin/mysqld --basedir=/usr/local/mysql --datadir=/space/mysql/data --plugin-dir=/usr/local/mysql/lib/plugin --user=mysql --log-error=/space/mysql/data/ha.cluster.master1.err --pid-file=/space/mysql/data/mysql.pid --socket=/space/mysql/data/mysql.sock --port=3306 root 2425 967 0 04:14 pts/0 00:00:00 grep mysq root@ha:/space/mysql/data# mysql -uroot Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.5.38-log Source distribution Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> mysql> mysql> select * from jumpserver.user -> ; +----+---------------+---------------+ | id | name | description | +----+---------------+---------------+ | 1 | gouyacai | administrator | | 2 | zhangjinsongt | admin | +----+---------------+---------------+ 2 rows in set (0.03 sec)
2、slave的状态
root@ha:~# ifconfig eth0 Link encap:Ethernet HWaddr 00:0c:29:38:b1:90 inet addr:192.168.3.148 Bcast:192.168.255.255 Mask:255.255.0.0 inet6 addr: fe80::20c:29ff:fe38:b190/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:34696 errors:0 dropped:15 overruns:0 frame:0 TX packets:2673 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:29799737 (28.4 MiB) TX bytes:434321 (424.1 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) root@ha:~# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) srcversion: 1A9F77B1CA5FF92235C2213 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r----- ns:0 nr:25128 dw:25128 dr:0 al:0 bm:4 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 root@ha:~# ps -ef |grep mysql root 1086 994 0 04:15 pts/0 00:00:00 grep mysql
以上状态可以确认一切正常，下面关掉master上的heartbeat服务，查看vip和mysql是否会切换到slave上面
1、主操作
root@ha:~# service heartbeat stop root@ha:~# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) srcversion: 1A9F77B1CA5FF92235C2213 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r----- ns:25312 nr:300 dw:1036 dr:31553 al:7 bm:4 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 root@ha:~# ifconfig eth0 Link encap:Ethernet HWaddr 00:0c:29:be:10:c3 inet addr:192.168.3.143 Bcast:192.168.255.255 Mask:255.255.0.0 inet6 addr: fe80::20c:29ff:febe:10c3/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:21995 errors:0 dropped:16 overruns:0 frame:0 TX packets:21930 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:3770763 (3.5 MiB) TX bytes:29091655 (27.7 MiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:163 errors:0 dropped:0 overruns:0 frame:0 TX packets:163 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:33071 (32.2 KiB) TX bytes:33071 (32.2 KiB) root@ha:~# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 49G 2.1G 44G 5% / udev 10M 0 10M 0% /dev tmpfs 98M 4.6M 93M 5% /run tmpfs 244M 0 244M 0% /dev/shm tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 244M 0 244M 0% /sys/fs/cgroup root@ha:~# ps -ef |grep mysql root 2770 2728 0 04:18 pts/0 00:00:00 grep mysql
可以看到，master停止了heartbeat之后，drbd分区已经umount，vip已经释放，mysql服务自动关闭，并且drbd状态已经变成了Secondary
2、下面是slave的状态
root@ha:~# ifconfig eth0 Link encap:Ethernet HWaddr 00:0c:29:38:b1:90 inet addr:192.168.3.148 Bcast:192.168.255.255 Mask:255.255.0.0 inet6 addr: fe80::20c:29ff:fe38:b190/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:39219 errors:0 dropped:16 overruns:0 frame:0 TX packets:3537 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:30556311 (29.1 MiB) TX bytes:861986 (841.7 KiB) eth0:0 Link encap:Ethernet HWaddr 00:0c:29:38:b1:90 inet addr:192.168.3.151 Bcast:192.168.255.255 Mask:255.255.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) root@ha:~# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 49G 2.1G 44G 5% / udev 10M 0 10M 0% /dev tmpfs 98M 4.6M 93M 5% /run tmpfs 244M 0 244M 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 244M 0 244M 0% /sys/fs/cgroup /dev/drbd0 20G 75M 19G 1% /space/mysql/data root@ha:~# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) srcversion: 1A9F77B1CA5FF92235C2213 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- ns:300 nr:25312 dw:25612 dr:6693 al:7 bm:4 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 root@ha:~# ps -ef |grep mysql root 1519 1 0 04:17 ? 00:00:00 /bin/sh /usr/local/mysql/bin/mysqld_safe --datadir=/space/mysql/data --pid-file=/space/mysql/data/mysql.pid mysql 1819 1519 0 04:17 ? 00:00:00 /usr/local/mysql/bin/mysqld --basedir=/usr/local/mysql --datadir=/space/mysql/data --plugin-dir=/usr/local/mysql/lib/plugin --user=mysql --log-error=/space/mysql/data/ha.cluster.slave1.err --pid-file=/space/mysql/data/mysql.pid --socket=/space/mysql/data/mysql.sock --port=3306 root 1861 994 0 04:20 pts/0 00:00:00 grep mysql
在看一下heartbeat的日志
Aug 14 04:17:01 ha CRON[1088]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) Aug 14 04:17:26 ha kernel: [ 1439.804089] block drbd0: peer( Primary -> Secondary ) Aug 14 04:17:26 ha heartbeat: [918]: info: Received shutdown notice from 'ha.cluster.master1'. Aug 14 04:17:26 ha heartbeat: [918]: info: Resources being acquired from ha.cluster.master1. Aug 14 04:17:26 ha heartbeat: [918]: debug: StartNextRemoteRscReq(): child count 1 Aug 14 04:17:26 ha heartbeat: [1090]: info: acquire local HA resources (standby). Aug 14 04:17:26 ha heartbeat: [1090]: info: local HA resource acquisition completed (standby). Aug 14 04:17:26 ha heartbeat: [918]: info: Standby resource acquisition done [all]. Aug 14 04:17:26 ha heartbeat: [918]: debug: StartNextRemoteRscReq(): child count 1 Aug 14 04:17:26 ha heartbeat: [1091]: info: No local resources [/usr/share/heartbeat/ResourceManager listkeys ha.cluster.slave1] to acquire. Aug 14 04:17:26 ha heartbeat: [1116]: debug: notify_world: setting SIGCHLD Handler to SIG_DFL Aug 14 04:17:26 ha harc(default)[1116]: [1122]: info: Running /etc/ha.d//rc.d/status status Aug 14 04:17:27 ha mach_down(default)[1127]: [1147]: info: Taking over resource group IPaddr::192.168.3.151/16/eth0:0 Aug 14 04:17:27 ha ResourceManager(default)[1148]: [1159]: info: Acquiring resource group: ha.cluster.master1 IPaddr::192.168.3.151/16/eth0:0 drbddisk::r0 Filesystem::/dev/drbd0::/space/mysql/data::ext3 mysqld Aug 14 04:17:27 ha /usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.3.151): [1203]: INFO: Resource is stopped Aug 14 04:17:27 ha ResourceManager(default)[1148]: [1218]: info: Running /etc/ha.d/resource.d/IPaddr 192.168.3.151/16/eth0:0 start Aug 14 04:17:27 ha IPaddr(IPaddr_192.168.3.151)[1243]: [1274]: INFO: Using calculated netmask for 192.168.3.151: 255.255.0.0 Aug 14 04:17:27 ha IPaddr(IPaddr_192.168.3.151)[1243]: [1296]: INFO: eval ifconfig eth0:0 192.168.3.151 netmask 255.255.0.0 broadcast 192.168.255.255 Aug 14 04:17:27 ha /usr/lib/ocf/resource.d//heartbeat/IPaddr(IPaddr_192.168.3.151): [1318]: INFO: Success Aug 14 04:17:27 ha ResourceManager(default)[1148]: [1347]: info: Running /etc/ha.d/resource.d/drbddisk r0 start Aug 14 04:17:27 ha kernel: [ 1440.194942] block drbd0: role( Secondary -> Primary ) Aug 14 04:17:27 ha /usr/lib/ocf/resource.d//heartbeat/Filesystem(Filesystem_/dev/d: [1407]: INFO: Resource is stopped Aug 14 04:17:27 ha ResourceManager(default)[1148]: [1422]: info: Running /etc/ha.d/resource.d/Filesystem /dev/drbd0 /space/mysql/data ext3 start Aug 14 04:17:27 ha Filesystem(Filesystem_/dev/drbd0)[1429]: [1460]: INFO: Running start for /dev/drbd0 on /space/mysql/data Aug 14 04:17:27 ha kernel: [ 1440.308138] EXT4-fs (drbd0): mounting ext3 file system using the ext4 subsystem Aug 14 04:17:27 ha kernel: [ 1440.327117] EXT4-fs (drbd0): mounted filesystem with ordered data mode. Opts: (null) Aug 14 04:17:27 ha /usr/lib/ocf/resource.d//heartbeat/Filesystem(Filesystem_/dev/d: [1479]: INFO: Success Aug 14 04:17:27 ha ResourceManager(default)[1148]: [1509]: info: Running /etc/init.d/mysqld start Aug 14 04:17:30 ha mach_down(default)[1127]: [1843]: info: /usr/share/heartbeat/mach_down: nice_failback: foreign resources acquired Aug 14 04:17:30 ha mach_down(default)[1127]: [1848]: info: mach_down takeover complete for node ha.cluster.master1. Aug 14 04:17:30 ha heartbeat: [918]: info: mach_down takeover complete. Aug 14 04:17:34 ha heartbeat: [918]: WARN: node ha.cluster.master1: is dead Aug 14 04:17:34 ha heartbeat: [918]: info: Dead node ha.cluster.master1 gave up resources. Aug 14 04:17:34 ha heartbeat: [918]: info: Link ha.cluster.master1:eth0 dead. Aug 14 04:20:01 ha CRON[1851]: (root) CMD (/sbin/ntpdate 192.168.2.57 &> /dev/null) root@ha:~# mysql -uroot Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.5.38-log Source distribution Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> select * from jumpserver.user; +----+---------------+---------------+ | id | name | description | +----+---------------+---------------+ | 1 | gouyacai | administrator | | 2 | zhangjinsongt | admin | +----+---------------+---------------+ 2 rows in set (0.03 sec) mysql>
切换成功
至此，大功告成。
转载于:https://blog.51cto.com/gouyc/2159825
156
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
