一. 环境描述
操作系统：RHEL 6.4
mysql ：5.6.27
| 角色 | IP 地址 | 主机名 | Server ID | 类型 |
| 主库 | 192.168.222.156 | oradb | 10 | 写入 |
| 备库、MHA管理服务器 | 192.168.222.157 | mysqldb2 | 11 | 读、监控集群组 |
二、部署MHA
1.主库安装 MHA NODE
1.1 安装需要的PERL模块
yum -y install *zlib-dev*
install.sh
#!/bin/bash
wget http://xrl.us/cpanm --no-check-certificate
mv cpanm /usr/bin
chmod 755 /usr/bin/cpanm
cat >/root/list<<EOF
install DBD::mysql
EOF
for package in `cat /root/list`
do
cpanm $package
done
1.2 安装 MHA NODE
从 https://github.com/yoshinorim/mha4mysql-node 下载最新版本的 mha4mysql-node 源代码（目前是 0.57版）
unzip mha4mysql-node-master.zip
cd mha4mysql-node-master
perl Makefile.PL
make
make install
2. 备库安装 MHA Manager
2.1 安装需要的perl模块
yum -y install *zlib-dev*
install.sh
#!/bin/bash
wget http://xrl.us/cpanm --no-check-certificate
mv cpanm /usr/bin
chmod 755 /usr/bin/cpanm
cat >/root/list<<EOF
install DBD::mysql
install Config::Tiny
install Log::Dispatch
install Parallel::ForkManager
install Time::HiRes
EOF
for package in `cat /root/list`
do
cpanm $package
done
2.2 安装MHA NODE
从 https://github.com/yoshinorim/mha4mysql-node 下载最新版本的 mha4mysql-node 源代码（目前是 0.57版）
unzip mha4mysql-node-master.zip
cd mha4mysql-node-master
perl Makefile.PL
make
make install
2.3 安装 MHA Manager
从 https://github.com/yoshinorim/mha4mysql-manager下载最新版本的 mha4mysql-manager 源代码（目前是 0.57版）
unzip mha4mysql-manager-master.zip
cd mha4mysql-manager-master
perl Makefile.PL
make
make install
3. 配置 SSH 登录无密码验证
3.1 主库配置到 MHA manager 节点（备库节点）SSH登录无密码验证
vi /etc/hosts
192.168.222.157 mysqldb2
192.168.222.156 oradb
ssh-keygen -t rsa
ssh-coyp-id -i /root/.ssh/id_rsa.pub root@192.168.222.157
ssh-coyp-id -i /root/.ssh/id_rsa.pub root@192.168.222.156
ssh root@192.168.222.156
ssh root@oradb
ssh root@192.168.222.157
ssh root@mysqldb2
3.2 MHA manager 节点（备库节点）配置到主库配置 SSH登录无密码验证
vi /etc/hosts
192.168.222.157 mysqldb2
192.168.222.156 oradb
ssh-keygen -t rsa
ssh-coyp-id -i /root/.ssh/id_rsa.pub root@192.168.222.157
ssh-coyp-id -i /root/.ssh/id_rsa.pub root@192.168.222.156
ssh root@192.168.222.156
ssh root@oradb
ssh root@192.168.222.157
ssh root@mysqldb2
4. 搭建主从复制
4.1 修改主库my.cnf 配置服务器ID为10
vi /app/oracle/oradata2/mysql-5.6.27/my.cnf
[mysqld]
server_id = 10
4.1 修改备库my.cnf 配置
vi /usr/local/mysql/my.cnf
[mysqld]
server_id = 10
# slave config
log_bin=/usr/local/mysql/data/mysql-bin
relay_log=/usr/local/mysql/data/relay-bin
binlog_format=row
relay_log_purge=0
read_only=1
4.2 在主库上备份
mysqldump --master-data=2 --single-transaction -R -A > all.sql
其中 --master-data=2代表备份时刻记录 master的 Binlog 位置和 Position，--single-transaction 意思是获取一致性快照， -R 意思是备份相关的存储过程
，-A代表备份所有的 schema。
4.3 在主库上创建复制用户和 MHA监控用户
mysql> grant replication slave on *.* to 'repl'@'192.168.222.%' identified by '123456';
mysql> grant all on *.* to 'root'@'192.168.222.%' identified by '123456'; /*MHA 监控用户*/
4.4 查看主库上备份时刻 Binlog 的名称和位置，MASTER_LOG和 MASTER_LOG_POS
head -n 30 all.sql|grep -i "CHANGE MASTER TO"
-- CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000068', MASTER_LOG_POS=325;
4.5 将备份复制到备库
scp all.sql root@192.168.222.157:/root/olm
4.6 在备库导入数据
mysql> source /root/olm/all.sql
mysql> flush privileges;
mysql>
change master to master_host='192.168.222.156',master_user='repl',master_password='123456',
master_port=3306,master_log_file='mysql-bin.000068',master_log_pos=325;
start slave;
--查看复制状态
show slave status \G
5. 配置MHA
5.1 创建 MHA 工作目录，并且创建相关配置文件
mkdir -p /etc/masterha
vi /etc/masterha/app1.cnf
[server default]
manager_log=/masterha/app1/app1.log
manager_workdir=/masterha/app1
master_binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data
master_ip_failover_script=/usr/local/bin/master_ip_failover
master_ip_online_change_script=/usr/local/bin/master_ip_online_change
password=123456
ping_interval=1
remote_workdir=/tmp
repl_password=123456
repl_user=repl
secondary_check_script=/usr/local/bin/masterha_secondary_check -s mysqldb2 -s oradb --user=root --master_host=oradb --master_ip='192.168.222.156' --master_port=3306
shutdown_script=""
ssh_user=root
user=root
[server1]
hostname=192.168.222.156
port=3306
[server2]
candidate_master=1
check_repl_delay=0
hostname=192.168.222.157
port=3306
5.2 设置定期清理中继日志的脚本
crontab -e
0 4 * * * /usr/local/bin/purge_relay_logs --user=root --password=123456 --disable_replay_log_purge --port=3306 --workdir=/usr/local/mysql/data --disable_relay_log_purge >> /usr/loca/masterha/log/purge_relay_logs.log 2>&1
5.2 编辑 /etc/bashrc 文件，在文件末尾添加以下内容，避免MHA执行故障切换时无法找到 mysqlbinlog
PATH="$PATH:/usr/local/mysql/bin"
export PATH
5.3 检查 SSH 的配置
masterha_check_ssh --conf=/etc/masterha/app1.cnf
5.4 检查MYSQL复制
masterha_check_repl --conf=/etc/masterha/app1.cnf
5.5 启动 MHA Manger 监控
tail -f /masterha/app1/app1.log
--在另一个SSH 窗口启动 MHA Manger 监控。--remove_dead_master_conf 该参数代表当发生主从切换后，老的主库的IP将会从配置文件中移除。
--ignore_last_failover 在缺省情况下，如果MHA检测到连续发生宕机，而且两次宕机时间不足8小时的话，则不会进行 Failover，之所以这样限制
--是为了避免 ping-pong效应。该参数代表忽悠上次 MHA 触发切换产生的文件，默认情况下，MHA 发生切换后将会在 /masterha/paa1下产生 app1.
--failover.complete 文件，下次再次切换的时候如果发现目录存在该文件将不允许触发切换，除非在第一次切换后收到 rm -rf /masterha/paa1/app1. --failover.complete,出于方便考虑，我们每次启动MHA时会添加 --ignore_last_failover 参数。
nohup masterha_manager --conf=/etc/masterha/app1.cnf --remove_dead_master_conf --ignore_last_failover< /dev/null >/masterha/app1/manager.log 2>&1 &
--查看 MHA Manager 监控是否正常
masterha_check_status --conf=/etc/masterha/app1.cnf
app1 (pid:4758) is running(0:PING_OK), master:192.168.222.156
--在默认情况下，10秒内状态为10：INITIALIZING_MONITOR，当状态转变为0:PING_OK后表明已经开启了到 MASTER 端的监控。
关闭MHA的命令如下：
masterha_stop --conf=/etc/masterha/app1.cnf
6. 配置 VIP
6.1 在主库下载集群心跳软件 keepalived，并进行安装
访问 http://www.keepalived.org/software/ 下载 keepalived-1.2.18.tar.gz
tar vzxf keepalived-1.2.18.tar.gz
cd keepalived-1.2.18
./configure --prefix=/usr/local/keepalived
make
make install
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cd /etc/keepalived/
vi keepalived.conf
--下面是添加的配置
[root@oradb ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id MySQL-ha
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.222.158/24
}
}
service keepalived start
--是用 ip add 命令可以看到 eth0上已经绑定了虚拟IP 192.168.222.158 ，注意使用 ifconfig 是看不到虚拟IP的
[root@oradb ~]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:53:7e:f0 brd ff:ff:ff:ff:ff:ff
inet 192.168.222.156/24 brd 192.168.222.255 scope global eth0
inet 192.168.222.158/24 scope global secondary eth0
inet6 fe80::20c:29ff:fe53:7ef0/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:53:7e:fa brd ff:ff:ff:ff:ff:ff
inet 192.168.1.11/24 brd 192.168.1.255 scope global eth1
inet6 fe80::20c:29ff:fe53:7efa/64 scope link
valid_lft forever preferred_lft forever
6.2 在备库下载集群心跳软件 keepalived，并进行安装
访问 http://www.keepalived.org/software/ 下载 keepalived-1.2.18.tar.gz
tar vzxf keepalived-1.2.18.tar.gz
cd keepalived-1.2.18
./configure --prefix=/usr/local/keepalived
make
make install
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cd /etc/keepalived/
vi keepalived.conf
--下面是添加的配置
! Configuration File for keepalived
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id MySQL-ha
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 120
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.222.158/24
}
}
service keepalived start
6.2 MHA引入 keepalived
在MHA监控节点（备库）编辑 master_ip_failover 脚本
vi /usr/local/bin/master_ip_failover
#!/usr/bin/env perl
# Copyright (C) 2011 DeNA Co.,Ltd.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
## Note: This is a sample script and is not complete. Modify the script based on your environment.
use strict;
use warnings FATAL => 'all';
use Getopt::Long;
use MHA::DBHelper;
my (
$command, $ssh_user, $orig_master_host,
$orig_master_ip, $orig_master_port, $new_master_host,
$new_master_ip, $new_master_port, $new_master_user,
$new_master_password, $cmd
);
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
'new_master_user=s' => \$new_master_user,
'new_master_password=s' => \$new_master_password,
);
exit &main();
sub main {
if ( $command eq "stop" || $command eq "stopssh" ) {
# $orig_master_host, $orig_master_ip, $orig_master_port are passed.
# If you manage master ip address at global catalog database,
# invalidate orig_master_ip here.
my $exit_code = 1;
eval {
# updating global catalog, etc
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {
# all arguments are passed.
# If you manage master ip address at global catalog database,
# activate new_master_ip here.
# You can also grant write access (create user, set read_only=0, etc) here.
my $exit_code = 10;
eval {
my $new_master_handler = new MHA::DBHelper();
# args: hostname, port, user, password, raise_error_or_not
$new_master_handler->connect( $new_master_ip, $new_master_port,
$new_master_user, $new_master_password, 1 );
## Set read_only=0 on the new master
$new_master_handler->disable_log_bin_local();
print "Set read_only=0 on the new master.\n";
$new_master_handler->disable_read_only();
## Creating an app user on the new master
print "Creating app user on the new master..\n";
#FIXME_xxx_create_user( $new_master_handler->{dbh} );
#$new_master_handler->enable_log_bin_local();
#$new_master_handler->disconnect();
## Update master ip on the catalog database, etc
#FIXME_xxx;
#$cmd = 'ssh root@192.168.222.156 \./lvs-admin stop';
#$cmd = 'ssh'.$ssh_user.'@'.$orig_master_ip.'\'./lvs-admin stop\'';
$cmd = 'ssh '.$ssh_user.'@'.$orig_master_ip.' \./lvs-admin stop';
system($cmd);
$exit_code = 0;
};
if ($@) {
warn $@;
# If you want to continue failover, exit 10.
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "status" ) {
# do nothing
exit 0;
}
else {
&usage();
exit 1;
}
}
sub usage {
"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}
在脚本中绿色背景的代码是新增的，红色背景的代码是注释的。
在主库添加关闭 keepalived 的脚本
vi /root/lvs-admin
case "$1" in
"stop")
echo;
echo "stop keepalived ......";
service keepalived stop
;;
"start")
echo;
echo "start keepalive ......";
service keepalived start
;;
*)
echo "please input you select!"
;;
esac
chmod 755 /root/lvs-admin
7. 测试主库故障MHA自动把备库提升为主库并把虚拟IP漂移到新的主库（原来的备库）
7.1 在备库停止复制默认有二进制日志没传到备库
mysql> stop slave;
7.2 sysbench 生成测试数据，在主库上的 sbtest 库下创建 sbtest 表，共100万条数据
mysql> create database sbtest;
./sysbench --num-threads=4 --test=oltp --oltp-table-size=1000000 --oltp-read-only=off --init-rng=on --max-requests=0 \
--oltp-dist-type=uniform --max-time=1800 --mysql-user=root --db-driver=test --mysql-table-engine=innodb \
--oltp-test-mode=complex prepare
7.3 关闭主库 MHA会自动把备份库提升为新的主库，并把虚拟IP漂移到新的主库（原来的备库）
--关闭主库
mysqladmin -u root shutdown
--在MHA监控节点（备库）实时查看MHA日志
tail -f /masterha/app1/app1.log
Thu Feb 2 23:53:06 2017 - [info] MHA::MasterMonitor version 0.57.
Thu Feb 2 23:53:07 2017 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln671] Master 192.168.222.156:3306 from which slave 192.168.222.157(192.168.222.157:330
6) replicates is not defined in the configuration file!
Thu Feb 2 23:53:07 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Thu Feb 2 23:53:07 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Thu Feb 2 23:53:07 2017 - [info] Got exit code 1 (Not master dead).
Thu Feb 2 23:54:15 2017 - [info] MHA::MasterMonitor version 0.57.
Thu Feb 2 23:54:16 2017 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln671] Master 192.168.222.156:3306 from which slave 192.168.222.157(192.168.222.157:330
6) replicates is not defined in the configuration file!
Thu Feb 2 23:54:16 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Thu Feb 2 23:54:16 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Thu Feb 2 23:54:16 2017 - [info] Got exit code 1 (Not master dead).
Thu Feb 2 23:58:13 2017 - [info] MHA::MasterMonitor version 0.57.
Thu Feb 2 23:58:14 2017 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln671] Master 192.168.222.156:3306 from which slave 192.168.222.157(192.168.222.157:330
6) replicates is not defined in the configuration file!
Thu Feb 2 23:58:14 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Thu Feb 2 23:58:14 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Thu Feb 2 23:58:14 2017 - [info] Got exit code 1 (Not master dead).
Thu Feb 2 23:59:48 2017 - [info] MHA::MasterMonitor version 0.57.
Thu Feb 2 23:59:49 2017 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln671] Master 192.168.222.156:3306 from which slave 192.168.222.157(192.168.222.157:330
6) replicates is not defined in the configuration file!
Thu Feb 2 23:59:49 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Thu Feb 2 23:59:49 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Thu Feb 2 23:59:49 2017 - [info] Got exit code 1 (Not master dead).
Fri Feb 3 00:04:10 2017 - [info] MHA::MasterMonitor version 0.57.
Fri Feb 3 00:04:11 2017 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln671] Master 192.168.222.156:3306 from which slave 192.168.222.157(192.168.222.157:330
6) replicates is not defined in the configuration file!
Fri Feb 3 00:04:11 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Fri Feb 3 00:04:11 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Fri Feb 3 00:04:11 2017 - [info] Got exit code 1 (Not master dead).
Fri Feb 3 00:09:04 2017 - [info] MHA::MasterMonitor version 0.57.
Fri Feb 3 00:09:05 2017 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln671] Master 192.168.222.156:3306 from which slave 192.168.222.157(192.168.222.157:330
6) replicates is not defined in the configuration file!
Fri Feb 3 00:09:05 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Fri Feb 3 00:09:05 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Fri Feb 3 00:09:05 2017 - [info] Got exit code 1 (Not master dead).
Fri Feb 3 00:13:13 2017 - [info] MHA::MasterMonitor version 0.57.
Fri Feb 3 00:13:14 2017 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln671] Master 192.168.222.156:3306 from which slave 192.168.222.157(192.168.222.157:330
6) replicates is not defined in the configuration file!
Fri Feb 3 00:13:14 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Fri Feb 3 00:13:14 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Fri Feb 3 00:13:14 2017 - [info] Got exit code 1 (Not master dead).
Fri Feb 3 00:20:21 2017 - [info] MHA::MasterMonitor version 0.57.
Fri Feb 3 00:20:23 2017 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln671] Master 192.168.222.156:3306 from which slave 192.168.222.157(192.168.222.157:330
6) replicates is not defined in the configuration file!
Fri Feb 3 00:20:23 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/share/perl5/MHA/Master
Monitor.pm line 329.
Fri Feb 3 00:20:23 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Fri Feb 3 00:20:23 2017 - [info] Got exit code 1 (Not master dead).
Fri Feb 3 00:21:54 2017 - [info] MHA::MasterMonitor version 0.57.
Fri Feb 3 00:21:55 2017 - [info] GTID failover mode = 0
Fri Feb 3 00:21:55 2017 - [info] Dead Servers:
Fri Feb 3 00:21:55 2017 - [info] Alive Servers:
Fri Feb 3 00:21:55 2017 - [info] 192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:21:55 2017 - [info] 192.168.222.157(192.168.222.157:3306)
Fri Feb 3 00:21:55 2017 - [info] Alive Slaves:
Fri Feb 3 00:21:55 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Fri Feb 3 00:21:55 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:21:55 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Fri Feb 3 00:21:55 2017 - [info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:21:55 2017 - [info] Checking slave configurations..
Fri Feb 3 00:21:55 2017 - [info] Checking replication filtering settings..
Fri Feb 3 00:21:55 2017 - [info] binlog_do_db= , binlog_ignore_db=
Fri Feb 3 00:21:55 2017 - [info] Replication filtering check ok.
Fri Feb 3 00:21:55 2017 - [info] GTID (with auto-pos) is not supported
Fri Feb 3 00:21:55 2017 - [info] Starting SSH connection tests..
Fri Feb 3 00:21:58 2017 - [info] All SSH connection tests passed successfully.
Fri Feb 3 00:21:58 2017 - [info] Checking MHA Node version..
Fri Feb 3 00:21:58 2017 - [info] Version check ok.
Fri Feb 3 00:21:58 2017 - [info] Checking SSH publickey authentication settings on the current master..
Fri Feb 3 00:21:59 2017 - [info] HealthCheck: SSH to 192.168.222.156 is reachable.
Fri Feb 3 00:21:59 2017 - [info] Master MHA Node version is 0.57.
Fri Feb 3 00:21:59 2017 - [info] Checking recovery script configurations on 192.168.222.156(192.168.222.156:3306)..
Fri Feb 3 00:21:59 2017 - [info] Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data --output_file=
/tmp/save_binary_logs_test --manager_version=0.57 --start_file=mysql-bin.000068
Fri Feb 3 00:21:59 2017 - [info] Connecting to root@192.168.222.156(192.168.222.156:22)..
Creating /tmp if not exists.. ok.
Checking output directory is accessible or not..
ok.
Binlog found at /app/oracle/oradata2/mysql-5.5.37/data, up to mysql-bin.000068
Fri Feb 3 00:21:59 2017 - [info] Binlog setting check done.
Fri Feb 3 00:21:59 2017 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Fri Feb 3 00:21:59 2017 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='root' --slave_host=192.168.222.157 --slave_ip=192.168.222.15
7 --slave_port=3306 --workdir=/tmp --target_version=5.6.27-log --manager_version=0.57 --relay_log_info=/usr/local/mysql/data/relay-log.info --relay_dir=/usr/local/mys
ql/data/ --slave_pass=xxx
Fri Feb 3 00:21:59 2017 - [info] Connecting to root@192.168.222.157(192.168.222.157:22)..
Checking slave recovery environment settings..
Opening /usr/local/mysql/data/relay-log.info ... ok.
Relay log found at /usr/local/mysql/data, up to relay-bin.000002
Temporary relay log file is /usr/local/mysql/data/relay-bin.000002
Testing mysql connection and privileges..Warning: Using a password on the command line interface can be insecure.
done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Fri Feb 3 00:22:00 2017 - [info] Slaves settings check done.
Fri Feb 3 00:22:00 2017 - [info]
192.168.222.156(192.168.222.156:3306) (current master)
+--192.168.222.157(192.168.222.157:3306)
Fri Feb 3 00:22:00 2017 - [info] Checking master_ip_failover_script status:
Fri Feb 3 00:22:00 2017 - [info] /usr/local/bin/master_ip_failover --command=status --ssh_user=root --orig_master_host=192.168.222.156 --orig_master_ip=192.168.222.
156 --orig_master_port=3306
Fri Feb 3 00:22:00 2017 - [info] OK.
Fri Feb 3 00:22:00 2017 - [warning] shutdown_script is not defined.
Fri Feb 3 00:22:00 2017 - [info] Set master ping interval 1 seconds.
Fri Feb 3 00:22:00 2017 - [info] Set secondary check script: /usr/local/bin/masterha_secondary_check -s mysqldb2 -s oradb --user=root --master_host=oradb --master_ip=
'192.168.222.156' --master_port=3306
Fri Feb 3 00:22:00 2017 - [info] Starting ping health check on 192.168.222.156(192.168.222.156:3306)..
Fri Feb 3 00:22:00 2017 - [info] Ping(SELECT) succeeded, waiting until MySQL doesn't respond..
Fri Feb 3 00:25:06 2017 - [warning] Got error on MySQL select ping: 2013 (Lost connection to MySQL server during query)
Fri Feb 3 00:25:06 2017 - [info] Executing secondary network check script: /usr/local/bin/masterha_secondary_check -s mysqldb2 -s oradb --user=root --master_host=orad
b --master_ip='192.168.222.156' --master_port=3306 --user=root --master_host=192.168.222.156 --master_ip=192.168.222.156 --master_port=3306 --master_user=root --ma
ster_password=123456 --ping_type=SELECT
Fri Feb 3 00:25:06 2017 - [info] Executing SSH check script: save_binary_logs --command=test --start_pos=4 --binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data --outpu
t_file=/tmp/save_binary_logs_test --manager_version=0.57 --binlog_prefix=mysql-bin
Fri Feb 3 00:25:07 2017 - [warning] Got error on MySQL connect: 2003 (Can't connect to MySQL server on '192.168.222.156' (111))
Fri Feb 3 00:25:07 2017 - [warning] Connection failed 2 time(s)..
Monitoring server mysqldb2 is reachable, Master is not reachable from mysqldb2. OK.
Fri Feb 3 00:25:08 2017 - [warning] Got error on MySQL connect: 2003 (Can't connect to MySQL server on '192.168.222.156' (111))
Fri Feb 3 00:25:08 2017 - [warning] Connection failed 3 time(s)..
Monitoring server oradb is reachable, Master is not reachable from oradb. OK.
Fri Feb 3 00:25:08 2017 - [info] Master is not reachable from all other monitoring servers. Failover should start.
Fri Feb 3 00:25:08 2017 - [info] HealthCheck: SSH to 192.168.222.156 is reachable.
Fri Feb 3 00:25:09 2017 - [warning] Got error on MySQL connect: 2003 (Can't connect to MySQL server on '192.168.222.156' (111))
Fri Feb 3 00:25:09 2017 - [warning] Connection failed 4 time(s)..
Fri Feb 3 00:25:09 2017 - [warning] Master is not reachable from health checker!
Fri Feb 3 00:25:09 2017 - [warning] Master 192.168.222.156(192.168.222.156:3306) is not reachable!
Fri Feb 3 00:25:09 2017 - [warning] SSH is reachable.
Fri Feb 3 00:25:09 2017 - [info] Connecting to a master server failed. Reading configuration file /etc/masterha_default.cnf and /etc/masterha/app1.cnf again, and tryi
ng to connect to all servers to check server status..
Fri Feb 3 00:25:09 2017 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Fri Feb 3 00:25:09 2017 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Fri Feb 3 00:25:09 2017 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Fri Feb 3 00:25:10 2017 - [warning] SQL Thread is stopped(no error) on 192.168.222.157(192.168.222.157:3306)
Fri Feb 3 00:25:10 2017 - [info] GTID failover mode = 0
Fri Feb 3 00:25:10 2017 - [info] Dead Servers:
Fri Feb 3 00:25:10 2017 - [info] 192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:25:10 2017 - [info] Alive Servers:
Fri Feb 3 00:25:10 2017 - [info] 192.168.222.157(192.168.222.157:3306)
Fri Feb 3 00:25:10 2017 - [info] Alive Slaves:
Fri Feb 3 00:25:10 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Fri Feb 3 00:25:10 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:25:10 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Fri Feb 3 00:25:10 2017 - [info] Checking slave configurations..
Fri Feb 3 00:25:10 2017 - [info] Checking replication filtering settings..
Fri Feb 3 00:25:10 2017 - [info] Replication filtering check ok.
Fri Feb 3 00:25:10 2017 - [info] Master is down!
Fri Feb 3 00:25:10 2017 - [info] Terminating monitoring script.
Fri Feb 3 00:25:10 2017 - [info] Got exit code 20 (Master dead).
Fri Feb 3 00:25:10 2017 - [info] MHA::MasterFailover version 0.57.
Fri Feb 3 00:25:10 2017 - [info] Starting master failover.
Fri Feb 3 00:25:10 2017 - [info]
Fri Feb 3 00:25:10 2017 - [info] * Phase 1: Configuration Check Phase..
Fri Feb 3 00:25:10 2017 - [info]
Fri Feb 3 00:25:11 2017 - [warning] SQL Thread is stopped(no error) on 192.168.222.157(192.168.222.157:3306)
Fri Feb 3 00:25:11 2017 - [info] GTID failover mode = 0
Fri Feb 3 00:25:11 2017 - [info] Dead Servers:
Fri Feb 3 00:25:11 2017 - [info] 192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:25:11 2017 - [info] Checking master reachability via MySQL(double check)...
Fri Feb 3 00:25:11 2017 - [info] ok.
Fri Feb 3 00:25:11 2017 - [info] Alive Servers:
Fri Feb 3 00:25:11 2017 - [info] 192.168.222.157(192.168.222.157:3306)
Fri Feb 3 00:25:11 2017 - [info] Alive Slaves:
Fri Feb 3 00:25:11 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Fri Feb 3 00:25:11 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:25:11 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Fri Feb 3 00:25:11 2017 - [info] Starting SQL thread on 192.168.222.157(192.168.222.157:3306) ..
Fri Feb 3 00:25:11 2017 - [info] done.
Fri Feb 3 00:25:11 2017 - [info] Starting Non-GTID based failover.
Fri Feb 3 00:25:11 2017 - [info]
Fri Feb 3 00:25:11 2017 - [info] ** Phase 1: Configuration Check Phase completed.
Fri Feb 3 00:25:11 2017 - [info]
Fri Feb 3 00:25:11 2017 - [info] * Phase 2: Dead Master Shutdown Phase..
Fri Feb 3 00:25:11 2017 - [info]
Fri Feb 3 00:25:11 2017 - [info] Forcing shutdown so that applications never connect to the current master..
Fri Feb 3 00:25:11 2017 - [info] Executing master IP deactivation script:
Fri Feb 3 00:25:11 2017 - [info] /usr/local/bin/master_ip_failover --orig_master_host=192.168.222.156 --orig_master_ip=192.168.222.156 --orig_master_port=3306 --com
mand=stopssh --ssh_user=root
Fri Feb 3 00:25:11 2017 - [info] done.
Fri Feb 3 00:25:11 2017 - [warning] shutdown_script is not set. Skipping explicit shutting down of the dead master.
Fri Feb 3 00:25:11 2017 - [info] * Phase 2: Dead Master Shutdown Phase completed.
Fri Feb 3 00:25:11 2017 - [info]
Fri Feb 3 00:25:11 2017 - [info] * Phase 3: Master Recovery Phase..
Fri Feb 3 00:25:11 2017 - [info]
Fri Feb 3 00:25:11 2017 - [info] * Phase 3.1: Getting Latest Slaves Phase..
Fri Feb 3 00:25:11 2017 - [info]
Fri Feb 3 00:25:11 2017 - [info] The latest binary log file/position on all slaves is mysql-bin.000068:530
Fri Feb 3 00:25:11 2017 - [info] Latest slaves (Slaves that received relay log files to the latest):
Fri Feb 3 00:25:11 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Fri Feb 3 00:25:11 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:25:11 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Fri Feb 3 00:25:11 2017 - [info] The oldest binary log file/position on all slaves is mysql-bin.000068:530
Fri Feb 3 00:25:11 2017 - [info] Oldest slaves:
Fri Feb 3 00:25:11 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Fri Feb 3 00:25:11 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:25:11 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Fri Feb 3 00:25:11 2017 - [info]
Fri Feb 3 00:25:11 2017 - [info] * Phase 3.2: Saving Dead Master's Binlog Phase..
Fri Feb 3 00:25:11 2017 - [info]
Fri Feb 3 00:25:12 2017 - [info] Fetching dead master's binary logs..
Fri Feb 3 00:25:12 2017 - [info] Executing command on the dead master 192.168.222.156(192.168.222.156:3306): save_binary_logs --command=save --start_file=mysql-bin.00
0068 --start_pos=530 --binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data --output_file=/tmp/saved_master_binlog_from_192.168.222.156_3306_20170203002510.binlog --hand
le_raw_binlog=1 --disable_log_bin=0 --manager_version=0.57
Creating /tmp if not exists.. ok.
Concat binary/relay logs from mysql-bin.000068 pos 530 to mysql-bin.000068 EOF into /tmp/saved_master_binlog_from_192.168.222.156_3306_20170203002510.binlog ..
Binlog Checksum enabled
Dumping binlog format description event, from position 0 to 120.. ok.
Dumping effective binlog data from /app/oracle/oradata2/mysql-5.5.37/data/mysql-bin.000068 position 530 to tail(61261921).. ok.
Binlog Checksum enabled
Concat succeeded.
Fri Feb 3 00:25:16 2017 - [info] scp from root@192.168.222.156:/tmp/saved_master_binlog_from_192.168.222.156_3306_20170203002510.binlog to local:/masterha/app1/saved_
master_binlog_from_192.168.222.156_3306_20170203002510.binlog succeeded.
Fri Feb 3 00:25:17 2017 - [info] HealthCheck: SSH to 192.168.222.157 is reachable.
Fri Feb 3 00:25:17 2017 - [info]
Fri Feb 3 00:25:17 2017 - [info] * Phase 3.3: Determining New Master Phase..
Fri Feb 3 00:25:17 2017 - [info]
Fri Feb 3 00:25:17 2017 - [info] Finding the latest slave that has all relay logs for recovering other slaves..
Fri Feb 3 00:25:17 2017 - [info] All slaves received relay logs to the same position. No need to resync each other.
Fri Feb 3 00:25:17 2017 - [info] Searching new master from slaves..
Fri Feb 3 00:25:17 2017 - [info] Candidate masters from the configuration file:
Fri Feb 3 00:25:17 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Fri Feb 3 00:25:17 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Fri Feb 3 00:25:17 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Fri Feb 3 00:25:17 2017 - [info] Non-candidate masters:
Fri Feb 3 00:25:17 2017 - [info] Searching from candidate_master slaves which have received the latest relay log events..
Fri Feb 3 00:25:17 2017 - [info] New master is 192.168.222.157(192.168.222.157:3306)
Fri Feb 3 00:25:17 2017 - [info] Starting master failover..
Fri Feb 3 00:25:17 2017 - [info]
From:
192.168.222.156(192.168.222.156:3306) (current master)
+--192.168.222.157(192.168.222.157:3306)
To:
192.168.222.157(192.168.222.157:3306) (new master)
Fri Feb 3 00:25:17 2017 - [info]
Fri Feb 3 00:25:17 2017 - [info] * Phase 3.3: New Master Diff Log Generation Phase..
Fri Feb 3 00:25:17 2017 - [info]
Fri Feb 3 00:25:17 2017 - [info] This server has all relay logs. No need to generate diff files from the latest slave.
Fri Feb 3 00:25:17 2017 - [info] Sending binlog..
Fri Feb 3 00:25:20 2017 - [info] scp from local:/masterha/app1/saved_master_binlog_from_192.168.222.156_3306_20170203002510.binlog to root@192.168.222.157:/tmp/saved_
master_binlog_from_192.168.222.156_3306_20170203002510.binlog succeeded.
Fri Feb 3 00:25:20 2017 - [info]
Fri Feb 3 00:25:20 2017 - [info] * Phase 3.4: Master Log Apply Phase..
Fri Feb 3 00:25:20 2017 - [info]
Fri Feb 3 00:25:20 2017 - [info] *NOTICE: If any error happens from this phase, manual recovery is needed.
Fri Feb 3 00:25:20 2017 - [info] Starting recovery on 192.168.222.157(192.168.222.157:3306)..
Fri Feb 3 00:25:20 2017 - [info] Generating diffs succeeded.
Fri Feb 3 00:25:20 2017 - [info] Waiting until all relay logs are applied.
Fri Feb 3 00:25:20 2017 - [info] done.
Fri Feb 3 00:25:20 2017 - [info] Getting slave status..
Fri Feb 3 00:25:20 2017 - [info] This slave(192.168.222.157)'s Exec_Master_Log_Pos equals to Read_Master_Log_Pos(mysql-bin.000068:530). No need to recover from Exec_M
aster_Log_Pos.
Fri Feb 3 00:25:20 2017 - [info] Connecting to the target slave host 192.168.222.157, running recover script..
Fri Feb 3 00:25:20 2017 - [info] Executing command: apply_diff_relay_logs --command=apply --slave_user='root' --slave_host=192.168.222.157 --slave_ip=192.168.222.157
--slave_port=3306 --apply_files=/tmp/saved_master_binlog_from_192.168.222.156_3306_20170203002510.binlog --workdir=/tmp --target_version=5.6.27-log --timestamp=201702
03002510 --handle_raw_binlog=1 --disable_log_bin=0 --manager_version=0.57 --slave_pass=xxx
Fri Feb 3 00:26:48 2017 - [info]
MySQL client version is 5.6.27. Using --binary-mode.
Applying differential binary/relay log files /tmp/saved_master_binlog_from_192.168.222.156_3306_20170203002510.binlog on 192.168.222.157:3306. This may take long time.
..
Applying log files succeeded.
Fri Feb 3 00:26:48 2017 - [info] All relay logs were successfully applied.
Fri Feb 3 00:26:48 2017 - [info] Getting new master's binlog name and position..
Fri Feb 3 00:26:48 2017 - [info] mysql-bin.000002:196188901
Fri Feb 3 00:26:48 2017 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='192.168.222.157', MASTER_POR
T=3306, MASTER_LOG_FILE='mysql-bin.000002', MASTER_LOG_POS=196188901, MASTER_USER='repl', MASTER_PASSWORD='xxx';
Fri Feb 3 00:26:48 2017 - [info] Executing master IP activate script:
Fri Feb 3 00:26:48 2017 - [info] /usr/local/bin/master_ip_failover --command=start --ssh_user=root --orig_master_host=192.168.222.156 --orig_master_ip=192.168.222.1
56 --orig_master_port=3306 --new_master_host=192.168.222.157 --new_master_ip=192.168.222.157 --new_master_port=3306 --new_master_user='root' --new_master_password=xx
x
Set read_only=0 on the new master.
Creating app user on the new master..
sh: sshroot@192.168.222.156./lvs-admin stop: No such file or directory
Fri Feb 3 00:26:49 2017 - [info] OK.
Fri Feb 3 00:26:49 2017 - [info] ** Finished master recovery successfully.
Fri Feb 3 00:26:49 2017 - [info] * Phase 3: Master Recovery Phase completed.
Fri Feb 3 00:26:49 2017 - [info]
Fri Feb 3 00:26:49 2017 - [info] * Phase 4: Slaves Recovery Phase..
Fri Feb 3 00:26:49 2017 - [info]
Fri Feb 3 00:26:49 2017 - [info] * Phase 4.1: Starting Parallel Slave Diff Log Generation Phase..
Fri Feb 3 00:26:49 2017 - [info]
Fri Feb 3 00:26:49 2017 - [info] Generating relay diff files from the latest slave succeeded.
Fri Feb 3 00:26:49 2017 - [info]
Fri Feb 3 00:26:49 2017 - [info] * Phase 4.2: Starting Parallel Slave Log Apply Phase..
Fri Feb 3 00:26:49 2017 - [info]
Fri Feb 3 00:26:49 2017 - [info] All new slave servers recovered successfully.
Fri Feb 3 00:26:49 2017 - [info]
Fri Feb 3 00:26:49 2017 - [info] * Phase 5: New master cleanup phase..
Fri Feb 3 00:26:49 2017 - [info]
Fri Feb 3 00:26:49 2017 - [info] Resetting slave info on the new master..
Fri Feb 3 00:26:49 2017 - [info] 192.168.222.157: Resetting slave info succeeded.
Fri Feb 3 00:26:49 2017 - [info] Master failover to 192.168.222.157(192.168.222.157:3306) completed successfully.
Fri Feb 3 00:26:49 2017 - [info] Deleted server1 entry from /etc/masterha/app1.cnf .
Fri Feb 3 00:26:49 2017 - [info]
----- Failover Report -----
app1: MySQL Master failover 192.168.222.156(192.168.222.156:3306) to 192.168.222.157(192.168.222.157:3306) succeeded
Master 192.168.222.156(192.168.222.156:3306) is down!
Check MHA Manager logs at mysqldb2:/masterha/app1/app1.log for details.
Started automated(non-interactive) failover.
Invalidated master IP address on 192.168.222.156(192.168.222.156:3306)
The latest slave 192.168.222.157(192.168.222.157:3306) has all relay logs for recovery.
Selected 192.168.222.157(192.168.222.157:3306) as a new master.
192.168.222.157(192.168.222.157:3306): OK: Applying all logs succeeded.
192.168.222.157(192.168.222.157:3306): OK: Activated master IP address.
Generating relay diff files from the latest slave succeeded.
192.168.222.157(192.168.222.157:3306): Resetting slave info succeeded.
Master failover to 192.168.222.157(192.168.222.157:3306) completed successfully.
三、部署 MHA遇到的问题
问题：
[root@localhost mha]# ./install.sh
--> Working on install
Fetching http://www.cpan.org/authors/id/D/DA/DAGOLDEN/install-0.01.tar.gz ... OK
Configuring install-0.01 ... OK
Building and testing install-0.01 ... FAIL
! Installing install failed. See /root/.cpanm/work/1485934105.3288/build.log for details. Retry with --force to force install it.
--> Working on DBD::mysql
Fetching http://www.cpan.org/authors/id/M/MI/MICHIELB/DBD-mysql-4.041.tar.gz ... OK
==> Found dependencies: DBI
--> Working on DBI
Fetching http://www.cpan.org/authors/id/T/TI/TIMB/DBI-1.636.tar.gz ... OK
Configuring DBI-1.636 ... N/A
! Configure failed for DBI-1.636. See /root/.cpanm/work/1485934125.3353/build.log for details.
! Installing the dependencies failed: Module 'DBI' is not installed
! Bailing out the installation for DBD-mysql-4.041.
问题：
/usr/bin/ld: cannot find -lz
usr/bin/ld: cannot find -lz
collect2: ld returned 1 exit status
make: *** [blib/arch/auto/DBD/mysql/mysql.so] Error 1
-> FAIL Installing DBD::mysql failed. See /root/.cpanm/work/1485938925.8246/build.log for details. Retry with --force to force install it.
2 distributions installed
处理：
yum -y install *zlib-dev*
[root@localhost mha]# ./install.sh
install is up to date. (0.01)
--> Working on DBD::mysql
Fetching http://www.cpan.org/authors/id/M/MI/MICHIELB/DBD-mysql-4.041.tar.gz ... OK
Configuring DBD-mysql-4.041 ... OK
Building and testing DBD-mysql-4.041 ... OK
Successfully installed DBD-mysql-4.041 (upgraded from 4.013)
1 distribution installed
[root@localhost mha4mysql-node-0.53]# perl Makefile.PL
*** Module::AutoInstall version 1.03
*** Checking for Perl dependencies...
[Core Features]
- DBI ...loaded. (1.609)
- DBD::mysql ...loaded. (4.041)
*** Module::AutoInstall configuration finished.
Checking if your kit is complete...
Looks good
Generating a Unix-style Makefile
Writing Makefile for mha4mysql::node
Writing MYMETA.yml and MYMETA.json
[root@localhost mha4mysql-node-0.53]# make
cp lib/MHA/BinlogManager.pm blib/lib/MHA/BinlogManager.pm
cp lib/MHA/BinlogPosFindManager.pm blib/lib/MHA/BinlogPosFindManager.pm
cp lib/MHA/BinlogPosFinderXid.pm blib/lib/MHA/BinlogPosFinderXid.pm
cp lib/MHA/BinlogHeaderParser.pm blib/lib/MHA/BinlogHeaderParser.pm
cp lib/MHA/BinlogPosFinder.pm blib/lib/MHA/BinlogPosFinder.pm
cp lib/MHA/BinlogPosFinderElp.pm blib/lib/MHA/BinlogPosFinderElp.pm
cp lib/MHA/NodeUtil.pm blib/lib/MHA/NodeUtil.pm
cp lib/MHA/SlaveUtil.pm blib/lib/MHA/SlaveUtil.pm
cp lib/MHA/NodeConst.pm blib/lib/MHA/NodeConst.pm
cp bin/apply_diff_relay_logs blib/script/apply_diff_relay_logs
"/usr/bin/perl" "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/apply_diff_relay_logs
cp bin/filter_mysqlbinlog blib/script/filter_mysqlbinlog
"/usr/bin/perl" "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/filter_mysqlbinlog
cp bin/purge_relay_logs blib/script/purge_relay_logs
"/usr/bin/perl" "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/purge_relay_logs
cp bin/save_binary_logs blib/script/save_binary_logs
"/usr/bin/perl" "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/save_binary_logs
Manifying 4 pod documents
[root@localhost mha4mysql-node-0.53]# make install
Manifying 4 pod documents
Installing /usr/local/share/perl5/MHA/NodeUtil.pm
Installing /usr/local/share/perl5/MHA/BinlogPosFindManager.pm
Installing /usr/local/share/perl5/MHA/NodeConst.pm
Installing /usr/local/share/perl5/MHA/BinlogPosFinder.pm
Installing /usr/local/share/perl5/MHA/BinlogHeaderParser.pm
Installing /usr/local/share/perl5/MHA/BinlogPosFinderXid.pm
Installing /usr/local/share/perl5/MHA/BinlogPosFinderElp.pm
Installing /usr/local/share/perl5/MHA/BinlogManager.pm
Installing /usr/local/share/perl5/MHA/SlaveUtil.pm
Installing /usr/local/share/man/man1/purge_relay_logs.1
Installing /usr/local/share/man/man1/filter_mysqlbinlog.1
Installing /usr/local/share/man/man1/save_binary_logs.1
Installing /usr/local/share/man/man1/apply_diff_relay_logs.1
Installing /usr/local/bin/apply_diff_relay_logs
Installing /usr/local/bin/save_binary_logs
Installing /usr/local/bin/filter_mysqlbinlog
Installing /usr/local/bin/purge_relay_logs
Appending installation info to /usr/lib64/perl5/perllocal.pod
问题10：从 https://github.com/yoshinorim/mha4mysql-node 下载 mha4mysql-node 安装
[root@mysqldb2 mha4mysql-node-master]# perl Makefile.PL
Can't locate inc/Module/Install.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at Makefile.PL line 1.
BEGIN failed--compilation aborted at Makefile.PL line 1.
处理：
--查看 Makefile.PL 第一行的内容
[root@mysqldb2 mha4mysql-node-master]# vi Makefile.PL
use inc::Module::Install;
--安装模块
cpanm inc::Module::Install
[root@mysqldb2 mha4mysql-node-master]# perl Makefile.PL
include /root/olm/mha/mha4mysql-node-master/inc/Module/Install.pm
include inc/Module/Install/Metadata.pm
include inc/Module/Install/Base.pm
include inc/Module/Install/Makefile.pm
include inc/Module/Install/Scripts.pm
include inc/Module/Install/AutoInstall.pm
include inc/Module/Install/Include.pm
include inc/Module/AutoInstall.pm
*** Module::AutoInstall version 1.17
*** Checking for Perl dependencies...
[Core Features]
- DBI ...loaded. (1.609)
- DBD::mysql ...loaded. (4.041)
*** Module::AutoInstall configuration finished.
include inc/Module/Install/WriteAll.pm
include inc/Module/Install/Win32.pm
include inc/Module/Install/Can.pm
include inc/Module/Install/Fetch.pm
Checking if your kit is complete...
Warning: the following files are missing in your kit:
META.yml
Please inform the author.
Generating a Unix-style Makefile
Writing Makefile for mha4mysql::node
Writing MYMETA.yml and MYMETA.json
Writing META.yml
[root@mysqldb2 mha4mysql-node-master]# make
cp lib/MHA/BinlogManager.pm blib/lib/MHA/BinlogManager.pm
cp lib/MHA/BinlogPosFindManager.pm blib/lib/MHA/BinlogPosFindManager.pm
cp lib/MHA/BinlogPosFinderXid.pm blib/lib/MHA/BinlogPosFinderXid.pm
cp lib/MHA/BinlogHeaderParser.pm blib/lib/MHA/BinlogHeaderParser.pm
cp lib/MHA/BinlogPosFinder.pm blib/lib/MHA/BinlogPosFinder.pm
cp lib/MHA/BinlogPosFinderElp.pm blib/lib/MHA/BinlogPosFinderElp.pm
cp lib/MHA/NodeUtil.pm blib/lib/MHA/NodeUtil.pm
cp lib/MHA/SlaveUtil.pm blib/lib/MHA/SlaveUtil.pm
cp lib/MHA/NodeConst.pm blib/lib/MHA/NodeConst.pm
cp bin/apply_diff_relay_logs blib/script/apply_diff_relay_logs
"/usr/bin/perl" "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/apply_diff_relay_logs
cp bin/filter_mysqlbinlog blib/script/filter_mysqlbinlog
"/usr/bin/perl" "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/filter_mysqlbinlog
cp bin/purge_relay_logs blib/script/purge_relay_logs
"/usr/bin/perl" "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/purge_relay_logs
cp bin/save_binary_logs blib/script/save_binary_logs
"/usr/bin/perl" "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/save_binary_logs
Manifying 4 pod documents
[root@mysqldb2 mha4mysql-node-master]# make install
Manifying 4 pod documents
Installing /usr/local/share/perl5/MHA/NodeUtil.pm
Installing /usr/local/share/perl5/MHA/NodeConst.pm
Installing /usr/local/share/perl5/MHA/BinlogHeaderParser.pm
Installing /usr/local/share/perl5/MHA/BinlogManager.pm
Installing /usr/local/share/perl5/MHA/SlaveUtil.pm
Installing /usr/local/share/man/man1/purge_relay_logs.1
Installing /usr/local/share/man/man1/filter_mysqlbinlog.1
Installing /usr/local/share/man/man1/save_binary_logs.1
Installing /usr/local/share/man/man1/apply_diff_relay_logs.1
Installing /usr/local/bin/apply_diff_relay_logs
Installing /usr/local/bin/save_binary_logs
Installing /usr/local/bin/purge_relay_logs
Appending installation info to /usr/lib64/perl5/perllocal.pod
问题11：处理完问题10后
[root@mysqldb2 debian]# /usr/local/bin/masterha_check_ssh --conf=/etc/masterha/app1.cnf
"NI_NUMERICHOST" is not exported by the Socket module
"getaddrinfo" is not exported by the Socket module
"getnameinfo" is not exported by the Socket module
Can't continue after import errors at /usr/local/share/perl5/MHA/NodeUtil.pm line 29.
BEGIN failed--compilation aborted at /usr/local/share/perl5/MHA/NodeUtil.pm line 29.
Compilation failed in require at /usr/local/share/perl5/MHA/SlaveUtil.pm line 27.
BEGIN failed--compilation aborted at /usr/local/share/perl5/MHA/SlaveUtil.pm line 27.
Compilation failed in require at /usr/local/share/perl5/MHA/DBHelper.pm line 26.
BEGIN failed--compilation aborted at /usr/local/share/perl5/MHA/DBHelper.pm line 26.
Compilation failed in require at /usr/local/share/perl5/MHA/HealthCheck.pm line 30.
BEGIN failed--compilation aborted at /usr/local/share/perl5/MHA/HealthCheck.pm line 30.
Compilation failed in require at /usr/local/share/perl5/MHA/Server.pm line 28.
BEGIN failed--compilation aborted at /usr/local/share/perl5/MHA/Server.pm line 28.
Compilation failed in require at /usr/local/share/perl5/MHA/Config.pm line 29.
BEGIN failed--compilation aborted at /usr/local/share/perl5/MHA/Config.pm line 29.
Compilation failed in require at /usr/local/share/perl5/MHA/SSHCheck.pm line 32.
BEGIN failed--compilation aborted at /usr/local/share/perl5/MHA/SSHCheck.pm line 32.
Compilation failed in require at /usr/local/bin/masterha_check_ssh line 25.
BEGIN failed--compilation aborted at /usr/local/bin/masterha_check_ssh line 25.
处理：
[root@mysqldb2 mha4mysql-node-master]# cpanm Socket
--> Working on Socket
Fetching http://www.cpan.org/authors/id/P/PE/PEVANS/Socket-2.024.tar.gz ... OK
==> Found dependencies: ExtUtils::Constant
--> Working on ExtUtils::Constant
Fetching http://www.cpan.org/authors/id/N/NW/NWCLARK/ExtUtils-Constant-0.23.tar.gz ... OK
Configuring ExtUtils-Constant-0.16 ... OK
Building and testing ExtUtils-Constant-0.23 ... OK
Successfully installed ExtUtils-Constant-0.23 (upgraded from 0.22)
Configuring Socket-2.024 ... OK
Building and testing Socket-2.024 ... OK
Successfully installed Socket-2.024 (upgraded from 1.82)
2 distributions installed
[root@mysqldb2 mha4mysql-manager-master]# /usr/local/bin/masterha_check_ssh --conf=/etc/masterha/app1.cnf
Wed Feb 1 19:41:34 2017 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Wed Feb 1 19:41:34 2017 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Wed Feb 1 19:41:34 2017 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Wed Feb 1 19:41:34 2017 - [info] Starting SSH connection tests..
Wed Feb 1 19:41:51 2017 - [debug]
Wed Feb 1 19:41:35 2017 - [debug] Connecting via SSH from root@192.168.222.157(192.168.222.157:22) to root@192.168.222.156(192.168.222.156:22)..
Wed Feb 1 19:41:50 2017 - [debug] ok.
Wed Feb 1 19:41:55 2017 - [debug]
Wed Feb 1 19:41:34 2017 - [debug] Connecting via SSH from root@192.168.222.156(192.168.222.156:22) to root@192.168.222.157(192.168.222.157:22)..
Wed Feb 1 19:41:55 2017 - [debug] ok.
Wed Feb 1 19:41:55 2017 - [info] All SSH connection tests passed successfully.
问题12：
[root@mysqldb2 mha]# masterha_check_repl --conf=/etc/masterha/app1.cnf
Wed Feb 1 20:04:52 2017 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Wed Feb 1 20:04:52 2017 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:04:52 2017 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:04:52 2017 - [info] MHA::MasterMonitor version 0.57.
Creating directory /masterha/app1.. done.
Wed Feb 1 20:04:57 2017 - [info] GTID failover mode = 0
Wed Feb 1 20:04:57 2017 - [info] Dead Servers:
Wed Feb 1 20:04:57 2017 - [info] Alive Servers:
Wed Feb 1 20:04:57 2017 - [info] 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:04:57 2017 - [info] 192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:04:57 2017 - [info] Alive Slaves:
Wed Feb 1 20:04:57 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27 (oldest major version between slaves) log-bin:disabled
Wed Feb 1 20:04:57 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:04:57 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Wed Feb 1 20:04:57 2017 - [info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:04:57 2017 - [info] Checking slave configurations..
Wed Feb 1 20:04:57 2017 - [warning] log-bin is not set on slave 192.168.222.157(192.168.222.157:3306). This host cannot be a master.
Wed Feb 1 20:04:57 2017 - [info] Checking replication filtering settings..
Wed Feb 1 20:04:57 2017 - [info] binlog_do_db= , binlog_ignore_db=
Wed Feb 1 20:04:57 2017 - [info] Replication filtering check ok.
Wed Feb 1 20:04:57 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln364] None of slaves can be master. Check failover configuration file or log-bin settings in my.cnf
Wed Feb 1 20:04:57 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/bin/masterha_check_repl line 48.
Wed Feb 1 20:04:57 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Wed Feb 1 20:04:57 2017 - [info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!
处理：
vi /usr/local/mysql/my.cnf
log_bin=/usr/local/mysql/data/mysql-bin
#relay_log=/usr/local/mysql/data/relay-bin
skip_slave_start
/usr/local/mysql/bin/mysqld_safe --defaults-file=/usr/local/mysql/my.cnf --user=mysql &
mysql> start slave;
[root@mysqldb2 mysql]# masterha_check_repl --conf=/etc/masterha/app1.cnf
Wed Feb 1 20:16:43 2017 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Wed Feb 1 20:16:43 2017 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:16:43 2017 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:16:43 2017 - [info] MHA::MasterMonitor version 0.57.
Wed Feb 1 20:16:44 2017 - [info] GTID failover mode = 0
Wed Feb 1 20:16:44 2017 - [info] Dead Servers:
Wed Feb 1 20:16:44 2017 - [info] Alive Servers:
Wed Feb 1 20:16:44 2017 - [info] 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:16:44 2017 - [info] 192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:16:44 2017 - [info] Alive Slaves:
Wed Feb 1 20:16:44 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Wed Feb 1 20:16:44 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:16:44 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Wed Feb 1 20:16:44 2017 - [info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:16:44 2017 - [info] Checking slave configurations..
Wed Feb 1 20:16:44 2017 - [info] read_only=1 is not set on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:16:44 2017 - [warning] relay_log_purge=0 is not set on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:16:44 2017 - [info] Checking replication filtering settings..
Wed Feb 1 20:16:44 2017 - [info] binlog_do_db= , binlog_ignore_db=
Wed Feb 1 20:16:44 2017 - [info] Replication filtering check ok.
Wed Feb 1 20:16:44 2017 - [info] GTID (with auto-pos) is not supported
Wed Feb 1 20:16:44 2017 - [info] Starting SSH connection tests..
Wed Feb 1 20:16:50 2017 - [info] All SSH connection tests passed successfully.
Wed Feb 1 20:16:50 2017 - [info] Checking MHA Node version..
Wed Feb 1 20:16:51 2017 - [info] Version check ok.
Wed Feb 1 20:16:51 2017 - [info] Checking SSH publickey authentication settings on the current master..
Wed Feb 1 20:16:51 2017 - [info] HealthCheck: SSH to 192.168.222.156 is reachable.
Wed Feb 1 20:16:51 2017 - [info] Master MHA Node version is 0.57.
Wed Feb 1 20:16:51 2017 - [info] Checking recovery script configurations on 192.168.222.156(192.168.222.156:3306)..
Wed Feb 1 20:16:51 2017 - [info] Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/home/binlog --output_file=/tmp/save_binary_logs_test --manager_version=0.57 --start_file=mysql-bin.000058
Wed Feb 1 20:16:51 2017 - [info] Connecting to root@192.168.222.156(192.168.222.156:22)..
Failed to save binary log: Binlog not found from /home/binlog! If you got this error at MHA Manager, please set "master_binlog_dir=/path/to/binlog_directory_of_the_master" correctly in the MHA Manager's configuration file and try again.
at /usr/local/bin/save_binary_logs line 123.
eval {...} called at /usr/local/bin/save_binary_logs line 70
main::main() called at /usr/local/bin/save_binary_logs line 66
Wed Feb 1 20:16:51 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln161] Binlog setting check failed!
Wed Feb 1 20:16:51 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln408] Master configuration failed.
Wed Feb 1 20:16:51 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/bin/masterha_check_repl line 48.
Wed Feb 1 20:16:51 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Wed Feb 1 20:16:51 2017 - [info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!
问题12已经处理好，但出现了新的问题，命名为问题13
问题13：
[root@mysqldb2 mysql]# masterha_check_repl --conf=/etc/masterha/app1.cnf
Wed Feb 1 20:16:43 2017 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Wed Feb 1 20:16:43 2017 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:16:43 2017 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:16:43 2017 - [info] MHA::MasterMonitor version 0.57.
Wed Feb 1 20:16:44 2017 - [info] GTID failover mode = 0
Wed Feb 1 20:16:44 2017 - [info] Dead Servers:
Wed Feb 1 20:16:44 2017 - [info] Alive Servers:
Wed Feb 1 20:16:44 2017 - [info] 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:16:44 2017 - [info] 192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:16:44 2017 - [info] Alive Slaves:
Wed Feb 1 20:16:44 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Wed Feb 1 20:16:44 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:16:44 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Wed Feb 1 20:16:44 2017 - [info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:16:44 2017 - [info] Checking slave configurations..
Wed Feb 1 20:16:44 2017 - [info] read_only=1 is not set on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:16:44 2017 - [warning] relay_log_purge=0 is not set on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:16:44 2017 - [info] Checking replication filtering settings..
Wed Feb 1 20:16:44 2017 - [info] binlog_do_db= , binlog_ignore_db=
Wed Feb 1 20:16:44 2017 - [info] Replication filtering check ok.
Wed Feb 1 20:16:44 2017 - [info] GTID (with auto-pos) is not supported
Wed Feb 1 20:16:44 2017 - [info] Starting SSH connection tests..
Wed Feb 1 20:16:50 2017 - [info] All SSH connection tests passed successfully.
Wed Feb 1 20:16:50 2017 - [info] Checking MHA Node version..
Wed Feb 1 20:16:51 2017 - [info] Version check ok.
Wed Feb 1 20:16:51 2017 - [info] Checking SSH publickey authentication settings on the current master..
Wed Feb 1 20:16:51 2017 - [info] HealthCheck: SSH to 192.168.222.156 is reachable.
Wed Feb 1 20:16:51 2017 - [info] Master MHA Node version is 0.57.
Wed Feb 1 20:16:51 2017 - [info] Checking recovery script configurations on 192.168.222.156(192.168.222.156:3306)..
Wed Feb 1 20:16:51 2017 - [info] Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/home/binlog --output_file=/tmp/save_binary_logs_test --manager_version=0.57 --start_file=mysql-bin.000058
Wed Feb 1 20:16:51 2017 - [info] Connecting to root@192.168.222.156(192.168.222.156:22)..
Failed to save binary log: Binlog not found from /home/binlog! If you got this error at MHA Manager, please set "master_binlog_dir=/path/to/binlog_directory_of_the_master" correctly in the MHA Manager's configuration file and try again.
at /usr/local/bin/save_binary_logs line 123.
eval {...} called at /usr/local/bin/save_binary_logs line 70
main::main() called at /usr/local/bin/save_binary_logs line 66
Wed Feb 1 20:16:51 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln161] Binlog setting check failed!
Wed Feb 1 20:16:51 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln408] Master configuration failed.
Wed Feb 1 20:16:51 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/bin/masterha_check_repl line 48.
Wed Feb 1 20:16:51 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Wed Feb 1 20:16:51 2017 - [info] Got exit code 1 (Not master dead).
处理：
修改 MHA配置文件中主库二进制日志的路径
[server default]
manager_log=/masterha/app1/app1.log
manager_workdir=/masterha/app1
#master_binlog_dir=/home/binlog 修改前的配置
master_binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data
master_ip_failover_script=/usr/local/bin/master_ip_failover
master_ip_online_change_script=/usr/local/bin/master_ip_online_change
password=123456
ping_interval=1
remote_workdir=/tmp
repl_password=123456
repl_user=repl
report_script=/usr/local/bin/send_report
secondary_check_script=/usr/local/bin/masterha_secondary_check -s '192.168.222.157' -s '192.168.222.156' --user=root --master_ip='192.168.222.156' --master_port=3306
shutdown_script=""
ssh_user=root
user=root
[server1]
hostname=192.168.222.156
port=3306
[server2]
hostname=192.168.222.157
port=3306
candidate_master=1
check_repl_delay=0
问题14：Can't exec "mysqlbinlog": No such file or directory at /usr/local/share/perl5/MHA/BinlogManager.pm line 106.
[root@mysqldb2 mysql]# masterha_check_repl --conf=/etc/masterha/app1.cnf
Wed Feb 1 20:25:05 2017 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Wed Feb 1 20:25:05 2017 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:25:05 2017 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:25:05 2017 - [info] MHA::MasterMonitor version 0.57.
Wed Feb 1 20:25:06 2017 - [info] GTID failover mode = 0
Wed Feb 1 20:25:06 2017 - [info] Dead Servers:
Wed Feb 1 20:25:06 2017 - [info] Alive Servers:
Wed Feb 1 20:25:06 2017 - [info] 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:25:06 2017 - [info] 192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:25:06 2017 - [info] Alive Slaves:
Wed Feb 1 20:25:06 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Wed Feb 1 20:25:06 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:25:06 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Wed Feb 1 20:25:06 2017 - [info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:25:06 2017 - [info] Checking slave configurations..
Wed Feb 1 20:25:06 2017 - [info] read_only=1 is not set on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:25:06 2017 - [warning] relay_log_purge=0 is not set on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:25:06 2017 - [info] Checking replication filtering settings..
Wed Feb 1 20:25:06 2017 - [info] binlog_do_db= , binlog_ignore_db=
Wed Feb 1 20:25:06 2017 - [info] Replication filtering check ok.
Wed Feb 1 20:25:06 2017 - [info] GTID (with auto-pos) is not supported
Wed Feb 1 20:25:06 2017 - [info] Starting SSH connection tests..
Wed Feb 1 20:25:13 2017 - [info] All SSH connection tests passed successfully.
Wed Feb 1 20:25:13 2017 - [info] Checking MHA Node version..
Wed Feb 1 20:25:13 2017 - [info] Version check ok.
Wed Feb 1 20:25:13 2017 - [info] Checking SSH publickey authentication settings on the current master..
Wed Feb 1 20:25:13 2017 - [info] HealthCheck: SSH to 192.168.222.156 is reachable.
Wed Feb 1 20:25:14 2017 - [info] Master MHA Node version is 0.57.
Wed Feb 1 20:25:14 2017 - [info] Checking recovery script configurations on 192.168.222.156(192.168.222.156:3306)..
Wed Feb 1 20:25:14 2017 - [info] Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data --output_file=/tmp/save_binary_logs_test --manager_version=0.57 --start_file=mysql-bin.000058
Wed Feb 1 20:25:14 2017 - [info] Connecting to root@192.168.222.156(192.168.222.156:22)..
Creating /tmp if not exists.. ok.
Checking output directory is accessible or not..
ok.
Binlog found at /app/oracle/oradata2/mysql-5.5.37/data, up to mysql-bin.000058
Wed Feb 1 20:25:14 2017 - [info] Binlog setting check done.
Wed Feb 1 20:25:14 2017 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Wed Feb 1 20:25:14 2017 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='root' --slave_host=192.168.222.157 --slave_ip=192.168.222.157 --slave_port=3306 --workdir=/tmp --target_version=5.6.27-log --manager_version=0.57 --relay_log_info=/usr/local/mysql/data/relay-log.info --relay_dir=/usr/local/mysql/data/ --slave_pass=xxx
Wed Feb 1 20:25:14 2017 - [info] Connecting to root@192.168.222.157(192.168.222.157:22)..
Can't exec "mysqlbinlog": No such file or directory at /usr/local/share/perl5/MHA/BinlogManager.pm line 106.
mysqlbinlog version command failed with rc 1:0, please verify PATH, LD_LIBRARY_PATH, and client options
at /usr/local/bin/apply_diff_relay_logs line 493.
Wed Feb 1 20:25:14 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln208] Slaves settings check failed!
Wed Feb 1 20:25:14 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln416] Slave configuration failed.
Wed Feb 1 20:25:14 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/bin/masterha_check_repl line 48.
Wed Feb 1 20:25:14 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Wed Feb 1 20:25:14 2017 - [info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!
处理：
[root@mysqldb2 mysql]# vi /etc/bashrc
#在文件的末尾添加
PATH="$PATH:/usr/local/mysql/bin"
export PATH
问题15：Wed Feb 1 20:31:04 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. Can't exec "/usr/local/bin/master_ip_failover": No such file or directory at /usr/local/share/perl5/MHA/ManagerUtil.pm line 68.
[root@mysqldb2 mysql]# masterha_check_repl --conf=/etc/masterha/app1.cnf
Wed Feb 1 20:30:56 2017 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Wed Feb 1 20:30:56 2017 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:30:56 2017 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:30:56 2017 - [info] MHA::MasterMonitor version 0.57.
Wed Feb 1 20:30:57 2017 - [info] GTID failover mode = 0
Wed Feb 1 20:30:57 2017 - [info] Dead Servers:
Wed Feb 1 20:30:57 2017 - [info] Alive Servers:
Wed Feb 1 20:30:57 2017 - [info] 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:30:57 2017 - [info] 192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:30:57 2017 - [info] Alive Slaves:
Wed Feb 1 20:30:57 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Wed Feb 1 20:30:57 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:30:57 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Wed Feb 1 20:30:57 2017 - [info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:30:57 2017 - [info] Checking slave configurations..
Wed Feb 1 20:30:57 2017 - [info] read_only=1 is not set on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:30:57 2017 - [warning] relay_log_purge=0 is not set on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:30:57 2017 - [info] Checking replication filtering settings..
Wed Feb 1 20:30:57 2017 - [info] binlog_do_db= , binlog_ignore_db=
Wed Feb 1 20:30:57 2017 - [info] Replication filtering check ok.
Wed Feb 1 20:30:57 2017 - [info] GTID (with auto-pos) is not supported
Wed Feb 1 20:30:57 2017 - [info] Starting SSH connection tests..
Wed Feb 1 20:31:03 2017 - [info] All SSH connection tests passed successfully.
Wed Feb 1 20:31:03 2017 - [info] Checking MHA Node version..
Wed Feb 1 20:31:03 2017 - [info] Version check ok.
Wed Feb 1 20:31:03 2017 - [info] Checking SSH publickey authentication settings on the current master..
Wed Feb 1 20:31:04 2017 - [info] HealthCheck: SSH to 192.168.222.156 is reachable.
Wed Feb 1 20:31:04 2017 - [info] Master MHA Node version is 0.57.
Wed Feb 1 20:31:04 2017 - [info] Checking recovery script configurations on 192.168.222.156(192.168.222.156:3306)..
Wed Feb 1 20:31:04 2017 - [info] Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data --output_file=/tmp/save_binary_logs_test --manager_version=0.57 --start_file=mysql-bin.000058
Wed Feb 1 20:31:04 2017 - [info] Connecting to root@192.168.222.156(192.168.222.156:22)..
Creating /tmp if not exists.. ok.
Checking output directory is accessible or not..
ok.
Binlog found at /app/oracle/oradata2/mysql-5.5.37/data, up to mysql-bin.000058
Wed Feb 1 20:31:04 2017 - [info] Binlog setting check done.
Wed Feb 1 20:31:04 2017 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Wed Feb 1 20:31:04 2017 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='root' --slave_host=192.168.222.157 --slave_ip=192.168.222.157 --slave_port=3306 --workdir=/tmp --target_version=5.6.27-log --manager_version=0.57 --relay_log_info=/usr/local/mysql/data/relay-log.info --relay_dir=/usr/local/mysql/data/ --slave_pass=xxx
Wed Feb 1 20:31:04 2017 - [info] Connecting to root@192.168.222.157(192.168.222.157:22)..
Checking slave recovery environment settings..
Opening /usr/local/mysql/data/relay-log.info ... ok.
Relay log found at /usr/local/mysql/data, up to mysqldb2-relay-bin.000005
Temporary relay log file is /usr/local/mysql/data/mysqldb2-relay-bin.000005
Testing mysql connection and privileges..Warning: Using a password on the command line interface can be insecure.
done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Wed Feb 1 20:31:04 2017 - [info] Slaves settings check done.
Wed Feb 1 20:31:04 2017 - [info]
192.168.222.156(192.168.222.156:3306) (current master)
+--192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:31:04 2017 - [info] Checking replication health on 192.168.222.157..
Wed Feb 1 20:31:04 2017 - [info] ok.
Wed Feb 1 20:31:04 2017 - [info] Checking master_ip_failover_script status:
Wed Feb 1 20:31:04 2017 - [info] /usr/local/bin/master_ip_failover --command=status --ssh_user=root --orig_master_host=192.168.222.156 --orig_master_ip=192.168.222.156 --orig_master_port=3306
Wed Feb 1 20:31:04 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. Can't exec "/usr/local/bin/master_ip_failover": No such file or directory at /usr/local/share/perl5/MHA/ManagerUtil.pm line 68.
Wed Feb 1 20:31:04 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Wed Feb 1 20:31:04 2017 - [info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!
Wed Feb 1 20:31:04 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln229] Failed to get master_ip_failover_script status with return code 1:0.
Wed Feb 1 20:31:04 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/bin/masterha_check_repl line 48.
Wed Feb 1 20:31:04 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Wed Feb 1 20:31:04 2017 - [info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!
处理：
[root@mysqldb2 scripts]# cp master_ip_failover /usr/local/bin
[root@mysqldb2 scripts]# pwd
/root/olm/mha/mha4mysql-manager-master/samples/scripts
问题16：
[root@mysqldb2 scripts]# masterha_check_repl --conf=/etc/masterha/app1.cnf
Wed Feb 1 20:40:41 2017 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Wed Feb 1 20:40:41 2017 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:40:41 2017 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Wed Feb 1 20:40:41 2017 - [info] MHA::MasterMonitor version 0.57.
Wed Feb 1 20:40:43 2017 - [info] GTID failover mode = 0
Wed Feb 1 20:40:43 2017 - [info] Dead Servers:
Wed Feb 1 20:40:43 2017 - [info] Alive Servers:
Wed Feb 1 20:40:43 2017 - [info] 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:40:43 2017 - [info] 192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:40:43 2017 - [info] Alive Slaves:
Wed Feb 1 20:40:43 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Wed Feb 1 20:40:43 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:40:43 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Wed Feb 1 20:40:43 2017 - [info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Wed Feb 1 20:40:43 2017 - [info] Checking slave configurations..
Wed Feb 1 20:40:43 2017 - [info] read_only=1 is not set on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:40:43 2017 - [warning] relay_log_purge=0 is not set on slave 192.168.222.157(192.168.222.157:3306).
Wed Feb 1 20:40:43 2017 - [info] Checking replication filtering settings..
Wed Feb 1 20:40:43 2017 - [info] binlog_do_db= , binlog_ignore_db=
Wed Feb 1 20:40:43 2017 - [info] Replication filtering check ok.
Wed Feb 1 20:40:43 2017 - [info] GTID (with auto-pos) is not supported
Wed Feb 1 20:40:43 2017 - [info] Starting SSH connection tests..
Wed Feb 1 20:40:59 2017 - [info] All SSH connection tests passed successfully.
Wed Feb 1 20:40:59 2017 - [info] Checking MHA Node version..
Wed Feb 1 20:40:59 2017 - [info] Version check ok.
Wed Feb 1 20:40:59 2017 - [info] Checking SSH publickey authentication settings on the current master..
Wed Feb 1 20:40:59 2017 - [info] HealthCheck: SSH to 192.168.222.156 is reachable.
Wed Feb 1 20:40:59 2017 - [info] Master MHA Node version is 0.57.
Wed Feb 1 20:40:59 2017 - [info] Checking recovery script configurations on 192.168.222.156(192.168.222.156:3306)..
Wed Feb 1 20:40:59 2017 - [info] Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data --output_file=/tmp/save_binary_logs_test --manager_version=0.57 --start_file=mysql-bin.000058
Wed Feb 1 20:40:59 2017 - [info] Connecting to root@192.168.222.156(192.168.222.156:22)..
Creating /tmp if not exists.. ok.
Checking output directory is accessible or not..
ok.
Binlog found at /app/oracle/oradata2/mysql-5.5.37/data, up to mysql-bin.000058
Wed Feb 1 20:41:00 2017 - [info] Binlog setting check done.
Wed Feb 1 20:41:00 2017 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Wed Feb 1 20:41:00 2017 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='root' --slave_host=192.168.222.157 --slave_ip=192.168.222.157 --slave_port=3306 --workdir=/tmp --target_version=5.6.27-log --manager_version=0.57 --relay_log_info=/usr/local/mysql/data/relay-log.info --relay_dir=/usr/local/mysql/data/ --slave_pass=xxx
Wed Feb 1 20:41:00 2017 - [info] Connecting to root@192.168.222.157(192.168.222.157:22)..
Checking slave recovery environment settings..
Opening /usr/local/mysql/data/relay-log.info ... ok.
Relay log found at /usr/local/mysql/data, up to mysqldb2-relay-bin.000005
Temporary relay log file is /usr/local/mysql/data/mysqldb2-relay-bin.000005
Testing mysql connection and privileges..Warning: Using a password on the command line interface can be insecure.
done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Wed Feb 1 20:41:00 2017 - [info] Slaves settings check done.
Wed Feb 1 20:41:00 2017 - [info]
192.168.222.156(192.168.222.156:3306) (current master)
+--192.168.222.157(192.168.222.157:3306)
Wed Feb 1 20:41:00 2017 - [info] Checking replication health on 192.168.222.157..
Wed Feb 1 20:41:00 2017 - [info] ok.
Wed Feb 1 20:41:00 2017 - [info] Checking master_ip_failover_script status:
Wed Feb 1 20:41:00 2017 - [info] /usr/local/bin/master_ip_failover --command=status --ssh_user=root --orig_master_host=192.168.222.156 --orig_master_ip=192.168.222.156 --orig_master_port=3306
Bareword "FIXME_xxx" not allowed while "strict subs" in use at /usr/local/bin/master_ip_failover line 93.
Execution of /usr/local/bin/master_ip_failover aborted due to compilation errors.
Wed Feb 1 20:41:00 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln229] Failed to get master_ip_failover_script status with return code 255:0.
Wed Feb 1 20:41:00 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/bin/masterha_check_repl line 48.
Wed Feb 1 20:41:00 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Wed Feb 1 20:41:00 2017 - [info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!
处理：
暂时注释 master_ip_failover 脚本
############################################################################################################################
Keepalived
############################################################################################################################
问题17：
./configure --prefix=/usr/local/keepalived
configure: error:
!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSL headers files. !!!
处理：
yum -y install openssl-devel
问题18：
[root@mysqldb2 keepalived-1.3.2]# make
Making all in lib
make[1]: Entering directory `/root/olm/mha/keepalived-1.3.2/lib'
make all-am
make[2]: Entering directory `/root/olm/mha/keepalived-1.3.2/lib'
CC memory.o
CC utils.o
CC notify.o
notify.c: 在函数‘find_path’中:
notify.c:226: 警告：此函数中的‘sgid_num’在使用前可能未初始化
CC timer.o
CC scheduler.o
CC vector.o
CC list.o
CC html.o
CC parser.o
CC signals.o
CC logger.o
CC rttables.o
AR liblib.a
make[2]: Leaving directory `/root/olm/mha/keepalived-1.3.2/lib'
make[1]: Leaving directory `/root/olm/mha/keepalived-1.3.2/lib'
Making all in keepalived
make[1]: Entering directory `/root/olm/mha/keepalived-1.3.2/keepalived'
Making all in core
make[2]: Entering directory `/root/olm/mha/keepalived-1.3.2/keepalived/core'
CC main.o
CC daemon.o
CC pidfile.o
CC layer4.o
CC smtp.o
CC global_data.o
CC global_parser.o
CC process.o
CC namespaces.o
namespaces.c: 在函数‘setns’中:
namespaces.c:187: 错误：‘SYS_setns’未声明(在此函数内第一次使用)
namespaces.c:187: 错误：(即使在一个函数内多次出现，每个未声明的标识符在其
namespaces.c:187: 错误：所在的函数内也只报告一次。)
make[2]: *** [namespaces.o] 错误 1
make[2]: Leaving directory `/root/olm/mha/keepalived-1.3.2/keepalived/core'
make[1]: *** [all-recursive] 错误 1
make[1]: Leaving directory `/root/olm/mha/keepalived-1.3.2/keepalived'
make: *** [all-recursive] 错误 1
处理：
--安装 glibc 库仍无法解决问题
yum -y install glibc*
--使用低版本keepalived （keepalived-1.2.18）
问题19：
[root@oradb sysbench]# ./sysbench --test=olm --oltp-table-size=1000000 --oltp-read-only=off --init-rng=on --num-threads=16 --max-requests=0 --oltp-dist-type=uniform --max-time=1800 --mysql-user=root --db-driver=mysql --mysql-table-engine=innodb --oltp-test-mode=complexprepare
./sysbench: /usr/lib64/libmysqlclient.so.18: no version information available (required by ./sysbench)
处理：
ln -s /app/oracle/oradata2/mysql-5.6.27/lib/libmysqlclient.so /usr/lib64/libmysqlclient.so.18
问题：20
[root@mysqldb2 ~]# masterha_check_repl --conf=/etc/masterha/app1.cnf
Thu Feb 2 19:13:17 2017 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu Feb 2 19:13:17 2017 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Thu Feb 2 19:13:17 2017 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Thu Feb 2 19:13:17 2017 - [info] MHA::MasterMonitor version 0.57.
Thu Feb 2 19:13:17 2017 - [error][/usr/local/share/perl5/MHA/Server.pm, ln180] Got MySQL error when connecting 192.168.222.157(192.168.222.157:3306) :1045:Access denied for user 'root'@'mysqldb2' (using password: YES), but this is not a MySQL crash. Check MySQL server settings.
Thu Feb 2 19:13:17 2017 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln301] at /usr/local/share/perl5/MHA/ServerManager.pm line 297.
Thu Feb 2 19:13:18 2017 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln309] Got fatal error, stopping operations
Thu Feb 2 19:13:18 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations. at /usr/local/share/perl5/MHA/MasterMonitor.pm line 329.
Thu Feb 2 19:13:18 2017 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Thu Feb 2 19:13:18 2017 - [info] Got exit code 1 (Not master dead).
MySQL Replication Health is NOT OK!
原因：从库没有MHA监控用户
处理：
1.再主库创建监控用户
2.然后导出DUMP文件
3.导入备库创建 主从复制
4.执行权限刷新 flush privileges;
问题21：
Thu Feb 2 19:56:56 2017 - [info] Connecting to the target slave host 192.168.222.157, running recover script..
Thu Feb 2 19:56:56 2017 - [info] Executing command: apply_diff_relay_logs --command=apply --slave_user='root' --slave_host=192.168.222.157 --slave_ip=192.168.222.157 --slave_port=3306 --apply_files=/tmp/saved_master_binlog_from_192.168.222.156_3306_20170202195650.binlog --workdir=/tmp --target_version=5.6.27-log --timestamp=20170202195650 --handle_raw_binlog=1 --disable_log_bin=0 --manager_version=0.57 --slave_pass=xxx
Thu Feb 2 19:56:57 2017 - [info]
MySQL client version is 5.6.27. Using --binary-mode.
Applying differential binary/relay log files /tmp/saved_master_binlog_from_192.168.222.156_3306_20170202195650.binlog on 192.168.222.157:3306. This may take long time.
.....
Thu Feb 2 19:56:57 2017 - [info]
MySQL client version is 5.6.27. Using --binary-mode.
Applying differential binary/relay log files /tmp/saved_master_binlog_from_192.168.222.156_3306_20170202195650.binlog on 192.168.222.157:3306. This may take long time...
FATAL: applying log files failed with rc 1:0!
Error logs from mysqldb2:/tmp/relay_log_apply_for_192.168.222.157_3306_20170202195650_err.log (the last 200 lines)..
.....
at line 197: Cannot execute statement: impossible to write to binary log since statement is in row format and BINLOG_FORMAT = STATEMENT
....
--------------
Bye
at /usr/local/bin/apply_diff_relay_logs line 515.
eval {...} called at /usr/local/bin/apply_diff_relay_logs line 475
main::main() called at /usr/local/bin/apply_diff_relay_logs line 120
Thu Feb 2 19:56:57 2017 - [error][/usr/local/share/perl5/MHA/MasterFailover.pm, ln1398] Applying diffs failed with return code 22:0.
Thu Feb 2 19:56:57 2017 - [error][/usr/local/share/perl5/MHA/MasterFailover.pm, ln1561] Recovering master server failed.
Thu Feb 2 19:56:57 2017 - [error][/usr/local/share/perl5/MHA/ManagerUtil.pm, ln177] Got ERROR: at /usr/local/bin/masterha_manager line 65.
Thu Feb 2 19:56:57 2017 - [info]
----- Failover Report -----
app1: MySQL Master failover 192.168.222.156(192.168.222.156:3306)
Master 192.168.222.156(192.168.222.156:3306) is down!
Check MHA Manager logs at mysqldb2:/masterha/app1/app1.log for details.
Started automated(non-interactive) failover.
Invalidated master IP address on 192.168.222.156(192.168.222.156:3306)
The latest slave 192.168.222.157(192.168.222.157:3306) has all relay logs for recovery.
Selected 192.168.222.157(192.168.222.157:3306) as a new master.
Recovering master server failed.
Got Error so couldn't continue failover from here.
原因：二进制日志格式错误，需要修改为行格式
处理：
1.修改主库和备库的二进制日志格式
vi my.cnf
#添加
binlog_format=row
重启主备库MYSQL
2. 启动mha监控
nohup masterha_manager --conf=/etc/masterha/app1.cnf --remove_dead_master_conf --ignore_last_failover< /dev/null >/masterha/app1/manager.log 2>&1 &
Thu Feb 2 20:24:51 2017 - [info] MHA::MasterMonitor version 0.57.
Thu Feb 2 20:24:52 2017 - [info] GTID failover mode = 0
Thu Feb 2 20:24:52 2017 - [info] Dead Servers:
Thu Feb 2 20:24:52 2017 - [info] Alive Servers:
Thu Feb 2 20:24:52 2017 - [info] 192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:24:52 2017 - [info] 192.168.222.157(192.168.222.157:3306)
Thu Feb 2 20:24:52 2017 - [info] Alive Slaves:
Thu Feb 2 20:24:52 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Thu Feb 2 20:24:52 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:24:52 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Thu Feb 2 20:24:52 2017 - [info] Current Alive Master: 192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:24:52 2017 - [info] Checking slave configurations..
Thu Feb 2 20:24:52 2017 - [info] read_only=1 is not set on slave 192.168.222.157(192.168.222.157:3306).
Thu Feb 2 20:24:52 2017 - [warning] relay_log_purge=0 is not set on slave 192.168.222.157(192.168.222.157:3306).
Thu Feb 2 20:24:52 2017 - [info] Checking replication filtering settings..
Thu Feb 2 20:24:52 2017 - [info] binlog_do_db= , binlog_ignore_db=
Thu Feb 2 20:24:52 2017 - [info] Replication filtering check ok.
Thu Feb 2 20:24:52 2017 - [info] GTID (with auto-pos) is not supported
Thu Feb 2 20:24:52 2017 - [info] Starting SSH connection tests..
Thu Feb 2 20:24:54 2017 - [info] All SSH connection tests passed successfully.
Thu Feb 2 20:24:54 2017 - [info] Checking MHA Node version..
Thu Feb 2 20:24:54 2017 - [info] Version check ok.
Thu Feb 2 20:24:54 2017 - [info] Checking SSH publickey authentication settings on the current master..
Thu Feb 2 20:24:54 2017 - [info] HealthCheck: SSH to 192.168.222.156 is reachable.
Thu Feb 2 20:24:55 2017 - [info] Master MHA Node version is 0.57.
Thu Feb 2 20:24:55 2017 - [info] Checking recovery script configurations on 192.168.222.156(192.168.222.156:3306)..
Thu Feb 2 20:24:55 2017 - [info] Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data --output_file=/tmp/save_binary_logs_test --manager_version=0.57 --start_file=mysql-bin.000064
Thu Feb 2 20:24:55 2017 - [info] Connecting to root@192.168.222.156(192.168.222.156:22)..
Creating /tmp if not exists.. ok.
Checking output directory is accessible or not..
ok.
Binlog found at /app/oracle/oradata2/mysql-5.5.37/data, up to mysql-bin.000064
Thu Feb 2 20:24:55 2017 - [info] Binlog setting check done.
Thu Feb 2 20:24:55 2017 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Thu Feb 2 20:24:55 2017 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='root' --slave_host=192.168.222.157 --slave_ip=192.168.222.157 --slave_port=3306 --workdir=/tmp --target_version=5.6.27-log --manager_version=0.57 --relay_log_info=/usr/local/mysql/data/relay-log.info --relay_dir=/usr/local/mysql/data/ --slave_pass=xxx
Thu Feb 2 20:24:55 2017 - [info] Connecting to root@192.168.222.157(192.168.222.157:22)..
Checking slave recovery environment settings..
Opening /usr/local/mysql/data/relay-log.info ... ok.
Relay log found at /usr/local/mysql/data, up to relay-bin.000005
Temporary relay log file is /usr/local/mysql/data/relay-bin.000005
Testing mysql connection and privileges..Warning: Using a password on the command line interface can be insecure.
done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Thu Feb 2 20:24:56 2017 - [info] Slaves settings check done.
Thu Feb 2 20:24:56 2017 - [info]
192.168.222.156(192.168.222.156:3306) (current master)
+--192.168.222.157(192.168.222.157:3306)
Thu Feb 2 20:24:56 2017 - [info] Checking master_ip_failover_script status:
Thu Feb 2 20:24:56 2017 - [info] /usr/local/bin/master_ip_failover --command=status --ssh_user=root --orig_master_host=192.168.222.156 --orig_master_ip=192.168.222.156 --orig_master_port=3306
Thu Feb 2 20:24:56 2017 - [info] OK.
Thu Feb 2 20:24:56 2017 - [warning] shutdown_script is not defined.
Thu Feb 2 20:24:56 2017 - [info] Set master ping interval 1 seconds.
Thu Feb 2 20:24:56 2017 - [info] Set secondary check script: /usr/local/bin/masterha_secondary_check -s oradb --user=root --master_host=oradb --master_ip='192.168.222.156' --master_port=3306
Thu Feb 2 20:24:56 2017 - [info] Starting ping health check on 192.168.222.156(192.168.222.156:3306)..
Thu Feb 2 20:24:56 2017 - [info] Ping(SELECT) succeeded, waiting until MySQL doesn't respond..
3. 关闭主库
mysqladmin -u root shutdown
4. 启动MHA监控
nohup masterha_manager --conf=/etc/masterha/app1.cnf --remove_dead_master_conf --ignore_last_failover< /dev/null >/masterha/app1/manager.log 2>&1 &
Thu Feb 2 20:37:56 2017 - [warning] Got error on MySQL select ping: 2013 (Lost connection to MySQL server during query)
Thu Feb 2 20:37:56 2017 - [info] Executing secondary network check script: /usr/local/bin/masterha_secondary_check -s oradb --user=root --master_host=oradb --master_ip='192.168.222.156' --master_port=3306 --user=root --master_host=192.168.222.156 --master_ip=192.168.222.156 --master_port=3306 --master_user=root --master_password=123456 --ping_type=SELECT
Thu Feb 2 20:37:56 2017 - [info] Executing SSH check script: save_binary_logs --command=test --start_pos=4 --binlog_dir=/app/oracle/oradata2/mysql-5.5.37/data --output_file=/tmp/save_binary_logs_test --manager_version=0.57 --binlog_prefix=mysql-bin
Thu Feb 2 20:37:57 2017 - [warning] Got error on MySQL connect: 2003 (Can't connect to MySQL server on '192.168.222.156' (111))
Thu Feb 2 20:37:57 2017 - [warning] Connection failed 2 time(s)..
Thu Feb 2 20:37:58 2017 - [warning] Got error on MySQL connect: 2003 (Can't connect to MySQL server on '192.168.222.156' (111))
Thu Feb 2 20:37:58 2017 - [warning] Connection failed 3 time(s)..
Thu Feb 2 20:37:59 2017 - [warning] Got error on MySQL connect: 2003 (Can't connect to MySQL server on '192.168.222.156' (111))
Thu Feb 2 20:37:59 2017 - [warning] Connection failed 4 time(s)..
Thu Feb 2 20:38:01 2017 - [warning] HealthCheck: Got timeout on checking SSH connection to 192.168.222.156! at /usr/local/share/perl5/MHA/HealthCheck.pm line 342.
Monitoring server oradb is reachable, Master is not reachable from oradb. OK.
Thu Feb 2 20:38:08 2017 - [info] Master is not reachable from all other monitoring servers. Failover should start.
Thu Feb 2 20:38:08 2017 - [warning] Master is not reachable from health checker!
Thu Feb 2 20:38:08 2017 - [warning] Master 192.168.222.156(192.168.222.156:3306) is not reachable!
Thu Feb 2 20:38:08 2017 - [warning] SSH is NOT reachable.
Thu Feb 2 20:38:08 2017 - [info] Connecting to a master server failed. Reading configuration file /etc/masterha_default.cnf and /etc/masterha/app1.cnf again, and trying to connect to all servers to check server status..
Thu Feb 2 20:38:08 2017 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu Feb 2 20:38:08 2017 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Thu Feb 2 20:38:08 2017 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Thu Feb 2 20:38:09 2017 - [warning] SQL Thread is stopped(no error) on 192.168.222.157(192.168.222.157:3306)
Thu Feb 2 20:38:09 2017 - [info] GTID failover mode = 0
Thu Feb 2 20:38:09 2017 - [info] Dead Servers:
Thu Feb 2 20:38:09 2017 - [info] 192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:38:09 2017 - [info] Alive Servers:
Thu Feb 2 20:38:09 2017 - [info] 192.168.222.157(192.168.222.157:3306)
Thu Feb 2 20:38:09 2017 - [info] Alive Slaves:
Thu Feb 2 20:38:09 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Thu Feb 2 20:38:09 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:38:09 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Thu Feb 2 20:38:09 2017 - [info] Checking slave configurations..
Thu Feb 2 20:38:09 2017 - [info] read_only=1 is not set on slave 192.168.222.157(192.168.222.157:3306).
Thu Feb 2 20:38:09 2017 - [warning] relay_log_purge=0 is not set on slave 192.168.222.157(192.168.222.157:3306).
Thu Feb 2 20:38:09 2017 - [info] Checking replication filtering settings..
Thu Feb 2 20:38:09 2017 - [info] Replication filtering check ok.
Thu Feb 2 20:38:09 2017 - [info] Master is down!
Thu Feb 2 20:38:09 2017 - [info] Terminating monitoring script.
Thu Feb 2 20:38:09 2017 - [info] Got exit code 20 (Master dead).
Thu Feb 2 20:38:09 2017 - [info] MHA::MasterFailover version 0.57.
Thu Feb 2 20:38:09 2017 - [info] Starting master failover.
Thu Feb 2 20:38:09 2017 - [info]
Thu Feb 2 20:38:09 2017 - [info] * Phase 1: Configuration Check Phase..
Thu Feb 2 20:38:09 2017 - [info]
Thu Feb 2 20:38:10 2017 - [warning] SQL Thread is stopped(no error) on 192.168.222.157(192.168.222.157:3306)
Thu Feb 2 20:38:10 2017 - [info] GTID failover mode = 0
Thu Feb 2 20:38:10 2017 - [info] Dead Servers:
Thu Feb 2 20:38:10 2017 - [info] 192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:38:10 2017 - [info] Checking master reachability via MySQL(double check)...
Thu Feb 2 20:38:10 2017 - [info] ok.
Thu Feb 2 20:38:10 2017 - [info] Alive Servers:
Thu Feb 2 20:38:10 2017 - [info] 192.168.222.157(192.168.222.157:3306)
Thu Feb 2 20:38:10 2017 - [info] Alive Slaves:
Thu Feb 2 20:38:10 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Thu Feb 2 20:38:10 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:38:10 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Thu Feb 2 20:38:10 2017 - [info] Starting SQL thread on 192.168.222.157(192.168.222.157:3306) ..
Thu Feb 2 20:38:10 2017 - [info] done.
Thu Feb 2 20:38:10 2017 - [info] Starting Non-GTID based failover.
Thu Feb 2 20:38:10 2017 - [info]
Thu Feb 2 20:38:10 2017 - [info] ** Phase 1: Configuration Check Phase completed.
Thu Feb 2 20:38:10 2017 - [info]
Thu Feb 2 20:38:10 2017 - [info] * Phase 2: Dead Master Shutdown Phase..
Thu Feb 2 20:38:10 2017 - [info]
Thu Feb 2 20:38:10 2017 - [info] Forcing shutdown so that applications never connect to the current master..
Thu Feb 2 20:38:10 2017 - [info] Executing master IP deactivation script:
Thu Feb 2 20:38:10 2017 - [info] /usr/local/bin/master_ip_failover --orig_master_host=192.168.222.156 --orig_master_ip=192.168.222.156 --orig_master_port=3306 --command=stop
Thu Feb 2 20:38:10 2017 - [info] done.
Thu Feb 2 20:38:10 2017 - [warning] shutdown_script is not set. Skipping explicit shutting down of the dead master.
Thu Feb 2 20:38:10 2017 - [info] * Phase 2: Dead Master Shutdown Phase completed.
Thu Feb 2 20:38:10 2017 - [info]
Thu Feb 2 20:38:10 2017 - [info] * Phase 3: Master Recovery Phase..
Thu Feb 2 20:38:10 2017 - [info]
Thu Feb 2 20:38:10 2017 - [info] * Phase 3.1: Getting Latest Slaves Phase..
Thu Feb 2 20:38:10 2017 - [info]
Thu Feb 2 20:38:10 2017 - [info] The latest binary log file/position on all slaves is mysql-bin.000064:120
Thu Feb 2 20:38:10 2017 - [info] Latest slaves (Slaves that received relay log files to the latest):
Thu Feb 2 20:38:11 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Thu Feb 2 20:38:11 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:38:11 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Thu Feb 2 20:38:11 2017 - [info] The oldest binary log file/position on all slaves is mysql-bin.000064:120
Thu Feb 2 20:38:11 2017 - [info] Oldest slaves:
Thu Feb 2 20:38:11 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Thu Feb 2 20:38:11 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:38:11 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [info] * Phase 3.2: Saving Dead Master's Binlog Phase..
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [warning] Dead Master is not SSH reachable. Could not save it's binlogs. Transactions that were not sent to the latest slave (Read_Master_Log_Pos to the tail of the dead master's binlog) were lost.
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [info] * Phase 3.3: Determining New Master Phase..
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [info] Finding the latest slave that has all relay logs for recovering other slaves..
Thu Feb 2 20:38:11 2017 - [info] All slaves received relay logs to the same position. No need to resync each other.
Thu Feb 2 20:38:11 2017 - [info] Searching new master from slaves..
Thu Feb 2 20:38:11 2017 - [info] Candidate masters from the configuration file:
Thu Feb 2 20:38:11 2017 - [info] 192.168.222.157(192.168.222.157:3306) Version=5.6.27-log (oldest major version between slaves) log-bin:enabled
Thu Feb 2 20:38:11 2017 - [info] Replicating from 192.168.222.156(192.168.222.156:3306)
Thu Feb 2 20:38:11 2017 - [info] Primary candidate for the new Master (candidate_master is set)
Thu Feb 2 20:38:11 2017 - [info] Non-candidate masters:
Thu Feb 2 20:38:11 2017 - [info] Searching from candidate_master slaves which have received the latest relay log events..
Thu Feb 2 20:38:11 2017 - [info] New master is 192.168.222.157(192.168.222.157:3306)
Thu Feb 2 20:38:11 2017 - [info] Starting master failover..
Thu Feb 2 20:38:11 2017 - [info]
From:
192.168.222.156(192.168.222.156:3306) (current master)
+--192.168.222.157(192.168.222.157:3306)
To:
192.168.222.157(192.168.222.157:3306) (new master)
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [info] * Phase 3.3: New Master Diff Log Generation Phase..
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [info] This server has all relay logs. No need to generate diff files from the latest slave.
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [info] * Phase 3.4: Master Log Apply Phase..
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [info] *NOTICE: If any error happens from this phase, manual recovery is needed.
Thu Feb 2 20:38:11 2017 - [info] Starting recovery on 192.168.222.157(192.168.222.157:3306)..
Thu Feb 2 20:38:11 2017 - [info] This server has all relay logs. Waiting all logs to be applied..
Thu Feb 2 20:38:11 2017 - [info] done.
Thu Feb 2 20:38:11 2017 - [info] All relay logs were successfully applied.
Thu Feb 2 20:38:11 2017 - [info] Getting new master's binlog name and position..
Thu Feb 2 20:38:11 2017 - [info] mysql-bin.000003:120
Thu Feb 2 20:38:11 2017 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='192.168.222.157', MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.000003', MASTER_LOG_POS=120, MASTER_USER='repl', MASTER_PASSWORD='xxx';
Thu Feb 2 20:38:11 2017 - [info] Executing master IP activate script:
Thu Feb 2 20:38:11 2017 - [info] /usr/local/bin/master_ip_failover --command=start --ssh_user=root --orig_master_host=192.168.222.156 --orig_master_ip=192.168.222.156 --orig_master_port=3306 --new_master_host=192.168.222.157 --new_master_ip=192.168.222.157 --new_master_port=3306 --new_master_user='root' --new_master_password=xxx
Undefined subroutine &main::FIXME_xxx_create_user called at /usr/local/bin/master_ip_failover line 89.
Set read_only=0 on the new master.
Creating app user on the new master..
Thu Feb 2 20:38:11 2017 - [error][/usr/local/share/perl5/MHA/MasterFailover.pm, ln1589] Failed to activate master IP address for 192.168.222.157(192.168.222.157:3306) with return code 10:0
Thu Feb 2 20:38:11 2017 - [warning] Proceeding.
Thu Feb 2 20:38:11 2017 - [info] ** Finished master recovery successfully.
Thu Feb 2 20:38:11 2017 - [info] * Phase 3: Master Recovery Phase completed.
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [info] * Phase 4: Slaves Recovery Phase..
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [info] * Phase 4.1: Starting Parallel Slave Diff Log Generation Phase..
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [info] Generating relay diff files from the latest slave succeeded.
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [info] * Phase 4.2: Starting Parallel Slave Log Apply Phase..
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [info] All new slave servers recovered successfully.
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [info] * Phase 5: New master cleanup phase..
Thu Feb 2 20:38:11 2017 - [info]
Thu Feb 2 20:38:11 2017 - [info] Resetting slave info on the new master..
Thu Feb 2 20:38:11 2017 - [info] 192.168.222.157: Resetting slave info succeeded.
Thu Feb 2 20:38:11 2017 - [info] Master failover to 192.168.222.157(192.168.222.157:3306) completed successfully.
Thu Feb 2 20:38:11 2017 - [info] Deleted server1 entry from /etc/masterha/app1.cnf .
Thu Feb 2 20:38:11 2017 - [info]
----- Failover Report -----
app1: MySQL Master failover 192.168.222.156(192.168.222.156:3306) to 192.168.222.157(192.168.222.157:3306) succeeded
Master 192.168.222.156(192.168.222.156:3306) is down!
Check MHA Manager logs at mysqldb2:/masterha/app1/app1.log for details.
Started automated(non-interactive) failover.
Invalidated master IP address on 192.168.222.156(192.168.222.156:3306)
The latest slave 192.168.222.157(192.168.222.157:3306) has all relay logs for recovery.
Selected 192.168.222.157(192.168.222.157:3306) as a new master.
192.168.222.157(192.168.222.157:3306): OK: Applying all logs succeeded.
Failed to activate master IP address for 192.168.222.157(192.168.222.157:3306) with return code 10:0
Generating relay diff files from the latest slave succeeded.
192.168.222.157(192.168.222.157:3306): Resetting slave info succeeded.
Master failover to 192.168.222.157(192.168.222.157:3306) completed successfully.
来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/21582653/viewspace-2133104/，如需转载，请注明出处，否则将追究法律责任。
转载于:http://blog.itpub.net/21582653/viewspace-2133104/
扫一扫
2149
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
