mysql 高可用架构 mha 之二 master_ip_failover

最新推荐文章于 2021-05-08 16:58:08 发布
最新推荐文章于 2021-05-08 16:58:08 发布
阅读量225 收藏
点赞数
文章标签： 数据库 运维
原文链接：http://www.cnblogs.com/ctypyb2002/p/9792911.html
版权

os:centos 7.4
mysql: 5.7
mha: 0.58

上一篇blog介绍了 mha 的安装，这一篇blog介绍下 master_ip_failover_script 的设置。

ip 规划如下：

192.168.56.101 node1 (mha manager)

192.168.56.102 node2 (mysql master)
192.168.56.103 node3 (mysql master candicate)
192.168.56.104 node4 (mysql slave)

追加 mysql master 的 vip 192.168.56.100/24

master_ip_failover参数

mha manager 节点上

# vi /etc/masterha/app1.cnf 

#自动failover时候的切换脚本
master_ip_failover_script= /usr/local/bin/master_ip_failover

master_ip_failover_script 指的是mysql master 失败时执行的切换脚本。
没有使用 keepalived ，通过脚本的方式管理vip。

# cp /usr/local/bin/master_ip_failover /usr/local/bin/master_ip_failover.bak
# vi /usr/local/bin/master_ip_failover

#!/usr/bin/env perl
use strict;
use warnings FATAL => 'all';

use Getopt::Long;

my (
    $command,          $ssh_user,        $orig_master_host, $orig_master_ip,
    $orig_master_port, $new_master_host, $new_master_ip,    $new_master_port
);

my $vip = '192.168.56.100';
my $brdc = '192.168.56.255';
my $ifdev = 'enp0s8';
my $key = '1';
my $ssh_start_vip = "/usr/sbin/ip addr add $vip/24 brd $brdc dev $ifdev label $ifdev:$key;/usr/sbin/arping -q -A -c 1 -I $ifdev $vip;iptables -F;";
my $ssh_stop_vip = "/usr/sbin/ip addr del $vip/24 dev $ifdev label $ifdev:$key";

GetOptions(
    'command=s'          => \$command,
    'ssh_user=s'         => \$ssh_user,
    'orig_master_host=s' => \$orig_master_host,
    'orig_master_ip=s'   => \$orig_master_ip,
    'orig_master_port=i' => \$orig_master_port,
    'new_master_host=s'  => \$new_master_host,
    'new_master_ip=s'    => \$new_master_ip,
    'new_master_port=i'  => \$new_master_port,
);

exit &main();

sub main {

    print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";

    if ( $command eq "stop" || $command eq "stopssh" ) {

        my $exit_code = 1;
        eval {
            print "Disabling the VIP on old master: $orig_master_host \n";
            &stop_vip();
            $exit_code = 0;
        };
        if ($@) {
            warn "Got Error: $@\n";
            exit $exit_code;
        }
        exit $exit_code;
    }
    elsif ( $command eq "start" ) {

        my $exit_code = 10;
        eval {
            print "Enabling the VIP - $vip on the new master - $new_master_host \n";
            &start_vip();
            $exit_code = 0;
        };
        if ($@) {
            warn $@;
            exit $exit_code;
        }
        exit $exit_code;
    }
    elsif ( $command eq "status" ) {
        print "Checking the Status of the script.. OK \n";
        exit 0;
    }
    else {
        &usage();
        exit 1;
    }
}
sub start_vip() {
    `ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
    `ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}

sub usage {
    print
    "Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}

更换ip后，一定要执行下 arping

检查复制环境ssh

# masterha_check_ssh --conf=/etc/masterha/app1.cnf

检查整个复制环境

# masterha_check_repl --conf=/etc/masterha/app1.cnf

Tue Aug  7 03:44:13 2018 - [info] Checking master_ip_failover_script status:
Tue Aug  7 03:44:13 2018 - [info]   /usr/local/bin/master_ip_failover --command=status --ssh_user=root --orig_master_host=192.168.56.102 --orig_master_ip=192.168.56.102 --orig_master_port=3306 


IN SCRIPT TEST====/usr/sbin/ip addr del 192.168.56.100/24 dev enp0s8 label enp0s8:1==/usr/sbin/ip addr add 192.168.56.100/24 brd 192.168.56.255 dev enp0s8 label enp0s8:1;/usr/sbin/arping -q -A -c 1 -I enp0s8 192.168.56.100;===

Checking the Status of the script.. OK 
Tue Aug  7 03:44:13 2018 - [info]  OK.

启动 mha manager

启动 manager

# nohup masterha_manager --conf=/etc/masterha/app1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/masterha/app1-manager.log 2>&1 &

查看 manager status

# masterha_check_status --conf=/etc/masterha/app1.cnf

查看 manager log 

# tail -n 1000 -f /var/log/masterha/app1-manager.log

验证 failover

node2上 kill 掉 mysqld 进程

# ps -ef|grep -i mysql
mysql     3114     1  0 Aug06 ?        00:00:51 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
root     15551 10466  0 Aug06 pts/1    00:00:00 mysql
root     25521 21213  0 03:52 pts/2    00:00:00 grep --color=auto -i mysql
# kill -9 3114

观察 mha manager 之前打开的日志输出

# tail -n 1000 -f /var/log/masterha/app1-manager.log

Tue Aug  7 03:53:08 2018 - [warning] Got error on MySQL select ping: 2013 (Lost connection to MySQL server during query)
Tue Aug  7 03:53:08 2018 - [info] Executing SSH check script: save_binary_logs --command=test --start_pos=4 --binlog_dir=/var/lib/mysql --output_file=/tmp/save_binary_logs_test --manager_version=0.58 --binlog_prefix=mysql-bin
Tue Aug  7 03:53:08 2018 - [info] Executing secondary network check script: /usr/local/bin/masterha_secondary_check -s node3 -s node2  --user=root  --master_host=192.168.56.102  --master_ip=192.168.56.102  --master_port=3306 --master_user=mha_mon --master_password=2wsx3edc --ping_type=SELECT
Monitoring server node3 is reachable, Master is not reachable from node3. OK.
Tue Aug  7 03:53:09 2018 - [info] HealthCheck: SSH to 192.168.56.102 is reachable.
Monitoring server node2 is reachable, Master is not reachable from node2. OK.
Tue Aug  7 03:53:09 2018 - [info] Master is not reachable from all other monitoring servers. Failover should start.
Tue Aug  7 03:53:13 2018 - [info] Ping(SELECT) succeeded, waiting until MySQL doesn't respond..

居然把node2上的mysqld 重新拉起来了。

node2 虚拟机掉电

虚拟机掉电验证的更直接。

# tail -n 1000 -f /var/log/masterha/app1-manager.log

Tue Aug  7 03:58:29 2018 - [warning] Got timeout on MySQL Ping(SELECT) child process and killed it! at /usr/local/share/perl5/MHA/HealthCheck.pm line 432.
Tue Aug  7 03:58:29 2018 - [info] Executing secondary network check script: /usr/local/bin/masterha_secondary_check -s node3 -s node2  --user=root  --master_host=192.168.56.102  --master_ip=192.168.56.102  --master_port=3306 --master_user=mha_mon --master_password=2wsx3edc --ping_type=SELECT
Tue Aug  7 03:58:29 2018 - [info] Executing SSH check script: save_binary_logs --command=test --start_pos=4 --binlog_dir=/var/lib/mysql --output_file=/tmp/save_binary_logs_test --manager_version=0.58 --binlog_prefix=mysql-bin
Tue Aug  7 03:58:34 2018 - [warning] Got error on MySQL connect: 2003 (Can't connect to MySQL server on '192.168.56.102' (110))
Tue Aug  7 03:58:34 2018 - [warning] Connection failed 2 time(s)..
Tue Aug  7 03:58:34 2018 - [warning] HealthCheck: Got timeout on checking SSH connection to 192.168.56.102! at /usr/local/share/perl5/MHA/HealthCheck.pm line 343.
Monitoring server node3 is reachable, Master is not reachable from node3. OK.
Tue Aug  7 03:58:39 2018 - [warning] Got error on MySQL connect: 2003 (Can't connect to MySQL server on '192.168.56.102' (110))
Tue Aug  7 03:58:39 2018 - [warning] Connection failed 3 time(s)..
ssh: connect to host node2 port 22: Connection timed out
Monitoring server node2 is NOT reachable!
Tue Aug  7 03:58:39 2018 - [warning] At least one of monitoring servers is not reachable from this script. This is likely a network problem. Failover should not happen.

居然输出 This is likely a network problem. Failover should not happen 。本来是测试 failover的，结果 mha 认为 Failover should not happen。

难道node2宕机，node3、node4存活，mha还认为是网络问题，不进行切换？是哪里设置出了问题吗？

再把 node2 加电起来，观察 mha manager 日志

# tail -n 1000 -f /var/log/masterha/app1-manager.log

Tue Aug  7 04:06:44 2018 - [warning] Got error on MySQL connect: 2003 (Can't connect to MySQL server on '192.168.56.102' (111))
Tue Aug  7 04:06:44 2018 - [warning] Connection failed 1 time(s)..
Tue Aug  7 04:06:44 2018 - [info] Executing secondary network check script: /usr/local/bin/masterha_secondary_check -s node3 -s node2  --user=root  --master_host=192.168.56.102  --master_ip=192.168.56.102  --master_port=3306 --master_user=mha_mon --master_password=2wsx3edc --ping_type=SELECT
Tue Aug  7 04:06:44 2018 - [info] Executing SSH check script: save_binary_logs --command=test --start_pos=4 --binlog_dir=/var/lib/mysql --output_file=/tmp/save_binary_logs_test --manager_version=0.58 --binlog_prefix=mysql-bin
Monitoring server node3 is reachable, Master is not reachable from node3. OK.
Monitoring server node2 is reachable, Master is not reachable from node2. OK.
Tue Aug  7 04:06:47 2018 - [info] Master is not reachable from all other monitoring servers. Failover should start.
Tue Aug  7 04:06:47 2018 - [info] HealthCheck: SSH to 192.168.56.102 is reachable.
···
···
···
Tue Aug  7 04:07:02 2018 - [info] New master is 192.168.56.103(192.168.56.103:3306)
Tue Aug  7 04:07:02 2018 - [info] Starting master failover..
Tue Aug  7 04:07:02 2018 - [info] 
From:
192.168.56.102(192.168.56.102:3306) (current master)
 +--192.168.56.103(192.168.56.103:3306)
 +--192.168.56.104(192.168.56.104:3306)

To:
192.168.56.103(192.168.56.103:3306) (new master)
 +--192.168.56.104(192.168.56.104:3306)
···
···
···
----- Failover Report -----

app1: MySQL Master failover 192.168.56.102(192.168.56.102:3306) to 192.168.56.103(192.168.56.103:3306) succeeded

Master 192.168.56.102(192.168.56.102:3306) is down!

Check MHA Manager logs at node1:/var/log/masterha/app1-manager.log for details.

Started automated(non-interactive) failover.
Invalidated master IP address on 192.168.56.102(192.168.56.102:3306)
The latest slave 192.168.56.103(192.168.56.103:3306) has all relay logs for recovery.
Selected 192.168.56.103(192.168.56.103:3306) as a new master.
192.168.56.103(192.168.56.103:3306): OK: Applying all logs succeeded.
192.168.56.103(192.168.56.103:3306): OK: Activated master IP address.
192.168.56.104(192.168.56.104:3306): This host has the latest relay log events.
Generating relay diff files from the latest slave succeeded.
192.168.56.104(192.168.56.104:3306): OK: Applying all logs succeeded. Slave started, replicating from 192.168.56.103(192.168.56.103:3306)
192.168.56.103(192.168.56.103:3306): Resetting slave info succeeded.
Master failover to 192.168.56.103(192.168.56.103:3306) completed successfully.

把node1 起来之后，提示出现 Failover should start

查看 node3 的ip

# ifconfig  enp0s8:1
enp0s8:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.56.100  netmask 255.255.255.0  broadcast 192.168.56.255
        ether 08:00:27:e8:24:31  txqueuelen 1000  (Ethernet)

查看 node4 的 mysql slave status，已经发生了变化。

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.56.103
                  Master_User: replicator
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000006
          Read_Master_Log_Pos: 947
               Relay_Log_File: node4-relay-bin.000002
                Relay_Log_Pos: 320
        Relay_Master_Log_File: mysql-bin.000006
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes

关闭 mha manager

关闭 manager

# masterha_stop --conf=/etc/masterha/app1.cnf
Stopped app1 successfully.

查看 manager log

# tail -n 1000 -f /var/log/masterha/app1-manager.log

vip 的一些操作

# ip addr add 192.168.56.100/24 brd 192.168.56.255 dev enp0s8 label enp0s8:1
# /usr/sbin/arping -q -A -c 1 -I enp0s8 192.168.56.100
# ip addr del 192.168.56.100/24 dev enp0s8 label enp0s8:1

slave 的设置

mysql> change master to
  master_host='192.168.56.102',
  master_user='replicator',
  master_password='2wsx3edc',
  master_port=3306,
  master_log_file='mysql-bin.000007',
  master_log_pos=154;
mysql> start slave;

参考：
https://github.com/yoshinorim/mha4mysql-manager/wiki
https://github.com/yoshinorim/mha4mysql-manager/wiki/Parameters

https://github.com/yoshinorim/mha4mysql-manager/releases
https://github.com/yoshinorim/mha4mysql-node/releases

https://github.com/yoshinorim/mha4mysql-manager
https://github.com/yoshinorim/mha4mysql-node

failover的过程，基本为以下步骤：

1.配置文件检查阶段，这个阶段会检查整个集群配置文件配置
2.宕机的master处理，这个阶段包括虚拟ip摘除操作，主机关机操作
3.复制dead maste和最新slave相差的relay log，并保存到MHA Manger具体的目录下
4.识别含有最新更新的slave
5.应用从master保存的二进制日志事件（binlog events）
6.提升一个slave为新的master进行复制
7.使其他的slave连接新的master进行复制

下面是完整日志输出

Tue Aug  7 04:06:59 2018 - [warning] Got error on MySQL connect: 2003 (Can't connect to MySQL server on '192.168.56.102' (111))
Tue Aug  7 04:06:59 2018 - [warning] Connection failed 4 time(s)..
Tue Aug  7 04:06:59 2018 - [warning] Master is not reachable from health checker!
Tue Aug  7 04:06:59 2018 - [warning] Master 192.168.56.102(192.168.56.102:3306) is not reachable!
Tue Aug  7 04:06:59 2018 - [warning] SSH is reachable.
Tue Aug  7 04:06:59 2018 - [info] Connecting to a master server failed. Reading configuration file /etc/masterha_default.cnf and /etc/masterha/app1.cnf again, and trying to connect to all servers to check server status..
Tue Aug  7 04:06:59 2018 - [info] Reading default configuration from /etc/masterha_default.cnf..
Tue Aug  7 04:06:59 2018 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Tue Aug  7 04:06:59 2018 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Tue Aug  7 04:07:00 2018 - [info] GTID failover mode = 0
Tue Aug  7 04:07:00 2018 - [info] Dead Servers:
Tue Aug  7 04:07:00 2018 - [info]   192.168.56.102(192.168.56.102:3306)
Tue Aug  7 04:07:00 2018 - [info] Alive Servers:
Tue Aug  7 04:07:00 2018 - [info]   192.168.56.103(192.168.56.103:3306)
Tue Aug  7 04:07:00 2018 - [info]   192.168.56.104(192.168.56.104:3306)
Tue Aug  7 04:07:00 2018 - [info] Alive Slaves:
Tue Aug  7 04:07:00 2018 - [info]   192.168.56.103(192.168.56.103:3306)  Version=5.7.22-log (oldest major version between slaves) log-bin:enabled
Tue Aug  7 04:07:00 2018 - [info]     Replicating from 192.168.56.102(192.168.56.102:3306)
Tue Aug  7 04:07:00 2018 - [info]   192.168.56.104(192.168.56.104:3306)  Version=5.7.23-log (oldest major version between slaves) log-bin:enabled
Tue Aug  7 04:07:00 2018 - [info]     Replicating from 192.168.56.102(192.168.56.102:3306)
Tue Aug  7 04:07:00 2018 - [info] Checking slave configurations..
Tue Aug  7 04:07:00 2018 - [info] Checking replication filtering settings..
Tue Aug  7 04:07:00 2018 - [info]  Replication filtering check ok.
Tue Aug  7 04:07:00 2018 - [info] Master is down!
Tue Aug  7 04:07:00 2018 - [info] Terminating monitoring script.
Tue Aug  7 04:07:00 2018 - [info] Got exit code 20 (Master dead).
Tue Aug  7 04:07:00 2018 - [info] MHA::MasterFailover version 0.58.
Tue Aug  7 04:07:00 2018 - [info] Starting master failover.
Tue Aug  7 04:07:00 2018 - [info] 
Tue Aug  7 04:07:00 2018 - [info] * Phase 1: Configuration Check Phase..
Tue Aug  7 04:07:00 2018 - [info] 
Tue Aug  7 04:07:01 2018 - [info] GTID failover mode = 0
Tue Aug  7 04:07:01 2018 - [info] Dead Servers:
Tue Aug  7 04:07:01 2018 - [info]   192.168.56.102(192.168.56.102:3306)
Tue Aug  7 04:07:01 2018 - [info] Checking master reachability via MySQL(double check)...
Tue Aug  7 04:07:01 2018 - [info]  ok.
Tue Aug  7 04:07:01 2018 - [info] Alive Servers:
Tue Aug  7 04:07:01 2018 - [info]   192.168.56.103(192.168.56.103:3306)
Tue Aug  7 04:07:01 2018 - [info]   192.168.56.104(192.168.56.104:3306)
Tue Aug  7 04:07:01 2018 - [info] Alive Slaves:
Tue Aug  7 04:07:01 2018 - [info]   192.168.56.103(192.168.56.103:3306)  Version=5.7.22-log (oldest major version between slaves) log-bin:enabled
Tue Aug  7 04:07:01 2018 - [info]     Replicating from 192.168.56.102(192.168.56.102:3306)
Tue Aug  7 04:07:01 2018 - [info]   192.168.56.104(192.168.56.104:3306)  Version=5.7.23-log (oldest major version between slaves) log-bin:enabled
Tue Aug  7 04:07:01 2018 - [info]     Replicating from 192.168.56.102(192.168.56.102:3306)
Tue Aug  7 04:07:01 2018 - [info] Starting Non-GTID based failover.
Tue Aug  7 04:07:01 2018 - [info] 
Tue Aug  7 04:07:01 2018 - [info] ** Phase 1: Configuration Check Phase completed.
Tue Aug  7 04:07:01 2018 - [info] 
Tue Aug  7 04:07:01 2018 - [info] * Phase 2: Dead Master Shutdown Phase..
Tue Aug  7 04:07:01 2018 - [info] 
Tue Aug  7 04:07:01 2018 - [info] Forcing shutdown so that applications never connect to the current master..
Tue Aug  7 04:07:01 2018 - [info] Executing master IP deactivation script:
Tue Aug  7 04:07:01 2018 - [info]   /usr/local/bin/master_ip_failover --orig_master_host=192.168.56.102 --orig_master_ip=192.168.56.102 --orig_master_port=3306 --command=stopssh --ssh_user=root  


IN SCRIPT TEST====/usr/sbin/ip addr del 192.168.56.100/24 dev enp0s8 label enp0s8:1==/usr/sbin/ip addr add 192.168.56.100/24 brd 192.168.56.255 dev enp0s8 label enp0s8:1;/usr/sbin/arping -q -A -c 1 -I enp0s8 192.168.56.100;===

Disabling the VIP on old master: 192.168.56.102 
RTNETLINK answers: Cannot assign requested address
Tue Aug  7 04:07:01 2018 - [info]  done.
Tue Aug  7 04:07:01 2018 - [warning] shutdown_script is not set. Skipping explicit shutting down of the dead master.
Tue Aug  7 04:07:01 2018 - [info] * Phase 2: Dead Master Shutdown Phase completed.
Tue Aug  7 04:07:01 2018 - [info] 
Tue Aug  7 04:07:01 2018 - [info] * Phase 3: Master Recovery Phase..
Tue Aug  7 04:07:01 2018 - [info] 
Tue Aug  7 04:07:01 2018 - [info] * Phase 3.1: Getting Latest Slaves Phase..
Tue Aug  7 04:07:01 2018 - [info] 
Tue Aug  7 04:07:01 2018 - [info] The latest binary log file/position on all slaves is mysql-bin.000006:154
Tue Aug  7 04:07:01 2018 - [info] Latest slaves (Slaves that received relay log files to the latest):
Tue Aug  7 04:07:01 2018 - [info]   192.168.56.103(192.168.56.103:3306)  Version=5.7.22-log (oldest major version between slaves) log-bin:enabled
Tue Aug  7 04:07:01 2018 - [info]     Replicating from 192.168.56.102(192.168.56.102:3306)
Tue Aug  7 04:07:01 2018 - [info]   192.168.56.104(192.168.56.104:3306)  Version=5.7.23-log (oldest major version between slaves) log-bin:enabled
Tue Aug  7 04:07:01 2018 - [info]     Replicating from 192.168.56.102(192.168.56.102:3306)
Tue Aug  7 04:07:01 2018 - [info] The oldest binary log file/position on all slaves is mysql-bin.000006:154
Tue Aug  7 04:07:01 2018 - [info] Oldest slaves:
Tue Aug  7 04:07:01 2018 - [info]   192.168.56.103(192.168.56.103:3306)  Version=5.7.22-log (oldest major version between slaves) log-bin:enabled
Tue Aug  7 04:07:01 2018 - [info]     Replicating from 192.168.56.102(192.168.56.102:3306)
Tue Aug  7 04:07:01 2018 - [info]   192.168.56.104(192.168.56.104:3306)  Version=5.7.23-log (oldest major version between slaves) log-bin:enabled
Tue Aug  7 04:07:01 2018 - [info]     Replicating from 192.168.56.102(192.168.56.102:3306)
Tue Aug  7 04:07:01 2018 - [info] 
Tue Aug  7 04:07:01 2018 - [info] * Phase 3.2: Saving Dead Master's Binlog Phase..
Tue Aug  7 04:07:01 2018 - [info] 
Tue Aug  7 04:07:02 2018 - [info] Fetching dead master's binary logs..
Tue Aug  7 04:07:02 2018 - [info] Executing command on the dead master 192.168.56.102(192.168.56.102:3306): save_binary_logs --command=save --start_file=mysql-bin.000006  --start_pos=154 --binlog_dir=/var/lib/mysql --output_file=/tmp/saved_master_binlog_from_192.168.56.102_3306_20180807040700.binlog --handle_raw_binlog=1 --disable_log_bin=0 --manager_version=0.58
  Creating /tmp if not exists..    ok.
 Concat binary/relay logs from mysql-bin.000006 pos 154 to mysql-bin.000006 EOF into /tmp/saved_master_binlog_from_192.168.56.102_3306_20180807040700.binlog ..
 Binlog Checksum enabled
  Dumping binlog format description event, from position 0 to 154.. ok.
  No need to dump effective binlog data from /var/lib/mysql/mysql-bin.000006 (pos starts 154, filesize 154). Skipping.
 Binlog Checksum enabled
 /tmp/saved_master_binlog_from_192.168.56.102_3306_20180807040700.binlog has no effective data events.
Event not exists.
Tue Aug  7 04:07:02 2018 - [info] Additional events were not found from the orig master. No need to save.
Tue Aug  7 04:07:02 2018 - [info] 
Tue Aug  7 04:07:02 2018 - [info] * Phase 3.3: Determining New Master Phase..
Tue Aug  7 04:07:02 2018 - [info] 
Tue Aug  7 04:07:02 2018 - [info] Finding the latest slave that has all relay logs for recovering other slaves..
Tue Aug  7 04:07:02 2018 - [info] All slaves received relay logs to the same position. No need to resync each other.
Tue Aug  7 04:07:02 2018 - [info] Searching new master from slaves..
Tue Aug  7 04:07:02 2018 - [info]  Candidate masters from the configuration file:
Tue Aug  7 04:07:02 2018 - [info]  Non-candidate masters:
Tue Aug  7 04:07:02 2018 - [info] New master is 192.168.56.103(192.168.56.103:3306)
Tue Aug  7 04:07:02 2018 - [info] Starting master failover..
Tue Aug  7 04:07:02 2018 - [info] 
From:
192.168.56.102(192.168.56.102:3306) (current master)
 +--192.168.56.103(192.168.56.103:3306)
 +--192.168.56.104(192.168.56.104:3306)

To:
192.168.56.103(192.168.56.103:3306) (new master)
 +--192.168.56.104(192.168.56.104:3306)
Tue Aug  7 04:07:02 2018 - [info] 
Tue Aug  7 04:07:02 2018 - [info] * Phase 3.4: New Master Diff Log Generation Phase..
Tue Aug  7 04:07:02 2018 - [info] 
Tue Aug  7 04:07:02 2018 - [info]  This server has all relay logs. No need to generate diff files from the latest slave.
Tue Aug  7 04:07:02 2018 - [info] 
Tue Aug  7 04:07:02 2018 - [info] * Phase 3.5: Master Log Apply Phase..
Tue Aug  7 04:07:02 2018 - [info] 
Tue Aug  7 04:07:02 2018 - [info] *NOTICE: If any error happens from this phase, manual recovery is needed.
Tue Aug  7 04:07:02 2018 - [info] Starting recovery on 192.168.56.103(192.168.56.103:3306)..
Tue Aug  7 04:07:02 2018 - [info]  This server has all relay logs. Waiting all logs to be applied.. 
Tue Aug  7 04:07:02 2018 - [info]   done.
Tue Aug  7 04:07:02 2018 - [info]  All relay logs were successfully applied.
Tue Aug  7 04:07:02 2018 - [info] Getting new master's binlog name and position..
Tue Aug  7 04:07:02 2018 - [info]  mysql-bin.000006:947
Tue Aug  7 04:07:02 2018 - [info]  All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='192.168.56.103', MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.000006', MASTER_LOG_POS=947, MASTER_USER='replicator', MASTER_PASSWORD='xxx';
Tue Aug  7 04:07:02 2018 - [info] Executing master IP activate script:
Tue Aug  7 04:07:02 2018 - [info]   /usr/local/bin/master_ip_failover --command=start --ssh_user=root --orig_master_host=192.168.56.102 --orig_master_ip=192.168.56.102 --orig_master_port=3306 --new_master_host=192.168.56.103 --new_master_ip=192.168.56.103 --new_master_port=3306 --new_master_user='mha_mon'   --new_master_password=xxx
Unknown option: new_master_user
Unknown option: new_master_password


IN SCRIPT TEST====/usr/sbin/ip addr del 192.168.56.100/24 dev enp0s8 label enp0s8:1==/usr/sbin/ip addr add 192.168.56.100/24 brd 192.168.56.255 dev enp0s8 label enp0s8:1;/usr/sbin/arping -q -A -c 1 -I enp0s8 192.168.56.100;===

Enabling the VIP - 192.168.56.100 on the new master - 192.168.56.103 
Tue Aug  7 04:07:02 2018 - [info]  OK.
Tue Aug  7 04:07:02 2018 - [info] Setting read_only=0 on 192.168.56.103(192.168.56.103:3306)..
Tue Aug  7 04:07:02 2018 - [info]  ok.
Tue Aug  7 04:07:02 2018 - [info] ** Finished master recovery successfully.
Tue Aug  7 04:07:02 2018 - [info] * Phase 3: Master Recovery Phase completed.
Tue Aug  7 04:07:02 2018 - [info] 
Tue Aug  7 04:07:02 2018 - [info] * Phase 4: Slaves Recovery Phase..
Tue Aug  7 04:07:02 2018 - [info] 
Tue Aug  7 04:07:02 2018 - [info] * Phase 4.1: Starting Parallel Slave Diff Log Generation Phase..
Tue Aug  7 04:07:02 2018 - [info] 
Tue Aug  7 04:07:02 2018 - [info] -- Slave diff file generation on host 192.168.56.104(192.168.56.104:3306) started, pid: 24628. Check tmp log /etc/masterha/app1/192.168.56.104_3306_20180807040700.log if it takes time..
Tue Aug  7 04:07:03 2018 - [info] 
Tue Aug  7 04:07:03 2018 - [info] Log messages from 192.168.56.104 ...
Tue Aug  7 04:07:03 2018 - [info] 
Tue Aug  7 04:07:02 2018 - [info]  This server has all relay logs. No need to generate diff files from the latest slave.
Tue Aug  7 04:07:03 2018 - [info] End of log messages from 192.168.56.104.
Tue Aug  7 04:07:03 2018 - [info] -- 192.168.56.104(192.168.56.104:3306) has the latest relay log events.
Tue Aug  7 04:07:03 2018 - [info] Generating relay diff files from the latest slave succeeded.
Tue Aug  7 04:07:03 2018 - [info] 
Tue Aug  7 04:07:03 2018 - [info] * Phase 4.2: Starting Parallel Slave Log Apply Phase..
Tue Aug  7 04:07:03 2018 - [info] 
Tue Aug  7 04:07:03 2018 - [info] -- Slave recovery on host 192.168.56.104(192.168.56.104:3306) started, pid: 24630. Check tmp log /etc/masterha/app1/192.168.56.104_3306_20180807040700.log if it takes time..
Tue Aug  7 04:07:04 2018 - [info] 
Tue Aug  7 04:07:04 2018 - [info] Log messages from 192.168.56.104 ...
Tue Aug  7 04:07:04 2018 - [info] 
Tue Aug  7 04:07:03 2018 - [info] Starting recovery on 192.168.56.104(192.168.56.104:3306)..
Tue Aug  7 04:07:03 2018 - [info]  This server has all relay logs. Waiting all logs to be applied.. 
Tue Aug  7 04:07:03 2018 - [info]   done.
Tue Aug  7 04:07:03 2018 - [info]  All relay logs were successfully applied.
Tue Aug  7 04:07:03 2018 - [info]  Resetting slave 192.168.56.104(192.168.56.104:3306) and starting replication from the new master 192.168.56.103(192.168.56.103:3306)..
Tue Aug  7 04:07:03 2018 - [info]  Executed CHANGE MASTER.
Tue Aug  7 04:07:03 2018 - [info]  Slave started.
Tue Aug  7 04:07:04 2018 - [info] End of log messages from 192.168.56.104.
Tue Aug  7 04:07:04 2018 - [info] -- Slave recovery on host 192.168.56.104(192.168.56.104:3306) succeeded.
Tue Aug  7 04:07:04 2018 - [info] All new slave servers recovered successfully.
Tue Aug  7 04:07:04 2018 - [info] 
Tue Aug  7 04:07:04 2018 - [info] * Phase 5: New master cleanup phase..
Tue Aug  7 04:07:04 2018 - [info] 
Tue Aug  7 04:07:04 2018 - [info] Resetting slave info on the new master..
Tue Aug  7 04:07:04 2018 - [info]  192.168.56.103: Resetting slave info succeeded.
Tue Aug  7 04:07:04 2018 - [info] Master failover to 192.168.56.103(192.168.56.103:3306) completed successfully.
Tue Aug  7 04:07:04 2018 - [info] Deleted server1 entry from /etc/masterha/app1.cnf .
Tue Aug  7 04:07:04 2018 - [info] 

----- Failover Report -----

app1: MySQL Master failover 192.168.56.102(192.168.56.102:3306) to 192.168.56.103(192.168.56.103:3306) succeeded

Master 192.168.56.102(192.168.56.102:3306) is down!

Check MHA Manager logs at node1:/var/log/masterha/app1-manager.log for details.

Started automated(non-interactive) failover.
Invalidated master IP address on 192.168.56.102(192.168.56.102:3306)
The latest slave 192.168.56.103(192.168.56.103:3306) has all relay logs for recovery.
Selected 192.168.56.103(192.168.56.103:3306) as a new master.
192.168.56.103(192.168.56.103:3306): OK: Applying all logs succeeded.
192.168.56.103(192.168.56.103:3306): OK: Activated master IP address.
192.168.56.104(192.168.56.104:3306): This host has the latest relay log events.
Generating relay diff files from the latest slave succeeded.
192.168.56.104(192.168.56.104:3306): OK: Applying all logs succeeded. Slave started, replicating from 192.168.56.103(192.168.56.103:3306)
192.168.56.103(192.168.56.103:3306): Resetting slave info succeeded.
Master failover to 192.168.56.103(192.168.56.103:3306) completed successfully.

转载于:https://www.cnblogs.com/ctypyb2002/p/9792911.html

weixin_30376083
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
master_ip_failover
05-22
mha脚本 可以使用脚本 进行检测
master_ip_failover 测试可以使用的IP 地址切换脚本 --for Mysql-Master-Ha
weixin_33885676的博客
09-10 132
#!/usr/bin/env perl use strict; use warnings FATAL => 'all'; use Getopt::Long; my ( $command, $ssh_user, $orig_master_host, $orig_master_ip, $orig_maste...
MHA master_ip_failover
java_w的专栏
12-18 174
#!/usr/bin/env perl use strict; use warnings FATAL => 'all'; use Getopt::Long; my ( $command, $ssh_user, $orig_master_host, $orig_master_ip, $orig_master_port, $new_master_host, $new_ma...
MySQL高可用架构MHA 搭建测试文档_v1.docx
04-23
本文档详细的教你一步一步搭建mha+keepalived安装文档，小白按照该文档也可搭建出mha+keepalived高可用架构
MySQL高可用架构MHA
01-09
MHA工作原理 部署MHA 1.配置四台主机ssh登录不用密码 ssh-keygen -t rsa ssh-copy-id 192.168.100.10（20.30.102） 2.上传安装包，解压编译安装 配置yum仓库 vim /etc/yum.repos.d/mhapath.repo ##进入yum仓库，...
MySQL高可用学习笔记mysqlrouter_MHA.docx
08-14
该文档前半部分是一步一步怎么配置mysql_router,然后测试，后半部分是一步一步配置常用的MHA高可用工具和测试，文档详细清楚，基本上按照文档步骤一步步做下去都能配置成功
master_ip_failover&master_ip_online_change.rar
09-27
master_ip_failover mha切换主节点脚本 perl脚本
mha4mysql-node mha4mysql-manager master_ip_failover 0.56
02-08
mha4mysql-manager-0.56-0.el6.noarch.rpm mha4mysql-node-0.56-0.el6.noarch.rpm master_ip_failover
MySQL高可用方案 MHA之二 master_ip_failover
weixin_33725515的博客
12-10 798
异步主从复制架构master:10.150.20.90 ed3jrdba90slave:10.15.20.97 ed3jrdba9710.150.20.132 ed3jrdba132manager:10.150.20.95 ed3jrdba95#新增VIPvip:10.150.20.200 四台机器的系统情况：OS:CentOS7.3MySQL:5.7.21MHA:0.5...
mysql双master ip_MySQL高可用方案 MHA之二 master_ip_failover
weixin_29734123的博客
01-19 448
异步主从复制架构master:10.150.20.90 ed3jrdba90slave:10.15.20.97 ed3jrdba9710.150.20.132 ed3jrdba132manager:10.150.20.95 ed3jrdba95#新增VIPvip:10.150.20.200四台机器的系统情况：OS:CentOS7.3MySQL:5.7.21MHA:0.58网...
shell 实现MHA的master_ip_failover
lingyu-youmo的专栏
12-09 1239
#!/bin/bash #--------------第一部分:变量及函数定义-----------------# ###Begin Variables define### ssh_port=22 cmd=/sbin/ifconfig vip=172.16.4.116 device=eth0:0 netmk=255.255.255.0 start_vip="${cmd} ${d
master_ip_failover以及 master_ip_online_change
xys2333的博客
02-27 934
1.master_ip_failover #!/usr/bin/env perl use strict; use warnings FATAL =&gt; 'all'; use Getopt::Long; my ( $command, $ssh_user, $orig_master_host, $orig_master_ip, $orig_ma...
mysql mha ip_MySQL MHA之 master_ip_failover.sh脚本
weixin_39806679的博客
01-27 165
master_ip_failover.sh脚本是用perl编写的，可以在mha-manager源码包中可以找到，下面给出的结合keepalived进行自动切换的脚本：[root@mha scripts]# cat master_ip_failover.sh#!/usr/bin/env perl# Copyright (C) 2011 DeNA Co.,Ltd.## This program i...
MHA-6 两个脚本 master_ip_failover master_ip_online_change
m493096871的博客
02-26 527
master_ip_failover #!/usr/bin/env perl use strict; use warnings FATAL =&gt; 'all'; use Getopt::Long; my (     $command,          $ssh_user,        $orig_master_host, $orig_master_ip,     $orig_maste...
技术分享 | MHA-MasterFailover 分析
ActionTech的博客
05-08 144
作者：王向 爱可生 DBA 团队成员，负责公司 DMP 产品的运维和客户 MySQL 问题的处理。擅长数据库故障处理。对数据库技术和 python 有着浓厚的兴趣。 本文来源：原创投稿 * 爱可生开源社区出品，原创内容未经授权不得随意使用，转载请联系小编并注明来源。 前言 MHA 出来将近 10 年的时间，作为一个开源产品，能活这么久，还有这么多人追捧。基本上可以说是一种 mysql 中的标准解决方案。 不过 MHA 已经不适合这个时代了。。。但是这不影响我们对他进行一波研究。 发生 master cr.
MHA自动failover测试系列-master主机关机,MHA的切换
cnktbkanb141909486的博客
11-21 335
环境： Master：192.168.100.50 Slave1：192.168.100.48 Slave2：192.168.100.141 （MHA管理节点） [root@test11g ~]# masterha_...
master_ip_failover_script脚本何时被调用
FaN()
03-01 581
文章目录master_ip_failover_script脚本调用启动masterha_manager时Failover时调用两次 master_ip_failover_script脚本调用 master_ip_failover_script脚本是failover时用的. 默认的示例脚本其实就一个main函数, 判断--command的值做操作, 就收三种: stop || stopssh st...
mysql高可用架构mha
最新发布
08-30
MHA（Master High Availability）是一种用于MySQL数据库的高可用架构解决方案。它由日本DeNA公司开发，旨在实现MySQL主从复制的自动故障转移和故障恢复。 MHA架构通常由以下几个组件组成： 1. MHA Manager：负责...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值