Mysql主从

1. 主从简介

MySQL的主从是对数据进行存储备份，从而避免影响业务

1.1 主从作用

实时灾备，用于故障切换
读写分离，提供查询服务

备份，避免影响业务

1.2 主从形式

主从的形式有很多种，根据需求来设置

• 一主一从

• 主主复制

• 一主多从----扩展系统读取的性能，因为读是在从库读取的

• 多主一从----5.7开始支持

• 联级复制

2. 主从复制原理

主库将所有的写操作记录到binlog日志中；
从库生成两个线程，一个I/O线程，一个SQL线程； I/O线程去请求主库的binlog；
主库会生成一个log dump线程，用来给从库的I/O线程传binlog；
从库会将得到的binlog日志写到relay log(中继日志)文件中；
SQL线程会读取中继日志文件，并解析成具体的操作执行，这样主从的操作就一致了，而最终的数据也就一致了。

3.主从复制配置

3.1 主从部署的必要条件

主库开启binlog日志（设置log-bin参数）
主从server-id不同（主数据库的标识符要比从数据库的标识符小）
从库服务器能连同主库

3.2 主从复制配置步骤

确保从数据库与主数据库里的数据一样
在主数据库里创建一个同步账号授权给从数据库使用
配置主数据库（修改配置文件）
配置从数据库（修改配置文件）

3.3 MySQL主从配置

3.3.1 确保从数据库与主数据库里的数据一样

[root@localhost ~]# mysql
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.22 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| study              |
| sys                |
| zabbix             |
+--------------------+
6 rows in set (0.00 sec)


mysql> FLUSH TABLES WITH READ LOCK;
Query OK, 0 rows affected (0.00 sec)

备份主库并将备份文件传送并到从库

[root@localhost ~]# mysqldump --all-databases > all-backup201902281600
[root@localhost ~]# scp all-backup201902281600 root@192.168.19.150:/root
The authenticity of host '192.168.19.150 (192.168.19.150)' can't be established.
ECDSA key fingerprint is SHA256:joH96ssRXEsZ+xT5vuH3Hx7UkKqWpRy7RKag/EE9Xfk.
ECDSA key fingerprint is MD5:7a:e8:22:17:79:6e:50:10:00:95:a4:f4:7a:65:1e:8c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.19.150' (ECDSA) to the list of known hosts.
root@192.168.19.150's password: 
all-backup201902281600                                                   100% 4530KB  37.2MB/s   00:00    

检查从数据库

[root@localhost ~]# mysql
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.22 MySQL Community Server (GPL)

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
4 rows in set (0.00 sec)

在主数据库里创建一个同步账号授权给从数据库使用

	mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql> CREATE USER 'cwl'@'192.168.19.150' IDENTIFIED BY 'cwl123!';
Query OK, 0 rows affected (0.01 sec)

mysql> GRANT REPLICATION SLAVE ON *.* TO 'cwl'@'192.168.19.150';
Query OK, 0 rows affected (0.00 sec)

配置从数据库

mysql> CHANGE MASTER TO
    -> MASTER_HOST='192.168.19.100',
    -> MASTER_USER='cwl',
    -> MASTER_PASSWORD='cwl123!',
    -> MASTER_LOG_FILE='mysql-bin.000001',
    -> MASTER_LOG_POS=154;
Query OK, 0 rows affected, 2 warnings (0.01 sec)

mysql> start slave;
Query OK, 0 rows affected (0.00 sec)

mysql> show slave status \G
*************************** 1. row ***************************
               Slave_IO_State: Connecting to master
                  Master_Host: 192.168.19.100
                  Master_User: cwl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000001
          Read_Master_Log_Pos: 154
               Relay_Log_File: mysql-relay-log.000001
                Relay_Log_Pos: 4
        Relay_Master_Log_File: mysql-bin.000001
             Slave_IO_Running: Yes   //必须为Yes
            Slave_SQL_Running: Yes   //必须为Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 154
              Relay_Log_Space: 154
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 2003
                Last_IO_Error: error connecting to master 'cwl@192.168.19.100:3306' - retry-time: 60  retries: 1
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 0
                  Master_UUID: 
             Master_Info_File: /var/lib/mysql/master.info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 190228 17:16:49
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: 
                Auto_Position: 0
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
1 row in set (0.00 sec)

服务端数据做修改

mysql> select * from xs;
Empty set (0.00 sec)

mysql> insert into xs values(1,'cwl',23),(2,'wxl',24);
Query OK, 2 rows affected (0.00 sec)
Records: 2  Duplicates: 0  Warnings: 0

mysql> select * from xs;
+----+------+------+
| id | name | age  |
+----+------+------+
|  1 | cwl  |   23 |
|  2 | wxl  |   24 |
+----+------+------+
2 rows in set (0.00 sec)

客户端测试

mysql> use study;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> select *from xs;
+----+------+------+
| id | name | age  |
+----+------+------+
|  1 | cwl  |   23 |
|  2 | wxl  |   24 |
+----+------+------+
2 rows in set (0.00 sec)

GTID主从备份

GTID是一个基于原始mysql服务器生成的一个已经被成功执行的全局事务ID，它由服务器ID以及事务ID组合而成。这个全局事务ID不仅仅在原始服务器器上唯一，在所有存在主从关系 的mysql服务器上也是唯一的。正是因为这样一个特性使得mysql的主从复制变得更加简单，以及数据库一致性更可靠。

4.1 GTID的概念

全局事务标识：global transaction identifiers；
GTID是一个事务一一对应，并且全局唯一ID；
一个GTID在一个服务器上只执行一次，避免重复执行导致数据混乱或者主从不一致；
GTID用来代替传统复制方法，不在使用MASTER_LOG_FILE+MASTER_LOG_POS开启复制。而是使用MASTER_AUTO_POSTION=1的方式开始复制；
MySQL-5.6.5开始支持的，MySQL-5.6.10后开始完善；
在传统的slave端，binlog是不用开启的，但是在GTID中slave端的binlog是必须开启的，目的是记录执行过的GTID（强制）

。

4.2 GTID的组成

GTID = source_id:transaction_id

source_id，用于鉴别原服务器，即mysql服务器唯一的的server_uuid，由于GTID会传递到slave，所以也可以理解为源ID。
transaction_id，为当前服务器上已提交事务的一个序列号，通常从1开始自增长的序列，一个数值对应一个事务。
示例：

3E11FA47-71CA-11E1-9E33-C80AA9429562:23
前面的一串为服务器的server_uuid，即3E11FA47-71CA-11E1-9E33-C80AA9429562：23，后面的23为transaction_id

4.3 GTID的优势

更简单的实现failover，不用以前那样在需要找log_file和log_pos；

更简单的搭建主从复制；

比传统的复制更加安全；

GTID是连续的没有空洞的，保证数据的一致性，零丢失。

4.4 GTID的工作原理

当一个事务在主库端执行并提交时，产生GTID，一同记录到binlog日志中；
binlog传输到slave,并存储到slave的relaylog后，读取这个GTID的这个值设置gtid_next变量，即告诉Slave，下一个要执行的GTID值；
sql线程从relay log中获取GTID，然后对比slave端的binlog是否有该GTID；
如果有记录，说明该GTID的事务已经执行，slave会忽略；
如果没有记录，slave就会执行该GTID事务，并记录该GTID到自身的binlog，在读取执行事务前会先检查其他session持有该GTID，确保不被重复执行；
在解析过程中会判断是否有主键，如果没有就用二级索引，如果没有就用全部扫描。

4.6 GTID主从的配置

在主库上做配置

[root@localhost ~]# vim /etc/my.cnf
[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
pid-file = /opt/data/mysql.pid
user = mysql
skip-name-resolve
skip-grant-tables
server-id = 5
log-bin = mysql_log
gtid_mode = ON
enforce-gtid-consistency = true

查看数据库
mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 |     154  |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)

在从库上做配置

[root@localhost ~]# vim /etc/my.cnf
[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
pid-file = /opt/data/mysql.pid
user = mysql
skip-name-resolve
server-id = 6
relay-log = mysql-relay-log
gtid_mode = ON
enforce-gtid-consistency = true

配置数据库
mysql> stop slave;
Query OK, 0 rows affected (0.00 sec)

mysql> change master to master_host='192.168.19.100',master_user='cwl',master_password='cwl123!',master_autoo_position=1;
Query OK, 0 rows affected, 2 warnings (0.00 sec)

mysql> start slave;
Query OK, 0 rows affected (0.00 sec)

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.19.100
                  Master_User: cwl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql_log.000001
          Read_Master_Log_Pos: 154
               Relay_Log_File: mysql-relay-log.000002
                Relay_Log_Pos: 367
        Relay_Master_Log_File: mysql_log.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 154
              Relay_Log_Space: 574
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 5
                  Master_UUID: b0e84843-384f-11e9-a559-000c296bbfd6
             Master_Info_File: /var/lib/mysql/master.info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: 
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
1 row in set (0.00 sec)

在主库上修改数据

mysql> use study;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed

mysql> insert into study value(3,'wdl',30)；
Query OK, 1 row affected (0.00 sec)
mysql> select * from study.xs;
+----+------+------+
| id | name | age  |
+----+------+------+
|  1 | cwl  |   23 |
|  2 | wxl  |   24 |
|  3 | wdl  |   30 |
+----+------+------+
3 rows in set (0.00 sec)

在从库上测试结果

mysql> select * from study.xs;
+----+------+------+
| id | name | age  |
+----+------+------+
|  1 | cwl  |   23 |
|  2 | wxl  |   24 |
|  3 | wdl  |   30 |
+----+------+------+
3 rows in set (0.00 sec)

用zabbix监测mysql主从故障

[root@localhost scripts]# vim my.sh 
#!/bin/bash
a=$(mysql -uroot -pming123 -e 'show slave status \G' 2>/dev/null |grep Slave_IO_Running|awk -F ': ' '{print $2}')
b=$(mysql -uroot -pming123  -e 'show slave status \G' 2>/dev/null |grep Slave_SQL_Running:|awk -F ': ' '{print $2}')
c=$(echo $a $b)
echo $c|grep 'Yes'|wc -w

[root@localhost scripts]# chmod +x my.sh 
[root@localhost scripts]# chown -R zabbix.zabbix my.sh 
[root@localhost scripts]# ll
total 8
-rwxr-xr-x. 1 zabbix zabbix 258 Feb 28 20:37 my.sh

[root@localhost scripts]# vim /usr/local/etc/zabbix_agentd.co
UserParameter=check_mysql,/scripts/1.sh 

用zabbix监测mysql主从延迟

[root@localhost ~]# vim /scripts/yc.sh 
#!/bin/bash
#!/bin/bash
#!/bin/bash
a=$( mysql -e 'show slave status \G'|grep  Exec_Master_Log_Pos|awk -F ': ' '{print $2}')
b=$(mysql -e 'show slave status \G'|grep Read_Master_Log_Pos:|awk -F ': ' '{print $2}')
c=$[ $a - $b ]
if [ $c -ne 0 ];then
  echo 1
else
    echo 0
fi 

[root@localhost scripts]# chmod +x yc.sh
[root@localhost scripts]# chown -R zabbix.zabbix yc.sh 
[root@localhost scripts]# ll
total 8
-rwxr-xr-x. 1 zabbix zabbix 258 Feb 28 20:37 my.sh
-rwxr-xr-x. 1 zabbix zabbix 279 Feb 28 20:54 yc.sh

[root@localhost ~]# vim /usr/local/etc/zabbix_agentd.conf
UserParameter=check_mysql1,/scripts/yc.sh