一.概念解释
1.数据库是按照数据结构来组织、存储和管理数据的仓库。每个数据库都有一个或多个不同的 API 用于创建,访问,管理,搜索和复制所保存的数据。我们也可以将数据存储在文件中,但是在文件中读写数据速度相对较慢。
2.mysql主从复制的原理:mysql要做到主从复制,其实依靠的是二进制日志,即:假设主服务器叫A,从服务器叫B;主从复制就是B跟着A学,A做什么,B就做什么。那么B怎么同步A的动作呢?现在A有一个日志功能,把自己所做的增删改查的动作。全都记录在日志中,B只需要拿到这份日志,照着日志上面的动作施加到自己身上就可以了。这样就实现了主从复制。
3.GDIT介绍:MySQL-5.6.2开始支持,MySQL-5.6.10后完善,GTID 分成两部分,一部分是服务的UUid,UUID保存在mysql数据目录的auto.cnf文件中,这是一个非常重要的文件,不能删除,这一部分是不会变的。另外一部分就是事务ID了,随着事务的增加,值一次递增,如下图
4.GTID的工作原理:
1、master更新数据时,会在事务前产生GTID,一同记录到binlog日志中。
2、slave端的i/o 线程将变更的binlog,写入到本地的relay log中。
3、sql线程从relay log中获取GTID,然后对比slave端的binlog是否有记录。
4、如果有记录,说明该GTID的事务已经执行,slave会忽略。
5、如果没有记录,slave就会从relay log中执行该GTID的事务,并记录到binlog。
6、在解析过程中会判断是否有主键,如果没有就用二级索引,如果没有就用全部扫描。
5. GTID的优点:
1.一个事务对应一个唯一ID,一个GTID在一个服务器上只会执行一次
2.GTID是用来代替传统复制的方法,GTID复制与普通复制模式的最大不同就是不需要指定二进制文件名和位置
3.减少手工干预和降低服务故障时间,当主机挂了之后通过软件从众多的备机中提升一台备机为主机
6.主从同步复制有以下几种方式:
(1)同步复制,master的变化,必须等待slave-1,slave-2,…,slave-n完成后才能返回。
(2)异步复制,master只需要完成自己的数据库操作即可,至于slaves是否收到二进制日志,是否完成操作,不用关心。MYSQL的默认设置。
(3)半同步复制,master只保证slaves中的一个操作成功,就返回,其他slave不管。这个功能,是由google为MYSQL引入的。
二.配置数据库的主从
1.主从数据库流程
1)一个主库负责写请求,更新数据
2)两个从库负责读请求,可以提高系统吞吐量
3)主库和从库之间同步数据
一般主从复制,有三个线程参与,都是单线程:Binlog Dump(主) —–>IO Thread (从) —–> SQL Thread(从)。
复制只能是单向的,从主节点(master)到从节点(slave)上.
master用户写入数据,生成event记录到binary-log(二进制日志)中.
slave接收master上传来的binlog,然后按顺序应用,重现master上的用户操作.
上述会产生数据不同步的问题:
解决方法:
全同步复制机制(组复制)、半同步复制、异步复制三种复制方案
2.主数据的配置
1.获取mysql安装包(可从官网上下载),并安装。
2.数据库安全初始化(密码必须要由数字,大小写字母,特殊字母组成,四者缺一不可,且必须大于8位)
[root@server1 ~]# systemctl start mysqld
#查看密码的两种方式
[root@server1 ~]# cat /var/log/mysqld.log | grep password
[root@server1 ~]# grep password /var/log/mysqld.log
3.编写mysql的配置文件并重启服务
[root@server1 ~]# vim /etc/my.cnf
28 log-bin=mysql-bin #打开二进制日志
29 server-id=1 #设置ID不能与别的主机重复
[root@server1 ~]# systemctl restart mysqld
4.创建用户并授权
[root@server1 mysql]# mysql -uroot -pWest+123redhat
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.7.24-log MySQL Community Server (GPL)
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
+--------------------+
4 rows in set (0.00 sec)
mysql> create user 'repl'@'172.25.16.%' identified by 'West+123redhat';
Query OK, 0 rows affected (0.01 sec)
mysql> grant replication slave on *.* to 'repl' @'172.25.16.%';
Query OK, 0 rows affected (0.00 sec)
名词解释:
replication 表示授权复制的权限
*.*表示所有数据库可以进行同步
repl表示授权名,可以随意填写
'172.25.16.%'表示授权
172.25.16.0/24的网段所有服务器可以同步,%表示任意
3.从库的配置
1.安装软件,2.数据里安全初始化(与主库:server1相同操作)
3.查看repl能否远程登陆
4.编辑配置文件并重新开启mysqld服务
[root@server2 ~]# vim /etc/my.cnf
28 server-id=2 #server-id与主库要区别开
[root@server2 ~]# systemctl restart mysqld
5.关联主从
[root@server2 mysql]# mysql -uroot -pWest+123redhat
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 6
Server version: 5.7.24 MySQL Community Server (GPL)
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> change master to
-> master_host='172.25.16.1',
-> master_user='repl',
-> master_password='West+123redhat'
-> ,
-> master_log_file='mysql-bin.000002',
-> master_log_pos=1011;
Query OK, 0 rows affected, 2 warnings (0.08 sec)
6.开启从并查看状态
mysql> start slave;
Query OK, 0 rows affected (0.00 sec)
mysql> show slave status\G;
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 172.25.16.1
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000002
Read_Master_Log_Pos: 1011
Relay_Log_File: server2-relay-bin.000002
Relay_Log_Pos: 320
Relay_Master_Log_File: mysql-bin.000002
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 1011
Relay_Log_Space: 529
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
Master_UUID: 198bdb31-b4fb-11e9-b08e-5254004772f0
Master_Info_File: /var/lib/mysql/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
3.测试
1.在主数据库中创建库,创建表,并插入数据
mysql> create database westos;
Query OK, 1 row affected (0.01 sec)
mysql> use westos;
Database changed
mysql> create table userlist (
-> username varchar(10) not null,
-> password varchar(15) not null);
Query OK, 0 rows affected (0.07 sec)
mysql> insert into userlist values ('user1','123');
Query OK, 1 row affected (0.09 sec)
mysql> select * from userlist;
+----------+----------+
| username | password |
+----------+----------+
| user1 | 123 |
+----------+----------+
2.在从库数据复制成功
三.基于GDIT主从复制
1.配置GDIT主从复制
1.修改主库和从库的mysql配置文件
vim /etc/my.cnf
systemctl restart mysqld
2.在从库中先停掉slave,然后重新创建连接(slave连接master时将使用基于GTID的复制协议,所以将MASTER_AUTO_POSITION=1)
2.测试
在主库userlist添加信息
从库同步信息
四.基于GDIT的半同步复制
1.配置半同步
1.主库:
mysql> install plugin rpl_semi_sync_master soname 'semisync_master.so';
Query OK, 0 rows affected (0.17 sec)
mysql> set global rpl_semi_sync_master_enabled=ON;
Query OK, 0 rows affected (0.00 sec)
mysql> select plugin_name,plugin_status
-> from information_schema.plugins
-> where plugin_name like '%semi%';
+----------------------+---------------+
| plugin_name | plugin_status |
+----------------------+---------------+
| rpl_semi_sync_master | ACTIVE |
+----------------------+---------------+
1 row in set (0.00 sec)
mysql> show variables like '%rpl%';
+-------------------------------------------+------------+
| Variable_name | Value |
+-------------------------------------------+------------+
| rpl_semi_sync_master_enabled | ON |
| rpl_semi_sync_master_timeout | 10000 |
| rpl_semi_sync_master_trace_level | 32 |
| rpl_semi_sync_master_wait_for_slave_count | 1 |
| rpl_semi_sync_master_wait_no_slave | ON |
| rpl_semi_sync_master_wait_point | AFTER_SYNC |
| rpl_stop_slave_timeout | 31536000 |
+-------------------------------------------+------------+
7 rows in set (0.00 sec)
mysql> show variables like '%semi%';
+-------------------------------------------+------------+
| Variable_name | Value |
+-------------------------------------------+------------+
| rpl_semi_sync_master_enabled | ON |
| rpl_semi_sync_master_timeout | 10000 |
| rpl_semi_sync_master_trace_level | 32 |
| rpl_semi_sync_master_wait_for_slave_count | 1 |
| rpl_semi_sync_master_wait_no_slave | ON |
| rpl_semi_sync_master_wait_point | AFTER_SYNC |
+-------------------------------------------+------------+
6 rows in set (0.01 sec)
mysql> show status like '%rpl_semi_sync%';
+--------------------------------------------+-------+
| Variable_name | Value |
+--------------------------------------------+-------+
| Rpl_semi_sync_master_clients | 0 |
| Rpl_semi_sync_master_net_avg_wait_time | 0 |
| Rpl_semi_sync_master_net_wait_time | 0 |
| Rpl_semi_sync_master_net_waits | 0 |
| Rpl_semi_sync_master_no_times | 0 |
| Rpl_semi_sync_master_no_tx | 0 |
| Rpl_semi_sync_master_status | ON |
| Rpl_semi_sync_master_timefunc_failures | 0 |
| Rpl_semi_sync_master_tx_avg_wait_time | 0 |
| Rpl_semi_sync_master_tx_wait_time | 0 |
| Rpl_semi_sync_master_tx_waits | 0 |
| Rpl_semi_sync_master_wait_pos_backtraverse | 0 |
| Rpl_semi_sync_master_wait_sessions | 0 |
| Rpl_semi_sync_master_yes_tx | 0 |
+--------------------------------------------+--
2.从库安装半同步插件
mysql> install plugin rpl_semi_sync_slave soname 'semisync_slave.so';
Query OK, 0 rows affected (0.01 sec)
mysql> select plugin_name,plugin_status
-> from information_schema.plugins
-> where plugin_name like '%semi%';
+---------------------+---------------+
| plugin_name | plugin_status |
+---------------------+---------------+
| rpl_semi_sync_slave | ACTIVE |
+---------------------+---------------+
1 row in set (0.00 sec)
mysql> set global rpl_semi_sync_slave_enabled=1;
Query OK, 0 rows affected (0.00 sec)
mysql> stop slave io_thread;
Query OK, 0 rows affected (0.01 sec)
mysql> start slave io_thread;
Query OK, 0 rows affected (0.00 sec)
mysql> show variables like '%semi%';
+---------------------------------+-------+
| Variable_name | Value |
+---------------------------------+-------+
| rpl_semi_sync_slave_enabled | ON |
| rpl_semi_sync_slave_trace_level | 32 |
+---------------------------------+-------+
2 rows in set (0.00 sec)
mysql> show variables like '%rpl%';
+---------------------------------+----------+
| Variable_name | Value |
+---------------------------------+----------+
| rpl_semi_sync_slave_enabled | ON |
| rpl_semi_sync_slave_trace_level | 32 |
| rpl_stop_slave_timeout | 31536000 |
+---------------------------------+----------+
3 rows in set (0.00 sec)
mysql> show status like '%rpl%';
+----------------------------+-------+
| Variable_name | Value |
+----------------------------+-------+
| Rpl_semi_sync_slave_status | ON |
+----------------------------+-------+
1 row in set (0.00 sec)
mysql> stop slave io_thread;
mysql> start slave io_thread;
注意:重启从库上的IO线程,如果不重启,则默认还是异步复制,重启后,slave会在master上注册为半同步复制的slave角色
测试:
半同步失败
1.关闭从库io进程
2主库插入数据,等待10s(可以设置,这里默认是10s)10s后会变成异步复制;主库中发现半同步失败次数+1
mysql> use westos;
mysql> insert into userlist values('user3','333');
Rpl_semi_sync_master_no_tx表示没有成功接收slave提交的次数,也就是使用半同步失败的次数,10s后没有得到反馈信息,会转为异步复制
此后从库没有更新数据
打开从库io进程,稍等一会发现同步数据成功