1. 环境信息
ip | 主机名 | mysqld端口 | replication通信端口 | 安装的软件 | 备注 |
10.45.53.31 | rac2 | 3306 | 33061 | mysql,mysql-shell | mgr多主集群 |
172.16.23.170 | shard12 | 3306 | 33061 | mysql,mysql-shell | mgr多主集群 |
172.16.23.171 | catdb | 3306 | 33061 | mysql,mysql-shell | mgr多主集群 |
10.45.53.30 | rac1 | null | null | mysql客户端,mysql-route | 测试mysql-route和读写分离 |
2. mysql-shell三节点安装配置
(1)安装rpm包
rpm -ivh mysql-shell-commercial-8.0.19-1.1.el7.x86_64
rpm -ivh mysql-shell-commercial-debuginfo-8.0.19-1.1.el7.x86_64
(2)建myshell用户,供连接mysql-shell,或者直接用root
GRANT SELECT ON mysql_innodb_cluster_metadata.* TO myshell@'%' identified by '123';
GRANT SELECT ON performance_schema.global_status TO myshell@'%';
GRANT SELECT ON performance_schema.replication_applier_configuration TO myshell@'%';
GRANT SELECT ON performance_schema.replication_applier_status TO myshell@'%';
GRANT SELECT ON performance_schema.replication_applier_status_by_coordinator TO myshell@'%';
GRANT SELECT ON performance_schema.replication_applier_status_by_worker TO myshell@'%';
GRANT SELECT ON performance_schema.replication_connection_configuration TO myshell@'%';
GRANT SELECT ON performance_schema.replication_connection_status TO myshell@'%';
GRANT SELECT ON performance_schema.replication_group_member_stats TO myshell@'%';
GRANT SELECT ON performance_schema.replication_group_members TO myshell@'%';
GRANT SELECT ON performance_schema.threads TO myshell@'%' WITH GRANT OPTION;
GRANT CREATE USER, FILE, PROCESS, RELOAD, REPLICATION CLIENT, REPLICATION SLAVE, SELECT, SHUTDOWN, SUPER ON *.* TO 'myshell'@'%' WITH GRANT OPTION;
GRANT DELETE, INSERT, UPDATE ON mysql.* TO 'myshell'@'%' WITH GRANT OPTION;
GRANT ALTER, ALTER ROUTINE, CREATE, CREATE ROUTINE, CREATE TEMPORARY TABLES, CREATE VIEW, DELETE, DROP, EVENT, EXECUTE, INDEX, INSERT, LOCK TABLES, REFERENCES, SHOW VIEW, TRIGGER, UPDATE ON mysql_innodb_cluster_metadata.* TO 'myshell'@'%' WITH GRANT OPTION;
GRANT ALTER, ALTER ROUTINE, CREATE, CREATE ROUTINE, CREATE TEMPORARY TABLES, CREATE VIEW, DELETE, DROP, EVENT, EXECUTE, INDEX, INSERT, LOCK TABLES, REFERENCES, SHOW VIEW, TRIGGER, UPDATE ON mysql_innodb_cluster_metadata_bkp.* TO 'myshell'@'%' WITH GRANT OPTION;
GRANT ALTER, ALTER ROUTINE, CREATE, CREATE ROUTINE, CREATE TEMPORARY TABLES, CREATE VIEW, DELETE, DROP, EVENT, EXECUTE, INDEX, INSERT, LOCK TABLES, REFERENCES, SHOW VIEW, TRIGGER, UPDATE ON mysql_innodb_cluster_metadata_previous.* TO 'myshell'@'%' WITH GRANT OPTION;
(3)检查三节点是否符合创建cluster
先停三节点组复制
STOP GROUP_REPLICATION;
节点一进入mysql-shell检查参数
mysqlsh myshell@rac2:3306
dba.checkInstanceConfiguration('myshell@rac2:3306')
若参数符合条件,检查结果如下
如果参数不通过,会有推荐值,建议修改my.cnf配置文件重启mysql或直接在线修改
节点二和节点三也可以在rac2主机上检查配置是否通过
dba.checkInstanceConfiguration('myshell@shard12:3306')
dba.checkInstanceConfiguration('myshell@catdb:3306')
(4)在一节点创建cluster--prodCluster
mysqlsh myshell@rac2:3306
var cluster = dba.createCluster('prodCluster')
创建的过程中,会有提示warning,说是需要执行dba.configureLocalInstance()来完善,这个命令三节点只能在各自的主机上进mysql-shell执行,执行过之后,会更新一些参数和修改完善/etc/my.cnf参数
dba.configureLocalInstance(myshell@rac2:3306)
(5)添加其他节点的实例信息
cluster.addInstance('myshell@shard12:3306');
添加的过程中,会有提示warning,说是需要执行dba.configureLocalInstance()来完善,这个命令三节点只能在各自的主机上进mysql-shell执行,执行过之后,会更新一些参数和修改完善/etc/my.cnf参数dba.configureLocalInstance('myshell@shard12:3306')
cluster.addInstance('myshell@catdb:3306')
由于catdb的组复制停的最晚,且其他节点的组复制停了之后,我又在catdb上做了dml操作,添加节点到集群时报错如下,意思是添加节点的gtid不能比cluster的其他两节点大
修复过程:mysqldump对一节点备份, mysqldump -uroot -p123 --single-transaction --master-data=2 --all-databases > all.sql,导入到catdb时报ERROR 1840 (HY000): @@GLOBAL.GTID_PURGED can only be set when @@GLOBAL.GTID_EXECUTED is empty,通过如下命令将GLOBAL.GTID_EXECUTED置空,重新导入成功
reset master;
source all.sql
然后重新将catdb通过rac2主机加入cluster,cluster.addInstance('myshell@catdb:3306')
catdb上执行:dba.configureLocalInstance('myshell@catdb:3306')
(6)查看cluster状态和配置
mysqlsh myshell@catdb:3306
var cluster = dba.getCluster()
cluster.status();
cluster.describe();
如果三节点均down,起了两个节点,另一节点没起,需要cluster.rescan()来重新定义cluster,第三个节点起了后,再重新加入集群
cluster.rescan()
cluster.addInstance()
mgr组复制已经自动被拉起来了
至此mysql innodb cluster部署完成,下面安装mysql-route实现集群的读写分离(负载均衡)
3.mysql-route安装配置,测试
(1)rpm安装mysql-route
The recommended deployment of MySQL Router is on the same host as the application.
When using a production deployment, we recommend deploying one MySQL Router instance to each machine used to host one of your client applications.
生产环境中建议将mysql-route安装在应用机器上,并且每一种应用安装配置一个mysql-route
rpm -ivh mysql-router-commercial-8.0.19-1.1.el7.x86_64.rpm
(2)配置 /etc/mysqlrouter/mysqlrouter.conf
RPM and Debian : /etc/mysqlrouter/mysqlrouter.conf
安装会自动生成/etc/mysqlrouter/mysqlrouter.conf,建立/run/mysqlrouter,/usr/lib64/mysqlrouter, /var/log/mysqlrouter
mysqlrouter --help 可以看到默认mysqlrouter的配置文件
多主模式下的mgr,可以这样配置 /etc/mysqlrouter/mysqlrouter.conf
# MySQL Router configuration file
#
# Documentation is available at
# http://dev.mysql.com/doc/mysql-router/en/
[DEFAULT]
logging_folder = /var/log/mysqlrouter
runtime_folder = /var/run/mysqlrouter
config_folder = /etc/mysqlrouter
connect_timeout = 30
read_timeout=30
#Maximum number of connections assigned to a routed destination MySQL server
max_connections = 1024
protocol=classic
[logger]
level = INFO
[routing:rw_mode]
#bind_address默认是127.0.0.1
#多主节点可以配置写负载均衡,将bind_address设为0.0.0.0
bind_address=0.0.0.0
#读写端口6446
bind_port=6446
mode = read-write
destinations = 10.45.53.31:3306,172.16.23.170:3306,172.16.23.171:3306
#根据group_replication_member_weight设置destinations三节点的顺序,否则读写模式访问从节点会报read-only #错而不会转到下一个节点
routing_strategy = round-robin
## read-write模式下:路由策略多主模式可以将其设为round-robin,单主模式可设first-available
[routing:ro_mode]
bind_address = 0.0.0.0
#读端口6447
bind_port = 6447
mode = read-only
destinations = 10.45.53.31:3306,172.16.23.170:3306,172.16.23.171:3306
routing_strategy = round-robin
## read-only模式下:单主模式和多主模式可设路由策略为round-robin
When using a single-primary cluster, read-write sessions are redirected to the single primary,
with a multi-primary cluster read-write sessions are redirected to one of the primary instances;
For incoming read-only connections MySQL Router redirects connections to one of the secondary instances in a round-robin fashion.
(轮询调度算法) 原理是每一次把来自用户的请求轮流分配给内部中的服务器,从1开始,直到N(内部服务器个数),然后重新开始循环
(3)启动mysql-router
service mysqlrouter start
6447和6446端口已在rac1(10.45.53.30)上启动
(4)测试
创建测试用户
mysql> create user root@'%' identified by '123';
mysql> grant all privileges on *.* to root@'%';
读测试:
for i in `seq 1 10`; do mysql -h 10.45.53.30 -P6447 -uroot -p123 -e "select @@hostname;"; done
可看出来读是轮询访问10.45.53.31:3306,172.16.23.170:3306,172.16.23.171:3306
写测试:
create table test2 (id int auto_increment primary key);
for i in `seq 1 10`; do mysql -h 10.45.53.30 -P6446 -uroot -p123 -e "insert into test.test2 values(null);"; done
由于三节点的auto_increment_offset值分别为2,3,4,auto_increment_increment为7,可以看出写操作轮询访问了三节点
4667端口不是只能读,多主模式下访问4667端口可读写;单主模式下,取决于mysqlrouter.conf的[routing:ro_mode]是否配置了主节点,若配置了,则可读写,若没配置,只可读,通常情况下是配置的
高可用测试
To test if high availability works, simulate an unexpected halt by killing an instance. The cluster detects the fact that the instance left the cluster and reconfigures itself. Exactly how the cluster reconfigures itself depends on whether you are using a single-primary or multi-primary cluster, and the role the instance serves within the cluster.
In single-primary mode:
If the current primary leaves the cluster, one of the secondary instances is elected as the new primary, with instances prioritized by the lowest server_uuid. MySQL Router redirects read-write connections to the newly elected primary.
If a current secondary leaves the cluster, MySQL Router stops redirecting read-only connections to the instance.
In multi-primary mode:
If a current "R/W" instance leaves the cluster, MySQL Router redirects read-write connections to other primaries.
If a current "R/O" instance leaves the cluster,MySQL Router stops redirecting read-only connections to the instance.
多主模式下停一个节点
停两个节点
其他
新增节点后需要修改conf文件并重新启动mysql-route
When MySQL Router is bootstrapped against a cluster, it records the server instance's addresses in its configuration file. If any additional instances are added to the cluster after bootstrapping the MySQL Router, they are not automatically detected and therefore are not used for connection routing.
To ensure that newly added instances are routed to correctly you must bootstrap MySQL Router against the cluster to read the updated metadata. This means that you must restart MySQL Router, include the --bootstrap option or modify /etc/mysqlrouter/mysqlrouter.conf.
实际上mysql-shell没什么大用处,可以不安装mysql-shell,只安装mysql-router即可实现读写分离
参考文档
https://dev.mysql.com/doc/refman/5.7/en/mysql-innodb-cluster-working-with-cluster.html