mysql数据库的主从配置读写分离
现在的大型系统为了实现大规模的数据并发。一个可以调整应用架构。一个是解决数据的并发问题。而数据库的主从配置跟读写分离就能很好的解决这个问题。
概述:搭设一台Master服务器(win7系统,Ip:localhost/ 172.17.0.41),搭设一台Slave服务器(虚拟机——一台Ubuntu 172.17.0.128)。
原理:主服务器(Master)负责网站写操作,从服务器负责查询操作。主从服务器利用MySQL的二进制日志文件,实现数据同步。二进制日志由主服务器产生,从服务器响应获取同步数据库。
实现:
1、windows我安装的是mysql_5.5.27.msi,ubantu安装的是mysql-5.6.22-linux-glibc2.5-i686.tar
安装数据的步骤这里不在赘述。可以自己百度步骤。
先说一下数据库的读写分离
要实现mysql的读写分离需要第三方的中间件。如ameoba
请自行下载ameoba、有关ameoba的知识请自行百度学习。这是国人自己开发的中间件。有详细的文档介绍。
配置ameoba
<?xml version="1.0" encoding="gbk"?>
<!DOCTYPE amoeba:configuration SYSTEM "amoeba.dtd">
<amoeba:configuration xmlns:amoeba="http://amoeba.meidusa.com/">
<proxy>
<!-- service class must implements com.meidusa.amoeba.service.Service -->
<service name="Amoeba for Mysql" class="com.meidusa.amoeba.net.ServerableConnectionManager">
<!-- port -->
<property name="port">8066</property>
<!-- bind ipAddress -->
<property name="ipAddress">127.0.0.1</property>
<property name="manager">${clientConnectioneManager}</property>
<property name="connectionFactory">
<bean class="com.meidusa.amoeba.mysql.net.MysqlClientConnectionFactory">
<property name="sendBufferSize">128</property>
<property name="receiveBufferSize">64</property>
</bean>
</property>
<property name="authenticator">
<bean class="com.meidusa.amoeba.mysql.server.MysqlClientAuthenticator">
<property name="user">root</property>
<property name="password">root</property>
<property name="filter">
<bean class="com.meidusa.amoeba.server.IPAccessController">
<property name="ipFile">${amoeba.home}/conf/access_list.conf</property>
</bean>
</property>
</bean>
</property>
</service>
<!-- server class must implements com.meidusa.amoeba.service.Service -->
<service name="Amoeba Monitor Server" class="com.meidusa.amoeba.monitor.MonitorServer">
<!-- port -->
<!-- default value: random number
<property name="port">9066</property>
-->
<!-- bind ipAddress -->
<property name="ipAddress">127.0.0.1</property>
<property name="daemon">true</property>
<property name="manager">${clientConnectioneManager}</property>
<property name="connectionFactory">
<bean class="com.meidusa.amoeba.monitor.net.MonitorClientConnectionFactory"></bean>
</property>
</service>
<runtime class="com.meidusa.amoeba.mysql.context.MysqlRuntimeContext">
<!-- proxy server net IO Read thread size -->
<property name="readThreadPoolSize">20</property>
<!-- proxy server client process thread size -->
<property name="clientSideThreadPoolSize">30</property>
<!-- mysql server data packet process thread size -->
<property name="serverSideThreadPoolSize">30</property>
<!-- per connection cache prepared statement size -->
<property name="statementCacheSize">500</property>
<!-- query timeout( default: 60 second , TimeUnit:second) -->
<property name="queryTimeout">60</property>
</runtime>
</proxy>
<!--
Each ConnectionManager will start as thread
manager responsible for the Connection IO read , Death Detection
-->
<connectionManagerList>
<connectionManager name="clientConnectioneManager" class="com.meidusa.amoeba.net.MultiConnectionManagerWrapper">
<property name="subManagerClassName">com.meidusa.amoeba.net.ConnectionManager</property>
<!--
default value is avaliable Processors
<property name="processors">5</property>
-->
</connectionManager>
<connectionManager name="defaultManager" class="com.meidusa.amoeba.net.MultiConnectionManagerWrapper">
<property name="subManagerClassName">com.meidusa.amoeba.net.AuthingableConnectionManager</property>
<!--
default value is avaliable Processors
<property name="processors">5</property>
-->
</connectionManager>
</connectionManagerList>
<!-- default using file loader -->
<dbServerLoader class="com.meidusa.amoeba.context.DBServerConfigFileLoader">
<property name="configFile">${amoeba.home}/conf/dbServers.xml</property>
</dbServerLoader>
<queryRouter class="com.meidusa.amoeba.mysql.parser.MysqlQueryRouter">
<property name="ruleLoader">
<bean class="com.meidusa.amoeba.route.TableRuleFileLoader">
<property name="ruleFile">${amoeba.home}/conf/rule.xml</property>
<property name="functionFile">${amoeba.home}/conf/ruleFunctionMap.xml</property>
</bean>
</property>
<property name="sqlFunctionFile">${amoeba.home}/conf/functionMap.xml</property>
<property name="LRUMapSize">1500</property>
<property name="defaultPool">master</property>
配置读写库
<property name="writePool">master</property>
<property name="readPool">multiPool</property>
<property name="needParse">true</property>
</queryRouter>
</amoeba:configuration>
<?xml version="1.0" encoding="gbk"?>
dbServers的配置
<!DOCTYPE amoeba:dbServers SYSTEM "dbserver.dtd">
<amoeba:dbServers xmlns:amoeba="http://amoeba.meidusa.com/">
<!--
Each dbServer needs to be configured into a Pool,
If you need to configure multiple dbServer with load balancing that can be simplified by the following configuration:
add attribute with name virtual = "true" in dbServer, but the configuration does not allow the element with name factoryConfig
such as 'multiPool' dbServer
-->
<dbServer name="abstractServer" abstractive="true">
<factoryConfig class="com.meidusa.amoeba.mysql.net.MysqlServerConnectionFactory">
<property name="manager">${defaultManager}</property>
<property name="sendBufferSize">64</property>
<property name="receiveBufferSize">128</property>
<!-- mysql port -->
<property name="port">3306</property>
<!-- mysql schema -->
<property name="schema">mysql</property>
<!-- mysql user -->
<property name="user">root</property>
<!-- mysql password-->
<property name="password">root</property>
</factoryConfig>
<poolConfig class="com.meidusa.amoeba.net.poolable.PoolableObjectPool">
<property name="maxActive">500</property>
<property name="maxIdle">500</property>
<property name="minIdle">10</property>
<property name="minEvictableIdleTimeMillis">600000</property>
<property name="timeBetweenEvictionRunsMillis">600000</property>
<property name="testOnBorrow">true</property>
<property name="testOnReturn">true</property>
<property name="testWhileIdle">true</property>
</poolConfig>
</dbServer>
<dbServer name="master" parent="abstractServer">
<factoryConfig>
<!-- mysql ip -->
<property name="ipAddress">127.0.0.1</property>
</factoryConfig>
</dbServer>
<dbServer name="slave" parent="abstractServer">
<factoryConfig>
<!-- mysql ip -->
<property name="ipAddress">172.17.0.128</property>
</factoryConfig>
</dbServer>
<dbServer name="multiPool" virtual="true">
<poolConfig class="com.meidusa.amoeba.server.MultipleServerPool">
<!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA-->
<property name="loadbalance">1</property>
<!-- Separated by commas,such as: server1,server2,server1 -->
<property name="poolNames">master,slave</property>
</poolConfig>
</dbServer>
</amoeba:dbServers>
在主库上执行授权
grant all on *.* to'root'@'172.17.%.%' identified by 'root';
grant all on *.* to'root'@'%mysql.com' identified by 'root';
flush privileges;
验证
两个数据库都有test_test表都要为空表
在dos执行连接进行验证。这里连接到amoeba
mysql -h127.0.0.1 -uroot -p -P8806
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 2097086015
Server version: 5.1.45-mysql-amoeba-proxy-2.2.0 Source distribution
Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
先对test_test表执行插入语句
insert into test_test (id.name)values('1','ddd');
之后在启动新的dos窗口登入主库master查询会有一条数据插入
而从库是没有数据插入的
从amoeba查询也是空表。这样基本实现了读写分离
2、配置数据的主从
配置Master主服务器
这里新建一个用户跟授权。
登入到主数据库命令行执行以下命令,并允许其他Slave服务器可以通过远程访问Master,通过该用户读取二进制日志,实现数据同步
(1) mysql>create user repl; //创建新用户
(2) mysql> GRANT REPLICATION SLAVE ON *.* TO 'repl'@'192.168.0.%' IDENTIFIED BY 'mysql';
(3) 找到MySQL安装文件夹修改my.Ini文件,在[mysqld]下面添加如下语句
1 server-id=1
2 log-bin=master-bin
3 log-bin-index=master-bin.index
(4)、执行show master status;查看状态
+-------------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------+----------+--------------+------------------+
| master-bin.000004 | 725 | | |
+-------------------+----------+--------------+------------------+
1 row in set (0.00 sec)
重启MySQL服务
3、配置从数据库
(1)、执行vim /etc/mysql/my.cf
在[mysqld]下面增加下面几行代码
server-id=2
relay-log-index=slave-relay-bin.index
relay-log=slave-relay-bin
重启MySQL服务
(2)登入数据库执行一下命令连接主库
change master to master_host='172.17.0.41', //Master 服务器Ip
master_port=3306,
master_user='repl',
master_password='mysql',
master_log_file='master-bin.000004',//Master服务器产生的日志
master_log_pos=0;
现在的大型系统为了实现大规模的数据并发。一个可以调整应用架构。一个是解决数据的并发问题。而数据库的主从配置跟读写分离就能很好的解决这个问题。
概述:搭设一台Master服务器(win7系统,Ip:localhost/ 172.17.0.41),搭设一台Slave服务器(虚拟机——一台Ubuntu 172.17.0.128)。
原理:主服务器(Master)负责网站写操作,从服务器负责查询操作。主从服务器利用MySQL的二进制日志文件,实现数据同步。二进制日志由主服务器产生,从服务器响应获取同步数据库。
实现:
1、windows我安装的是mysql_5.5.27.msi,ubantu安装的是mysql-5.6.22-linux-glibc2.5-i686.tar
安装数据的步骤这里不在赘述。可以自己百度步骤。
先说一下数据库的读写分离
要实现mysql的读写分离需要第三方的中间件。如ameoba
请自行下载ameoba、有关ameoba的知识请自行百度学习。这是国人自己开发的中间件。有详细的文档介绍。
配置ameoba
<?xml version="1.0" encoding="gbk"?>
<!DOCTYPE amoeba:configuration SYSTEM "amoeba.dtd">
<amoeba:configuration xmlns:amoeba="http://amoeba.meidusa.com/">
<proxy>
<!-- service class must implements com.meidusa.amoeba.service.Service -->
<service name="Amoeba for Mysql" class="com.meidusa.amoeba.net.ServerableConnectionManager">
<!-- port -->
<property name="port">8066</property>
<!-- bind ipAddress -->
<property name="ipAddress">127.0.0.1</property>
<property name="manager">${clientConnectioneManager}</property>
<property name="connectionFactory">
<bean class="com.meidusa.amoeba.mysql.net.MysqlClientConnectionFactory">
<property name="sendBufferSize">128</property>
<property name="receiveBufferSize">64</property>
</bean>
</property>
<property name="authenticator">
<bean class="com.meidusa.amoeba.mysql.server.MysqlClientAuthenticator">
<property name="user">root</property>
<property name="password">root</property>
<property name="filter">
<bean class="com.meidusa.amoeba.server.IPAccessController">
<property name="ipFile">${amoeba.home}/conf/access_list.conf</property>
</bean>
</property>
</bean>
</property>
</service>
<!-- server class must implements com.meidusa.amoeba.service.Service -->
<service name="Amoeba Monitor Server" class="com.meidusa.amoeba.monitor.MonitorServer">
<!-- port -->
<!-- default value: random number
<property name="port">9066</property>
-->
<!-- bind ipAddress -->
<property name="ipAddress">127.0.0.1</property>
<property name="daemon">true</property>
<property name="manager">${clientConnectioneManager}</property>
<property name="connectionFactory">
<bean class="com.meidusa.amoeba.monitor.net.MonitorClientConnectionFactory"></bean>
</property>
</service>
<runtime class="com.meidusa.amoeba.mysql.context.MysqlRuntimeContext">
<!-- proxy server net IO Read thread size -->
<property name="readThreadPoolSize">20</property>
<!-- proxy server client process thread size -->
<property name="clientSideThreadPoolSize">30</property>
<!-- mysql server data packet process thread size -->
<property name="serverSideThreadPoolSize">30</property>
<!-- per connection cache prepared statement size -->
<property name="statementCacheSize">500</property>
<!-- query timeout( default: 60 second , TimeUnit:second) -->
<property name="queryTimeout">60</property>
</runtime>
</proxy>
<!--
Each ConnectionManager will start as thread
manager responsible for the Connection IO read , Death Detection
-->
<connectionManagerList>
<connectionManager name="clientConnectioneManager" class="com.meidusa.amoeba.net.MultiConnectionManagerWrapper">
<property name="subManagerClassName">com.meidusa.amoeba.net.ConnectionManager</property>
<!--
default value is avaliable Processors
<property name="processors">5</property>
-->
</connectionManager>
<connectionManager name="defaultManager" class="com.meidusa.amoeba.net.MultiConnectionManagerWrapper">
<property name="subManagerClassName">com.meidusa.amoeba.net.AuthingableConnectionManager</property>
<!--
default value is avaliable Processors
<property name="processors">5</property>
-->
</connectionManager>
</connectionManagerList>
<!-- default using file loader -->
<dbServerLoader class="com.meidusa.amoeba.context.DBServerConfigFileLoader">
<property name="configFile">${amoeba.home}/conf/dbServers.xml</property>
</dbServerLoader>
<queryRouter class="com.meidusa.amoeba.mysql.parser.MysqlQueryRouter">
<property name="ruleLoader">
<bean class="com.meidusa.amoeba.route.TableRuleFileLoader">
<property name="ruleFile">${amoeba.home}/conf/rule.xml</property>
<property name="functionFile">${amoeba.home}/conf/ruleFunctionMap.xml</property>
</bean>
</property>
<property name="sqlFunctionFile">${amoeba.home}/conf/functionMap.xml</property>
<property name="LRUMapSize">1500</property>
<property name="defaultPool">master</property>
配置读写库
<property name="writePool">master</property>
<property name="readPool">multiPool</property>
<property name="needParse">true</property>
</queryRouter>
</amoeba:configuration>
<?xml version="1.0" encoding="gbk"?>
dbServers的配置
<!DOCTYPE amoeba:dbServers SYSTEM "dbserver.dtd">
<amoeba:dbServers xmlns:amoeba="http://amoeba.meidusa.com/">
<!--
Each dbServer needs to be configured into a Pool,
If you need to configure multiple dbServer with load balancing that can be simplified by the following configuration:
add attribute with name virtual = "true" in dbServer, but the configuration does not allow the element with name factoryConfig
such as 'multiPool' dbServer
-->
<dbServer name="abstractServer" abstractive="true">
<factoryConfig class="com.meidusa.amoeba.mysql.net.MysqlServerConnectionFactory">
<property name="manager">${defaultManager}</property>
<property name="sendBufferSize">64</property>
<property name="receiveBufferSize">128</property>
<!-- mysql port -->
<property name="port">3306</property>
<!-- mysql schema -->
<property name="schema">mysql</property>
<!-- mysql user -->
<property name="user">root</property>
<!-- mysql password-->
<property name="password">root</property>
</factoryConfig>
<poolConfig class="com.meidusa.amoeba.net.poolable.PoolableObjectPool">
<property name="maxActive">500</property>
<property name="maxIdle">500</property>
<property name="minIdle">10</property>
<property name="minEvictableIdleTimeMillis">600000</property>
<property name="timeBetweenEvictionRunsMillis">600000</property>
<property name="testOnBorrow">true</property>
<property name="testOnReturn">true</property>
<property name="testWhileIdle">true</property>
</poolConfig>
</dbServer>
<dbServer name="master" parent="abstractServer">
<factoryConfig>
<!-- mysql ip -->
<property name="ipAddress">127.0.0.1</property>
</factoryConfig>
</dbServer>
<dbServer name="slave" parent="abstractServer">
<factoryConfig>
<!-- mysql ip -->
<property name="ipAddress">172.17.0.128</property>
</factoryConfig>
</dbServer>
<dbServer name="multiPool" virtual="true">
<poolConfig class="com.meidusa.amoeba.server.MultipleServerPool">
<!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA-->
<property name="loadbalance">1</property>
<!-- Separated by commas,such as: server1,server2,server1 -->
<property name="poolNames">master,slave</property>
</poolConfig>
</dbServer>
</amoeba:dbServers>
在主库上执行授权
grant all on *.* to'root'@'172.17.%.%' identified by 'root';
grant all on *.* to'root'@'%mysql.com' identified by 'root';
flush privileges;
验证
两个数据库都有test_test表都要为空表
在dos执行连接进行验证。这里连接到amoeba
mysql -h127.0.0.1 -uroot -p -P8806
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 2097086015
Server version: 5.1.45-mysql-amoeba-proxy-2.2.0 Source distribution
Copyright (c) 2000, 2014, Oracle, SkySQL Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
先对test_test表执行插入语句
insert into test_test (id.name)values('1','ddd');
之后在启动新的dos窗口登入主库master查询会有一条数据插入
而从库是没有数据插入的
从amoeba查询也是空表。这样基本实现了读写分离
2、配置数据的主从
配置Master主服务器
这里新建一个用户跟授权。
登入到主数据库命令行执行以下命令,并允许其他Slave服务器可以通过远程访问Master,通过该用户读取二进制日志,实现数据同步
(1) mysql>create user repl; //创建新用户
(2) mysql> GRANT REPLICATION SLAVE ON *.* TO 'repl'@'192.168.0.%' IDENTIFIED BY 'mysql';
(3) 找到MySQL安装文件夹修改my.Ini文件,在[mysqld]下面添加如下语句
1 server-id=1
2 log-bin=master-bin
3 log-bin-index=master-bin.index
(4)、执行show master status;查看状态
+-------------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------+----------+--------------+------------------+
| master-bin.000004 | 725 | | |
+-------------------+----------+--------------+------------------+
1 row in set (0.00 sec)
重启MySQL服务
3、配置从数据库
(1)、执行vim /etc/mysql/my.cf
在[mysqld]下面增加下面几行代码
server-id=2
relay-log-index=slave-relay-bin.index
relay-log=slave-relay-bin
重启MySQL服务
(2)登入数据库执行一下命令连接主库
change master to master_host='172.17.0.41', //Master 服务器Ip
master_port=3306,
master_user='repl',
master_password='mysql',
master_log_file='master-bin.000004',//Master服务器产生的日志
master_log_pos=0;
(3)执行salve start;
问题
如果有Slave_SQL_Running: No
请执行
mysql> stop slave ;
mysql> set GLOBAL SQL_SLAVE_SKIP_COUNTER=1;
mysql> start slave ;
还有另外一种方案就是手动进行日志二进制文件的同步