mysql amoeba读写分离

一、Amoeba 是什么

Amoeba(变形虫)项目，专注 分布式数据库 proxy 开发。座落与Client、DB Server(s)之间。对客户端透明。具有负载均衡、高可用性、sql过滤、读写分离、可路由相关的query到目标数据库、可并发请求多台数据库合并结果。

主要解决：

• 降低 数据切分带来的复杂多数据库结构

• 提供切分规则并降低 数据切分规则 给应用带来的影响

• 降低db 与客户端的连接数

• 读写分离

二、为什么要用Amoeba

目前要实现mysql的主从读写分离，主要有以下几种方案：

1、  通过程序实现，网上很多现成的代码，比较复杂，如果添加从服务器要更改多台服务器的代码。

2、  通过mysql-proxy来实现，由于mysql-proxy的主从读写分离是通过lua脚本来实现，目前lua的脚本的开发跟不上节奏，而写没有完美的现成的脚本，因此导致用于生产环境的话风险比较大，据网上很多人说mysql-proxy的性能不高。

3、  自己开发接口实现，这种方案门槛高，开发成本高，不是一般的小公司能承担得起。

4、  利用阿里巴巴的开源项目Amoeba来实现，具有负载均衡、高可用性、sql过滤、读写分离、可路由相关的query到目标数据库，并且安装配置非常简单，但是目前还不支持事务。国产的开源软件，应该支持，目前正在使用，不发表太多结论，一切等测试完再发表结论吧，哈哈！



三、快速架设amoeba，实现mysql主从读写分离

1.安装环境：

 System： CentOS release 6.4 (Final)
 Master mysql：192.168.1.107
 Slave mysql：192.168.1.200
 Amoeba server：  192.168.1.107

2.安装配置mysql主从环境，mysql的安装省略，请自行编译或者用rpm安装

###在master mysql创建同步用户
创建用户
grant replication slave on *.* to 'repl'@'%' identified by '123456';
赋予权限
grant all privileges on *.* to 'repl'@'%';

刷新MySQL的系统权限相关表
flush privileges;

主从这里不做解释，可以参考http://blog.csdn.net/xy2204/article/details/9353941

ln -s /usr/java/jdk1.7.0_25 /usr/java/jdk1.6

vi /etc/profile

#添加如下两行内容

export JAVA_HOME=/usr/java/jdk1.6

export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH

#执行下面的命令使更改的内容生效

source /etc/profile
3、安装JDK环境

       下载jdk1.5或者更新版本,地址 http://java.sun.com/javase/downloads/index.jsp

       我用的是： jdk-7u25-linux-i586.rpm

安装jdk
rpm -ivh  jdk-7u25-linux-i586.rpm

添加环境变量

vi /etc/profile

#添加如下两行内容

export JAVA_HOME=/usr/java/jdk1.7.0_25

export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH

#执行下面的命令使更改的内容生效

source /etc/profile

4、  安装amoeba-mysql(http://sourceforge.net/projects/amoeba/files/Amoeba%20for%20mysql/)

下载amoeba-mysql

wget http://sourceforge.net/projects/amoeba/files/Amoeba%20for%20mysql/2.2.x/amoeba-mysql-binary-2.2.0.tar.gz/download

mkdir /usr/local/amoeba

tar -zxvf  amoeba-mysql-binary-2.2.0.tar.gz -C /usr/local/software/test/

配置文件位于conf目录下，执行文件位于bin目录下,解压后发现bin目录下的启动文件没有可执行权限，请执行：chmod -R +x /usr/local/amoeba/bin/

进入amoeba/conf

首先，在$AMOEBA_HOME/conf/dbServers.xml中配置一台数据库，如下：

Example 3.1. 简单的DB节点配置

<?xml version="1.0" encoding="gbk"?>


<!DOCTYPE amoeba:dbServers SYSTEM "dbserver.dtd">
<amoeba:dbServers xmlns:amoeba="http://amoeba.meidusa.com/">


<!-- 
Each dbServer needs to be configured into a Pool,
If you need to configure multiple dbServer with load balancing that can be simplified by the following configuration:
add attribute with name virtual = "true" in dbServer, but the configuration does not allow the element with name factoryConfig
such as 'multiPool' dbServer   
-->

<dbServer name="abstractServer" abstractive="true">
<factoryConfig class="com.meidusa.amoeba.mysql.net.MysqlServerConnectionFactory">
<property name="manager">${defaultManager}</property>
<property name="sendBufferSize">64</property>
<property name="receiveBufferSize">128</property>

<!-- mysql port -->
<property name="port">3306</property>

<!-- mysql schema -->
<property name="schema">test</property>

<!-- mysql user -->
<property name="user">repl</property>

<!--  mysql password -->
<property name="password">123456</property>
<!-- -->
</factoryConfig>


<poolConfig class="com.meidusa.amoeba.net.poolable.PoolableObjectPool">
<property name="maxActive">500</property>
<property name="maxIdle">500</property>
<property name="minIdle">10</property>
<property name="minEvictableIdleTimeMillis">600000</property>
<property name="timeBetweenEvictionRunsMillis">600000</property>
<property name="testOnBorrow">true</property>
<property name="testOnReturn">true</property>
<property name="testWhileIdle">true</property>
</poolConfig>
</dbServer>


<dbServer name="server1"  parent="abstractServer">
<factoryConfig>
<!-- mysql ip -->
<property name="ipAddress">192.168.1.107</property>
</factoryConfig>
</dbServer>

<dbServer name="server2"  parent="abstractServer">
<factoryConfig>
<!-- mysql ip -->
<property name="ipAddress">192.168.1.200</property>
</factoryConfig>
</dbServer>

<dbServer name="multiPool" virtual="true">
<poolConfig class="com.meidusa.amoeba.server.MultipleServerPool">
<!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA-->
<property name="loadbalance">1</property>

<!-- Separated by commas,such as: server1,server2,server1 -->
<property name="poolNames">server1,server2</property>
</poolConfig>
</dbServer>

</amoeba:dbServers>

再配置完dbServer.xml后，继续配置amoeba.xml文件：

Example 3.3. 一个基本的amoeba.xml例子

<?xml version="1.0" encoding="gbk"?>


<!DOCTYPE amoeba:configuration SYSTEM "amoeba.dtd">
<amoeba:configuration xmlns:amoeba="http://amoeba.meidusa.com/">


<proxy>

<!-- service class must implements com.meidusa.amoeba.service.Service -->
<service name="Amoeba for Mysql" class="com.meidusa.amoeba.net.ServerableConnectionManager">
<!-- port -->
<property name="port">8066</property>

<!-- bind ipAddress -->
<!-- 
<property name="ipAddress">127.0.0.1</property>
-->

<property name="manager">${clientConnectioneManager}</property>

<property name="connectionFactory">
<bean class="com.meidusa.amoeba.mysql.net.MysqlClientConnectionFactory">
<property name="sendBufferSize">128</property>
<property name="receiveBufferSize">64</property>
</bean>
</property>

<property name="authenticator">
<bean class="com.meidusa.amoeba.mysql.server.MysqlClientAuthenticator">

<property name="user">root</property>

<property name="password">123456</property>

<property name="filter">
<bean class="com.meidusa.amoeba.server.IPAccessController">
<property name="ipFile">${amoeba.home}/conf/access_list.conf</property>
</bean>
</property>
</bean>
</property>

</service>

<!-- server class must implements com.meidusa.amoeba.service.Service -->
<service name="Amoeba Monitor Server" class="com.meidusa.amoeba.monitor.MonitorServer">
<!-- port -->
<!--  default value: random number
<property name="port">9066</property>
-->
<!-- bind ipAddress -->
<property name="ipAddress">127.0.0.1</property>
<property name="daemon">true</property>
<property name="manager">${clientConnectioneManager}</property>
<property name="connectionFactory">
<bean class="com.meidusa.amoeba.monitor.net.MonitorClientConnectionFactory"></bean>
</property>

</service>

<runtime class="com.meidusa.amoeba.mysql.context.MysqlRuntimeContext">
<!-- proxy server net IO Read thread size -->
<property name="readThreadPoolSize">20</property>

<!-- proxy server client process thread size -->
<property name="clientSideThreadPoolSize">30</property>

<!-- mysql server data packet process thread size -->
<property name="serverSideThreadPoolSize">30</property>

<!-- per connection cache prepared statement size  -->
<property name="statementCacheSize">500</property>

<!-- query timeout( default: 60 second , TimeUnit:second) -->
<property name="queryTimeout">60</property>
</runtime>

</proxy>

<!-- 
Each ConnectionManager will start as thread
manager responsible for the Connection IO read , Death Detection
-->
<connectionManagerList>
<connectionManager name="clientConnectioneManager" class="com.meidusa.amoeba.net.MultiConnectionManagerWrapper">
<property name="subManagerClassName">com.meidusa.amoeba.net.ConnectionManager</property>
<!-- 
 default value is avaliable Processors 
<property name="processors">5</property>
-->
</connectionManager>
<connectionManager name="defaultManager" class="com.meidusa.amoeba.net.MultiConnectionManagerWrapper">
<property name="subManagerClassName">com.meidusa.amoeba.net.AuthingableConnectionManager</property>

<!-- 
 default value is avaliable Processors 
<property name="processors">5</property>
-->
</connectionManager>
</connectionManagerList>

<!-- default using file loader -->
<dbServerLoader class="com.meidusa.amoeba.context.DBServerConfigFileLoader">
<property name="configFile">${amoeba.home}/conf/dbServers.xml</property>
</dbServerLoader>




<queryRouter class="com.meidusa.amoeba.mysql.parser.MysqlQueryRouter">
<property name="ruleLoader">
<bean class="com.meidusa.amoeba.route.TableRuleFileLoader">
<property name="ruleFile">${amoeba.home}/conf/rule.xml</property>
<property name="functionFile">${amoeba.home}/conf/ruleFunctionMap.xml</property>
</bean>
</property>
<property name="sqlFunctionFile">${amoeba.home}/conf/functionMap.xml</property>
<property name="LRUMapSize">1500</property>
<property name="defaultPool">server1</property>

<property name="writePool">server1</property>
<property name="readPool">server2</property>
<property name="needParse">true</property>
</queryRouter>
</amoeba:configuration>

4.这些文件配置好后，就可以启动了
/usr/local/software/amoeba/bin/amoeba& start

检验启动是否成功（使用的是默认的8066端口）：

启动成功查看一下

[root@localhost /]# ps -aux | grep amoeba
Warning: bad syntax, perhaps a bogus '-'? See /usr/share/doc/procps-3.2.8/FAQ
root      6067  0.6 13.8 423480 70760 pts/2    Sl+  22:36   0:15 /usr/java/jdk1.7.0_25/bin/java -server -Xms256m -Xmx256m -Xss128k -Damoeba.home=/usr/local/software/amoeba -Dclassworlds.conf=/usr/local/software/amoeba/bin/amoeba.classworlds -classpath /usr/local/software/amoeba/lib/classworlds-1.0.jar org.codehaus.classworlds.Launcher start
root      6407  0.0  0.1   4280   752 pts/3    S+   23:15   0:00 grep amoeba

[root@localhost /]#  netstat -lnp | grep java

tcp        0      0 ::ffff:127.0.0.1:58193      :::*                        LISTEN      6067/java           
tcp        0      0 :::8066                     :::*                        LISTEN      6067/java    

5、  测试       

登录mysql使用如下命令（用户名密码和上面配置的要一致）：

mysql -uroot -p -h192.168.1.107 -P 8066

php代码测试：

header("Content-type: text/html; charset=utf-8");
$conn = mysql_connect('192.168.1.107:8066','root','123456') or die('no connect');
mysql_select_db('test',$conn) or die('no select db');
mysql_query('set names utf-8');


//mysql_query("INSERT INTO documents(title,content) VALUES('ttt','cccc')");
$query = mysql_query("SELECT * FROM documents");
while($row = mysql_fetch_assoc($query)){
 print_r($row);
}