Mycat之——程序指定分区分片

转载请注明出处：http://blog.csdn.net/l1028386804/article/details/77140429

一、简单描述

程序指定分区分片，就是说程序在运行的运行根据规则自动指定数据的分片结果，具体规则如下：

此规则是在运行阶段有应用自主决定路由到那个分片。
此方法为直接根据字符子串（必须是数字）计算分区号（由应用传递参数，显式指定分区号）。

二、实现数据分片

1、配置rule.xml

在rule.xml中添加如下配置：

<!--例如id=0-100000020在此配置中代表根据id中从startIndex=0，开始，截取siz=1位数字即0，0就是获取的分区，如果没传默认分配到defaultPartition-->
<function name="sharding-by-substring-customer" class="org.opencloudb.route.function.PartitionDirectBySubString">
    <property name="startIndex">0</property>
    <property name="size">1</property>
    <property name="partitionCount">8</property>
    <property name="defaultPartition">0</property>
</function>

<tableRule name="tr-sharding-substr-customer">
     <rule>
         <columns>CODE</columns>
          <algorithm>sharding-by-substring-customer</algorithm>
    </rule>
</tableRule>

2、配置schema.xml

<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://org.opencloudb/" >

	<schema name="lyzdb" checkSQLschema="false" sqlMaxLimit="100">
		<!-- global table is auto cloned to all defined data nodes ,so can join 
			with any table whose sharding node is in the same data node -->
		<table name="t_order" primaryKey="id" dataNode="dn1,dn2" rule="tr-sharding-substr-customer"/>
	</schema>
	<!-- <dataNode name="dn1$0-743" dataHost="localhost1" database="db$0-743" 
		/> -->
	<dataNode name="dn1" dataHost="localhost1" database="db1" />
	<dataNode name="dn2" dataHost="localhost1" database="db2" />
	<!--<dataNode name="dn4" dataHost="sequoiadb1" database="SAMPLE" />
	 <dataNode name="jdbc_dn1" dataHost="jdbchost" database="db1" /> 
	<dataNode	name="jdbc_dn2" dataHost="jdbchost" database="db2" /> 
	<dataNode name="jdbc_dn3" 	dataHost="jdbchost" database="db3" /> -->
	<dataHost name="localhost1" maxCon="1000" minCon="10" balance="0"
		writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
		<heartbeat>select user()</heartbeat>
		<!-- can have multi write hosts -->
		<writeHost host="hostM1" url="192.168.209.137:3306" user="root" password="root"></writeHost>
		<!--<writeHost host="hostS1" url="localhost:3316" user="root"-->
			<!--password="123456" />-->
		<!-- <writeHost host="hostM2" url="localhost:3316" user="root" password="123456"/> -->
	</dataHost>
</mycat:schema>

至此，Mycat的配置工作就完成了，下面我们一起来测试下Mycat的路由结果。

三、测试数据分片

1、创建数据表

mysql> explain  CREATE TABLE t_order (ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,CODE VARCHAR(64),SN VARCHAR(64),CREATE_TIME DATETIME);
+-----------+------------------------------------------------------------------------------------------------------------------------+
| DATA_NODE | SQL                                                                                                                    |
+-----------+------------------------------------------------------------------------------------------------------------------------+
| dn1       | CREATE TABLE t_order (ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,CODE VARCHAR(64),SN VARCHAR(64),CREATE_TIME DATETIME) |
| dn2       | CREATE TABLE t_order (ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,CODE VARCHAR(64),SN VARCHAR(64),CREATE_TIME DATETIME) |
+-----------+------------------------------------------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)

mysql>  CREATE TABLE t_order (ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,CODE VARCHAR(64),SN VARCHAR(64),CREATE_TIME DATETIME);
Query OK, 0 rows affected (0.38 sec)

查看Mycat日志如下：

 08/13 09:49:52.185  DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:113) -ServerConnection [id=1, schema=lyzdb, host=192.168.209.1, user=lyz,txIsolation=3, autocommit=true, schema=lyzdb]CREATE TABLE t_order (ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,CODE VARCHAR(64),SN VARCHAR(64),CREATE_TIME DATETIME), route={
   1 -> dn1{CREATE TABLE t_order (ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,CODE VARCHAR(64),SN VARCHAR(64),CREATE_TIME DATETIME)}
   2 -> dn2{CREATE TABLE t_order (ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,CODE VARCHAR(64),SN VARCHAR(64),CREATE_TIME DATETIME)}
} rrs 

由此可见，当创建数据表时，Mycat会将SQL路由到所有的数据分片。

2、录入数据

mysql> explain INSERT INTO t_order (ID,CODE,SN,CREATE_TIME) VALUES(1,'0-10000020','beijing_13-10000010',NOW());
+-----------+-------------------------------------------------------------------------------------------------+
| DATA_NODE | SQL                                                                                             |
+-----------+-------------------------------------------------------------------------------------------------+
| dn1       | INSERT INTO t_order (ID,CODE,SN,CREATE_TIME) VALUES(1,'0-10000020','beijing_13-10000010',NOW()) |
+-----------+-------------------------------------------------------------------------------------------------+
1 row in set (0.12 sec)

mysql> INSERT INTO t_order (ID,CODE,SN,CREATE_TIME) VALUES(1,'0-10000020','beijing_13-10000010',NOW());
Query OK, 1 row affected (0.05 sec)

mysql> explain INSERT INTO t_order (ID,CODE,SN,CREATE_TIME) VALUES(2,'1-10000030','tianjin_14-10000020',NOW());
+-----------+-------------------------------------------------------------------------------------------------+
| DATA_NODE | SQL                                                                                             |
+-----------+-------------------------------------------------------------------------------------------------+
| dn2       | INSERT INTO t_order (ID,CODE,SN,CREATE_TIME) VALUES(2,'1-10000030','tianjin_14-10000020',NOW()) |
+-----------+-------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> INSERT INTO t_order (ID,CODE,SN,CREATE_TIME) VALUES(2,'1-10000030','tianjin_14-10000020',NOW());
Query OK, 1 row affected (0.01 sec)

Mycat日志日下：

08/13 09:52:18.358  DEBUG [$_NIOREACTOR-0-RW] (ServerQueryHandler.java:56) -ServerConnection [id=1, schema=lyzdb, host=192.168.209.1, user=lyz,txIsolation=3, autocommit=true, schema=lyzdb]INSERT INTO t_order (ID,CODE,SN,CREATE_TIME) VALUES(1,'0-10000020','beijing_13-10000010',NOW())
08/13 09:52:18.359  DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:113) -ServerConnection [id=1, schema=lyzdb, host=192.168.209.1, user=lyz,txIsolation=3, autocommit=true, schema=lyzdb]INSERT INTO t_order (ID,CODE,SN,CREATE_TIME) VALUES(1,'0-10000020','beijing_13-10000010',NOW()), route={
   1 -> dn1{INSERT INTO t_order (ID,CODE,SN,CREATE_TIME) VALUES(1,'0-10000020','beijing_13-10000010',NOW())}
} rrs 

08/13 09:52:59.379  DEBUG [$_NIOREACTOR-0-RW] (ServerQueryHandler.java:56) -ServerConnection [id=1, schema=lyzdb, host=192.168.209.1, user=lyz,txIsolation=3, autocommit=true, schema=lyzdb]INSERT INTO t_order (ID,CODE,SN,CREATE_TIME) VALUES(2,'1-10000030','tianjin_14-10000020',NOW())
08/13 09:52:59.380  DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:113) -ServerConnection [id=1, schema=lyzdb, host=192.168.209.1, user=lyz,txIsolation=3, autocommit=true, schema=lyzdb]INSERT INTO t_order (ID,CODE,SN,CREATE_TIME) VALUES(2,'1-10000030','tianjin_14-10000020',NOW()), route={
   1 -> dn2{INSERT INTO t_order (ID,CODE,SN,CREATE_TIME) VALUES(2,'1-10000030','tianjin_14-10000020',NOW())}
} rrs 

由此可见，录入数据时，Mycat会根据路由规则，将SQL路由到指定的数据分片

3、指定数据分片字段查询

mysql> explain select * from t_order where code = '0-10000020';
+-----------+-----------------------------------------------------------+
| DATA_NODE | SQL                                                       |
+-----------+-----------------------------------------------------------+
| dn1       | SELECT * FROM t_order WHERE code = '0-10000020' LIMIT 100 |
+-----------+-----------------------------------------------------------+
1 row in set (0.16 sec)

mysql> select * from t_order where code = '0-10000020';
+----+------------+---------------------+---------------------+
| ID | CODE       | SN                  | CREATE_TIME         |
+----+------------+---------------------+---------------------+
|  1 | 0-10000020 | beijing_13-10000010 | 2017-08-13 09:52:18 |
+----+------------+---------------------+---------------------+
1 row in set (0.01 sec)

Mycat日志如下：

08/13 09:54:23.917  DEBUG [$_NIOREACTOR-0-RW] (EnchachePool.java:59) -SQLRouteCache add cache ,key:lyzdbselect * from t_order where code = '0-10000020' value:select * from t_order where code = '0-10000020', route={
   1 -> dn1{SELECT *
FROM t_order
WHERE code = '0-10000020'
LIMIT 100}
}

由此可见，执行简单的查询，如果指定分片字段，则走分片查询单个分片节点。

4、按照分片字段范围查询

mysql> explain select * from t_order  where CODE like '0-1%';
+-----------+--------------------------------------------------------+
| DATA_NODE | SQL                                                    |
+-----------+--------------------------------------------------------+
| dn1       | SELECT * FROM t_order WHERE CODE LIKE '0-1%' LIMIT 100 |
| dn2       | SELECT * FROM t_order WHERE CODE LIKE '0-1%' LIMIT 100 |
+-----------+--------------------------------------------------------+
2 rows in set (0.00 sec)

mysql> select * from t_order  where CODE like '0-1%';
+----+------------+---------------------+---------------------+
| ID | CODE       | SN                  | CREATE_TIME         |
+----+------------+---------------------+---------------------+
|  1 | 0-10000020 | beijing_13-10000010 | 2017-08-13 09:52:18 |
+----+------------+---------------------+---------------------+
1 row in set (0.01 sec)

Mycat日志如下：

08/13 09:55:48.742  DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:113) -ServerConnection [id=1, schema=lyzdb, host=192.168.209.1, user=lyz,txIsolation=3, autocommit=true, schema=lyzdb]select * from t_order  where CODE like '0-1%', route={
   1 -> dn1{SELECT *
FROM t_order
WHERE CODE LIKE '0-1%'
LIMIT 100}
   2 -> dn2{SELECT *
FROM t_order
WHERE CODE LIKE '0-1%'
LIMIT 100}
} rrs 

由此可见，如果分片字段范围的查询，则走所有节点去检索，哪怕只有一条数据在一个分片上，route路由也是走所有的分片进行检索查询。

5、按照非分片字段查询

mysql> explain select * from t_order  where SN='beijing_13-10000010';
+-----------+------------------------------------------------------------------+
| DATA_NODE | SQL                                                              |
+-----------+------------------------------------------------------------------+
| dn1       | SELECT * FROM t_order WHERE SN = 'beijing_13-10000010' LIMIT 100 |
| dn2       | SELECT * FROM t_order WHERE SN = 'beijing_13-10000010' LIMIT 100 |
+-----------+------------------------------------------------------------------+
2 rows in set (0.00 sec)
mysql>
mysql> select * from t_order  where SN='beijing_13-10000010';
+----+------------+---------------------+---------------------+
| ID | CODE       | SN                  | CREATE_TIME         |
+----+------------+---------------------+---------------------+
|  1 | 0-10000020 | beijing_13-10000010 | 2017-08-13 09:52:18 |
+----+------------+---------------------+---------------------+
1 row in set (0.01 sec)

Mycat日志如下：

08/13 09:56:53.947  DEBUG [$_NIOREACTOR-0-RW] (EnchachePool.java:59) -SQLRouteCache add cache ,key:lyzdb select * from t_order  where SN='beijing_13-10000010' value:select * from t_order  where SN='beijing_13-10000010', route={
   1 -> dn1{SELECT *
FROM t_order
WHERE SN = 'beijing_13-10000010'
LIMIT 100}
   2 -> dn2{SELECT *
FROM t_order
WHERE SN = 'beijing_13-10000010'
LIMIT 100}
}

由此可见，如果不走分片字段的查询，即使是单个数据，也要route路由所有的分片，走所有的分片进行查询。