mycat 离散分片(枚举分片)

mycat 离散分片(枚举分片)
1:枚举分片
枚举分片:通过在配置文件中配置可能的枚举id,自己配置分片,本规则适用于特定的场景,
比如有些业务需要按照省份或区县来做保存,而全国省份区县固定的

2:添加配置文件
<!-- 
    mapFile标识配置文件名称
    type默认值为0(0表示Integer,非零表示String)
    默认节点的作用:枚举分片时,如果碰到不识别的枚举值,就让它路由到默认节点    
    --> 
        <tableRule name="tr-hash-int-mc040301">
                 <rule>
                         <columns>PROVINCE</columns>
                         <algorithm>hash-int-mc040301</algorithm>
                 </rule>
        </tableRule>
        <function name="hash-int-mc040301" class="org.opencloudb.route.function.PartitionByFileMap">
                <property name="mapFile">partition-hash-int.txt</property> 
                <property name="defaultNode">0</property> 
                <property name="type">0</property>
        </function>

在schema.xml里面进行配置:
<table name="ORDER040301" primaryKey="ID" dataNode="dn1,dn2,dn3" rule="tr-hash-int-mc040301"/>

autopartition-long04.txt文件的配置, 
10000代表北京分片到第一个节点,10010代表上海分片到第二个节点,10020代表广州分片到第三个节点
[root@crm_1_21 conf]# more partition-hash-int04.txt 
10000=0 
10010=1 
10020=2
注意: 此配置非常简单,即预先制定可能的id范围到某个分片,所有的节点配置都是从0开始,及0代表节点1

[root@mycat1 conf]# /usr/local/mycat/bin/mycat restart
Stopping Mycat-server...
Mycat-server was not running.
Starting Mycat-server...

3:建表并且录入数据
[root@mycat1 conf]# mysql -uroot -poracle -h192.168.10.21 -P8066 -D TESTDB
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.5.8-mycat-1.5-alpha-20151221110028 MyCat Server (OpenCloundDB)
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> explain CREATE TABLE ORDER040301(ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,PROVINCE INT,SN VARCHAR(64),CREATE_TIME DATETIME);
+-----------+-----------------------------------------------------------------------------------------------------------------------+
| DATA_NODE | SQL                                                                                                                   |
+-----------+-----------------------------------------------------------------------------------------------------------------------+
| dn1       | CREATE TABLE ORDER040301(ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,PROVINCE INT,SN VARCHAR(64),CREATE_TIME DATETIME) |
| dn2       | CREATE TABLE ORDER040301(ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,PROVINCE INT,SN VARCHAR(64),CREATE_TIME DATETIME) |
| dn3       | CREATE TABLE ORDER040301(ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,PROVINCE INT,SN VARCHAR(64),CREATE_TIME DATETIME) |
+-----------+-----------------------------------------------------------------------------------------------------------------------+
3 rows in set (0.03 sec)

mysql> CREATE TABLE ORDER040301(ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,PROVINCE INT,SN VARCHAR(64),CREATE_TIME DATETIME);
Query OK, 0 rows affected (0.35 sec)

mysql> explain INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(1,10000,'beijing_10006_10000',NOW());
+-----------+-------------------------------------------------------------------------------------------------+
| DATA_NODE | SQL                                                                                             |
+-----------+-------------------------------------------------------------------------------------------------+
| dn1       | INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(1,10000,'beijing_10006_10000',NOW()) |
+-----------+-------------------------------------------------------------------------------------------------+
1 row in set (0.09 sec)

mysql> INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(1,10000,'beijing_10006_10000',NOW());
Query OK, 1 row affected (0.06 sec)

mysql> explain INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(2,10010,'shanghai_10010',NOW());
+-----------+--------------------------------------------------------------------------------------------+
| DATA_NODE | SQL                                                                                        |
+-----------+--------------------------------------------------------------------------------------------+
| dn2       | INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(2,10010,'shanghai_10010',NOW()) |
+-----------+--------------------------------------------------------------------------------------------+
1 row in set (0.02 sec)

mysql> INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(2,10010,'shanghai_10010',NOW());
Query OK, 1 row affected (0.04 sec)

mysql> explain INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(3,10020,'shanghai_10020',NOW());
+-----------+--------------------------------------------------------------------------------------------+
| DATA_NODE | SQL                                                                                        |
+-----------+--------------------------------------------------------------------------------------------+
| dn3       | INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(3,10020,'shanghai_10020',NOW()) |
+-----------+--------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(3,10020,'shanghai_10020',NOW());
Query OK, 1 row affected (0.03 sec)

4:对insert操作流程进行分析,查看mycat.log对路由过程做完整的分析
开始获取连接:
02/25 18:58:00.423  DEBUG [$_NIOREACTOR-2-RW] (ServerQueryHandler.java:56) -ServerConnection [id=2, schema=TESTDB, host=192.168.10.21, user=root,txIsolation=3, autocommit=true, schema=TESTDB]INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(3,10020,'shanghai_10020',NOW())
开始路由匹配:
02/25 18:58:00.423  DEBUG [$_NIOREACTOR-2-RW] (NonBlockingSession.java:113) -ServerConnection [id=2, schema=TESTDB, host=192.168.10.21, user=root,txIsolation=3, autocommit=true, schema=TESTDB]INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(3,10020,'shanghai_10020',NOW()), route={
   1 -> dn3{INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(3,10020,'shanghai_10020',NOW())}
} rrs 
02/25 18:58:00.424  DEBUG [$_NIOREACTOR-2-RW] (MySQLConnection.java:445) -con need syn ,total syn cmd 1 commands schema change:true con:MySQLConnection [id=5, lastTime=1456397880424, user=root, schema=db3, old shema=db2, borrowed=true, fromSlaveDB=false, threadId=47, charset=latin1, txIsolation=3, autocommit=true, attachment=dn3{INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(3,10020,'shanghai_10020',NOW())}, respHandler=SingleNodeHandler [node=dn3{INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(3,10020,'shanghai_10020',NOW())}, packetId=0], host=192.168.10.23, port=3306, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
匹配好路由后,直接往对应的dn3节点开始执行insert数据录入操作:
02/25 18:58:00.456  DEBUG [$_NIOREACTOR-1-RW] (NonBlockingSession.java:229) -release connection MySQLConnection [id=5, lastTime=1456397880422, user=root, schema=db3, old shema=db3, borrowed=true, fromSlaveDB=false, threadId=47, charset=latin1, txIsolation=3, autocommit=true, attachment=dn3{INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(3,10020,'shanghai_10020',NOW())}, respHandler=SingleNodeHandler [node=dn3{INSERT INTO ORDER040301(ID,PROVINCE,SN,CREATE_TIME) VALUES(3,10020,'shanghai_10020',NOW())}, packetId=0], host=192.168.10.23, port=3306, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
连接释放:
02/25 18:58:00.456  DEBUG [$_NIOREACTOR-1-RW] (PhysicalDatasource.java:403) -release channel MySQLConnection [id=5, lastTime=1456397880422, user=root, schema=db3, old shema=db3, borrowed=true, fromSlaveDB=false, threadId=47, charset=latin1, txIsolation=3, autocommit=true, attachment=null, respHandler=null, host=192.168.10.23, port=3306, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]

5:执行查询,如果指定分片字段ID字段,则走分片查询单个分片节点
mysql> explain select * from ORDER040301 t1 where PROVINCE=10000;
+-----------+---------------------------------------------------------------+
| DATA_NODE | SQL                                                           |
+-----------+---------------------------------------------------------------+
| dn1       | SELECT * FROM ORDER040301 t1 WHERE PROVINCE = 10000 LIMIT 100 |
+-----------+---------------------------------------------------------------+
1 row in set (0.14 sec)

查看mycat.log:
02/25 19:11:26.658  DEBUG [$_NIOREACTOR-2-RW] (ServerQueryHandler.java:56) -ServerConnection [id=2, schema=TESTDB, host=192.168.10.21, user=root,txIsolation=3, autocommit=true, schema=TESTDB]explain select * from ORDER040301 t1 where PROVINCE=10000
02/25 19:11:26.662  DEBUG [$_NIOREACTOR-2-RW] (EnchachePool.java:76) -SQLRouteCache  miss cache ,key:TESTDB select * from ORDER040301 t1 where PROVINCE=10000
02/25 19:11:26.793  DEBUG [$_NIOREACTOR-2-RW] (EnchachePool.java:59) -SQLRouteCache add cache ,key:TESTDB select * from ORDER040301 t1 where PROVINCE=10000 value:select * from ORDER040301 t1 where PROVINCE=10000, route={
   1 -> dn1{SELECT *
FROM ORDER040301 t1
WHERE PROVINCE = 10000
LIMIT 100}
}
02/25 19:11:28.706  DEBUG [Timer1] (ConnectionHeartBeatHandler.java:52) -do heartbeat for con MySQLConnection [id=6, lastTime=1456398688703, user=root, schema=db2, old shema=db2, borrowed=true, fromSlaveDB=false, threadId=43, charset=latin1, txIsolation=3, autocommit=true, attachment=null, respHandler=null, host=192.168.10.23, port=3306, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]
02/25 19:11:28.707  DEBUG [$_NIOREACTOR-2-RW] (ConnectionHeartBeatHandler.java:170) -received field eof  from MySQLConnection [id=6, lastTime=1456398688703, user=root, schema=db2, old shema=db2, borrowed=true, fromSlaveDB=false, threadId=43, charset=latin1, txIsolation=3, autocommit=true, attachment=null, respHandler=org.opencloudb.mysql.nio.handler.ConnectionHeartBeatHandler@31c513eb, host=192.168.10.23, port=3306, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]

6:如果分片字段范围的查询,则走所有节点去检索,哪怕只有一条数据在一个分片上,route路由也是走所有的分片进行检索查询
mysql> explain select * from ORDER040301 t1 where SN='beijing_10006_10000';
+-----------+-------------------------------------------------------------------------+
| DATA_NODE | SQL                                                                     |
+-----------+-------------------------------------------------------------------------+
| dn1       | SELECT * FROM ORDER040301 t1 WHERE SN = 'beijing_10006_10000' LIMIT 100 |
| dn2       | SELECT * FROM ORDER040301 t1 WHERE SN = 'beijing_10006_10000' LIMIT 100 |
| dn3       | SELECT * FROM ORDER040301 t1 WHERE SN = 'beijing_10006_10000' LIMIT 100 |
+-----------+-------------------------------------------------------------------------+
3 rows in set (0.02 sec)

mysql> select * from ORDER040301 t1 where SN='shanghai_10020';
+----+----------+----------------+---------------------+
| ID | PROVINCE | SN             | CREATE_TIME         |
+----+----------+----------------+---------------------+
|  3 |    10020 | shanghai_10020 | 2016-02-25 18:58:01 |
+----+----------+----------------+---------------------+
1 row in set (0.03 sec)

查看mycat.log:
02/25 19:26:20.655  DEBUG [$_NIOREACTOR-2-RW] (ServerQueryHandler.java:56) -ServerConnection [id=2, schema=TESTDB, host=192.168.10.21, user=root,txIsolation=3, autocommit=true, schema=TESTDB]select * from ORDER040301 t1 where SN='shanghai_10020'
02/25 19:26:20.656  DEBUG [$_NIOREACTOR-2-RW] (EnchachePool.java:76) -SQLRouteCache  miss cache ,key:TESTDBselect * from ORDER040301 t1 where SN='shanghai_10020'
02/25 19:26:20.657  DEBUG [$_NIOREACTOR-2-RW] (EnchachePool.java:59) -SQLRouteCache add cache ,key:TESTDBselect * from ORDER040301 t1 where SN='shanghai_10020' value:select * from ORDER040301 t1 where SN='shanghai_10020', route={
   1 -> dn1{SELECT *
FROM ORDER040301 t1
WHERE SN = 'shanghai_10020'
LIMIT 100}
   2 -> dn2{SELECT *
FROM ORDER040301 t1
WHERE SN = 'shanghai_10020'
LIMIT 100}
   3 -> dn3{SELECT *
FROM ORDER040301 t1
WHERE SN = 'shanghai_10020'
LIMIT 100}
}
02/25 19:26:20.673  DEBUG [$_NIOREACTOR-2-RW] (NonBlockingSession.java:113) -ServerConnection [id=2, schema=TESTDB, host=192.168.10.21, user=root,txIsolation=3, autocommit=true, schema=TESTDB]select * from ORDER040301 t1 where SN='shanghai_10020', route={
   1 -> dn1{SELECT *
FROM ORDER040301 t1
WHERE SN = 'shanghai_10020'
LIMIT 100}
   2 -> dn2{SELECT *
FROM ORDER040301 t1
WHERE SN = 'shanghai_10020'
LIMIT 100}
   3 -> dn3{SELECT *
FROM ORDER040301 t1
WHERE SN = 'shanghai_10020'
LIMIT 100}
} rrs 
02/25 19:26:20.673  DEBUG [$_NIOREACTOR-2-RW] (MultiNodeQueryHandler.java:82) -execute mutinode query select * from ORDER040301 t1 where SN='shanghai_10020'
02/25 19:26:20.673  DEBUG [$_NIOREACTOR-2-RW] (MultiNodeQueryHandler.java:97) -has data merge logic 
02/25 19:26:20.673  DEBUG [$_NIOREACTOR-2-RW] (PhysicalDBPool.java:452) -select read source mysql1 for dataHost:mysql1
02/25 19:26:20.673  DEBUG [$_NIOREACTOR-2-RW] (PhysicalDBPool.java:452) -select read source mysql1 for dataHost:mysql1
02/25 19:26:20.674  DEBUG [$_NIOREACTOR-2-RW] (PhysicalDBPool.java:452) -select read source mysql1 for dataHost:mysql1

7:如果不走分片字段的查询,即使是单个数据,也要route路由所有的分片,走所有的分片进行查询
mysql> explain select * from ORDER040301 t1 where PROVINCE < 10010;
+-----------+---------------------------------------------------------------+
| DATA_NODE | SQL                                                           |
+-----------+---------------------------------------------------------------+
| dn1       | SELECT * FROM ORDER040301 t1 WHERE PROVINCE < 10010 LIMIT 100 |
| dn2       | SELECT * FROM ORDER040301 t1 WHERE PROVINCE < 10010 LIMIT 100 |
| dn3       | SELECT * FROM ORDER040301 t1 WHERE PROVINCE < 10010 LIMIT 100 |
+-----------+---------------------------------------------------------------+
3 rows in set (0.00 sec)

mysql> select * from ORDER040301 t1 where PROVINCE < 10010;
+----+----------+---------------------+---------------------+
| ID | PROVINCE | SN                  | CREATE_TIME         |
+----+----------+---------------------+---------------------+
|  1 |    10000 | beijing_10006_10000 | 2016-02-25 19:22:15 |
+----+----------+---------------------+---------------------+
1 row in set (0.01 sec)

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值