示例推荐阅读顺序:readme.txt,AtomDataSourceDao,GroupDataSourceDao,BaseCrudSampleDaoLocal,BaseCrudSampleDao 然后看其它的
参考文档:
组成结构图:
Atom层(可独立使用)
–单个数据库的抽象
–动态化的jboss数据源,ip port 用户名密码都可以动态修改。
–TryLock模型
–动态阻止某个sql执行
–执行次数统计和限制
GROUP 层(可独立使用)
–读写分离
–权重
–写的HA切换
–读的HA切换
–允许动态添加新的slave节点
Matrix 层
–核心是规则引擎
•可以单独抽取出来,放在其他实现里,比如自己封装的ibatis DAO里面。
–通过规则引擎实现了动态扩容
–主要路径
•Sql解析->规则引擎计算->数据执行->合并结果
简单例子,使用本地配置文件:
tddl-sample/src/main/resources/rule_local/spring-context.xml
<?
xml
version
=
"1.0"
encoding
=
"gb2312"
?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" "http://www.springframework.org/dtd/spring-beans.dtd">
<
beans
>
<
bean
id
=
"metrixBean"
class
=
"com.taobao.tddl.client.jdbc.TDataSource"
init-method
=
"init"
>
<
property
name
=
"appRuleFile"
value
=
"classpath:rule_local/tddl-rule.xml"
/>
<
property
name
=
"useLocalConfig"
value
=
"false"
/>
<
property
name
=
"rwDataSourcePool"
>
<
map
>
<
entry
key
=
"group_sample_0"
value
=
"atom0:r10w10"
></
entry
>
<
entry
key
=
"group_sample_1"
value
=
"atom1:r10w10"
></
entry
>
</
map
>
</
property
>
</
bean
>
<
bean
id
=
"tddlDS"
class
=
"org.springframework.jdbc.core.JdbcTemplate"
>
<
property
name
=
"dataSource"
>
<
ref
bean
=
"metrixBean"
/>
</
property
>
</
bean
>
<
bean
id
=
"atom0"
class
=
"org.apache.commons.dbcp.BasicDataSource"
init-method
=
"createDataSource"
destroy-method
=
"close"
>
<
property
name
=
"driverClassName"
value
=
"com.mysql.jdbc.Driver"
></
property
>
<
property
name
=
"url"
value
=
"jdbc:mysql://192.168.152.128:3306/qatest_normal_0"
></
property
>
<
property
name
=
"username"
value
=
"null"
></
property
>
<
property
name
=
"password"
value
=
"null"
></
property
>
</
bean
>
<
bean
id
=
"atom1"
class
=
"org.apache.commons.dbcp.BasicDataSource"
init-method
=
"createDataSource"
destroy-method
=
"close"
>
<
property
name
=
"driverClassName"
value
=
"com.mysql.jdbc.Driver"
></
property
>
<
property
name
=
"url"
value
=
"jdbc:mysql://192.168.152.128:3306/qatest_normal_1"
></
property
>
<
property
name
=
"username"
value
=
"null"
></
property
>
<
property
name
=
"password"
value
=
"null"
></
property
>
</
bean
>
</
beans
>
|
tddl-sample/src/main/resources/rule_local/tddl-rule.xml
<?
xml
version
=
"1.0"
encoding
=
"gb2312"
?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" "http://www.springframework.org/dtd/spring-beans.dtd">
<
beans
>
<!-- 请先阅读README.txt! -->
<
bean
id
=
"vtabroot"
class
=
"com.taobao.tddl.interact.rule.VirtualTableRoot"
init-method
=
"init"
>
<
property
name
=
"dbType"
value
=
"MYSQL"
/>
<
property
name
=
"tableRules"
>
<
map
>
<
entry
key
=
"normaltbl"
value-ref
=
"normaltblBean"
/>
</
map
>
</
property
>
</
bean
>
<
bean
id
=
"normaltblBean"
class
=
"com.taobao.tddl.interact.rule.TableRule"
>
<
property
name
=
"dbNamePattern"
value
=
"group_sample_{0}"
/>
<!-- #pk,1,4# 取pk字段,单位是1,枚举是4个[1,4]-->
<
property
name
=
"dbRules"
value
=
"(#pk,1,4#.longValue() % 4).intdiv(2)"
/>
<
property
name
=
"tbNamePattern"
value
=
"normaltbl_{0000}"
/>
<
property
name
=
"tbRules"
value
=
"(#pk,1,4#).longValue()% 4"
/>
</
bean
>
</
beans
>
|
复杂例子,使用diamond:
diamond配置详情:
##1.创建matrix,用来管理group
##eg: com.taobao.tddl.v1_${appName}_dbgroups
com.taobao.tddl.v1_tddl_sample_dbgroups
group_sample_0,group_sample_1
##2.创建group,用来管理atom的读写权重
##eg: com.taobao.tddl.jdbc.group_V2.4.1_${dbGroupKey}
com.taobao.tddl.jdbc.group_V2.4.1_group_sample_0
qatest_normal_0:r10w10
com.taobao.tddl.jdbc.group_V2.4.1_group_sample_1
qatest_normal_1:r10w10
##3.创建atom,用来管理数据库database,包含三部分 global,app,passwd
##global eg:com.taobao.tddl.atom.global.${dbKey}
##app eg :com.taobao.tddl.atom.app.${appName}.${dbKey}
##pass eg :com.taobao.tddl.atom.passwd.${dbName}.${dbType(mysql|oracle)}.${userName}
com.taobao.tddl.atom.global.qatest_normal_0
ip=192.168.152.128
port=3306
dbName=qatest_normal_0
dbType=mysql
dbStatus=RW
com.taobao.tddl.atom.app.tddl_sample.qatest_normal_0
userName=tddl
minPoolSize=1
maxPoolSize=2
idleTimeout=10
blockingTimeout=5
preparedStatementCacheSize=15
connectionProperties=characterEncoding=gbk
com.taobao.tddl.atom.passwd.qatest_normal_0.mysql.tddl
encPasswd=66a64e5e6c3d5217d02860e788ab72a5
---------------------------------------------------------
com.taobao.tddl.atom.global.qatest_normal_1
ip=192.168.152.128
port=3306
dbName=qatest_normal_1
dbType=mysql
dbStatus=RW
com.taobao.tddl.atom.app.tddl_sample.qatest_normal_1
userName=tddl
minPoolSize=1
maxPoolSize=2
idleTimeout=10
blockingTimeout=5
preparedStatementCacheSize=15
connectionProperties=characterEncoding=gbk
com.taobao.tddl.atom.passwd.qatest_normal_1.mysql.tddl
encPasswd=66a64e5e6c3d5217d02860e788ab72a5
tddl-sequence
把sequence分部在不同的group,设置不同的起始值,相同的增量。
示例如下,sample_group_1增长前后的表格:
sample_group_0
|
0
|
---|---|
sample_group_1 | 1000 |
sample_group_2 | 2000 |
sample_group_3 | 3000 |
sample_group_0
|
0
|
---|---|
sample_group_1 | 5000 |
sample_group_2 | 2000 |
sample_group_3 | 3000 |
完善的功能TODO:
0.tddl完整的压力测试
1.diamond配置图形化
2.websql工具——自动执行sql,按规则生成库表,监控
3.数据实时迁移工具
常见问题:
1.支持跨库跨表的sql吗?
简单的包含分库分表字段的语句是支持的,如: pk是分库表的字段,能支持语句where pk in (2,3), where pk=2 or pk=3 ,如果要跨库跨表做distinct/sum 等,就不得不对多个子结果拿出来应用中再合并了。另外支持跨库表的排序合并(order by pk limit 1,10),count,min,max,默认是不支持的,需要在配置中打开:
<
bean
id
=
"normaltbl"
class
=
"com.taobao.tddl.common.config.beans.TableRule"
>
......
<
property
name
=
"disableFullTableScan"
value
=
"false"
></
property
>
</
bean
>
|
如果想做自定义的指定库表操作,RouteHelper可以改变tddl的解析:
private
void
setDBAndTab(Long agentId) {
Map<String, String> tbMap =
new
HashMap<String, String>();
DecimalFormat df =
new
DecimalFormat(
"00"
);
String index = df.format(agentId %
64
);
tbMap.put(
"atp_policy"
,
"atp_policy_"
+ index);
tbMap.put(
"atp_policy_list"
,
"atp_policy_list_"
+ index);
RouteHelper.executeByDBAndTab(
"AT_PM_GROUP"
, tbMap);
}
|
2.对事务的支持怎么样?
如果在同一个库是可以在外层切入事务的,如果跨库了当前还不支持分布式事务。
3.如果有一个用户操作跨库了,假设第一个库取出了buyer-product,第二个库取出了product-seller,那怎么做联合查询呢?
现在很多公司都是这么做:
(1)当buyer-product结果集少product-seller结果集巨大,先查询出它,然后放到内存中,然后通过product拼sql去查seller,这样应用的内存压力会比较大;
(2)当两个结果集都小的时候,可以都放到内存中合并;
(3)当两个结果集用得频繁,几乎常驻内存,可以在内存中去做筛选、过滤;
使用tddl后,可能会出现大量的内存合并,过滤操作,DB的运算压力转稼到了应用上。