在Hbase中选择多少个column family才合适呢?

下面主要说的是在设计Hbase schema的时候,要尽量只有一个column family,至于为什么主要从flush和compaction说起,它们触发的基本单位都是Region级别,所以当一个column family有大量的数据的时候会触发整个region里面的其他column family的memstore(其实这些memstore可能仅有少量的数据,还不需要flush的)也发生flush动作;另外compaction触发的条件是当store file的个数(不是总的store file的大小)达到一定数量的时候会发生,而flush产生的大量store file通常会导致compaction,flush/compaction会发生很多IO相关的负载,这对Hbase的整体性能有很大影响,所以选择合适的column family个数很重要。

 

下面是关于这方面的英文原文:

 

HBase currently does not do well with anything about two or three column families so keep the number of column families in your schema low. Currently, flushing and compactions are done on a per Region basis so if one column family is carrying the bulk of the data bringing on flushes, the adjacent families will also be flushed though the amount of data they carry is small. Compaction is currently triggered by the total number of files under a column family. Its not size based. When many column families the flushing and compaction interaction can make for a bunch of needless i/o loading (To be addressed by changing flushing and compaction to work on a per column family basis).

 

Try to make do with one column famliy if you can in your schemas. Only introduce a second and third column family in the case where data access is usually column scoped; i.e. you query one column family or the other but usually not both at the one time.

 

 

评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值