Hbase 流水笔记 四 Rowkey设计

RowKey的设计对数据的存储及读取至关重要。

问题1： Hotspotting  热点问题。

           Hotspotting occurs when a large amount of client traffic is directed at one node, or only a few nodes, of a cluster. This traffic may represent reads, writes, or other operations. The traffic overwhelms the single machine responsible for hosting that region, causing performance degradation and potentially leading to region unavailability. This can also have adverse effects on other regions hosted by the same region server as that host is unable to service the requested load. It is important to design data access patterns such that the cluster is fully and evenly utilized.

      大量的读 写操作在一个节点上或者少量的节点上，这样会造成性能瓶颈问题，可能导致服务崩溃，而不可访问，这是rowkey 设计不当造成的后果。

       例如：设计一张表 记录cpu 单位当前年一年的时间使用频率，举例说明不当还请指出。

       

tableName	cpu_frequency
rowKey	timestamp
cloumnFamily	info

       假如我们是用来split 预分区设定2019年1月1日0时0分0秒0毫秒开始和2019年2月1日0时。如何split预分区,这样做的会导致所有的数据都往一个分区里去写，导致其他的没有数据，我认为也可以理解为数据短暂的数据倾斜。当我们进行读取的时候，所有的请求都会在这个region上，就会造成服务器压力过大。

如何解决：

  1、反转时间戳，如 20190101000001   ------->10000000109102

  反转后数据虽然实际差1毫秒，反转后也会差距很大。可以使数据不至于全部都落到一个region中。

 

  2、如过设计的rowkey 不是因为时间戳设计不当，应该如何解决呢。

  例如 rowkey 格式如下：        

foo0001
foo0002
foo0003
foo0004

 

   Salting ，什么是加盐？

         Salting in this sense has nothing to do with cryptography, but refers to adding random data to the start of a row key. In this case, salting refers to adding a randomly-assigned prefix to the row key to cause it to sort differently than it otherwise would. The number of possible prefixes correspond to the number of regions you want to spread the data across. Salting can be helpful if you have a few "hot" row key patterns which come up over and over amongst other more evenly-distributed rows. Consider the following example, which shows that salting can spread write load across multiple RegionServers, and illustrates some of the negative implications for reads.

  可以给数据添加前缀，进行区分？

a-foo0003
b-foo0001
c-foo0004
d-foo0002

那我们设计的表的预split  可以使用以前缀 a ,b,c,d 作为前缀。

3、hash 值、md5 使rowkey 散列也是可以的。