HBase实战 2.2.5 背景知识：HBase读路径



2.2.5 Under the hood: the HBase read path

As a general rule, if you want fast access to data, keep it ordered and keep as much of
it as possible in memory. HBase accomplishes both of these goals, allowing it to serve
millisecond reads in most cases.

一般规律，如果你想快速访问数据，把数据排序和把数据尽可能多地保留在内存中。HBase实现了这两个目标，使得大多数情况下的数据读取达到毫秒级。

 A read against HBase must be reconciled between the
persisted HFiles and the data still in the MemStore. HBase has an LRU cache for reads.
This cache, also called the BlockCache, sits in the JVM heap alongside the MemStore.

从HBase读数据必须汇总持久化到HFiles中的数据和还在Memstore中的数据。为了快速读取，HBase设置了LRU缓存。该缓存，也称作BlockCache（块缓存），保存在JVM堆中，挨着MemStore。

The BlockCache is designed to keep frequently accessed data from the HFiles in memory
so as to avoid disk reads. Each column family has its own BlockCache.

BlockCache目的是把经常访问的来自HFile的数据缓存在内存中，避免读取磁盘。每一个列簇有一个BlockCache。

Understanding the BlockCache is an important part of understanding how to run
HBase at optimal performance. The “Block” in BlockCache is the unit of data that
HBase reads from disk in a single pass.

理解BlockCahe是理解HBase调优的关键一环。BlockCache中的“Block”是HBase一次从磁盘读取的数据单元。

The HFile is physically laid out as a sequence of blocks plus an index over those blocks. This means reading a block from HBase requires only looking up that block’s location in the index and retrieving it from disk.

HFile物理上是一系列的block加上一个这些block的索引。这意味着，读取一个block只需在索引中查找该block的位置，再从磁盘获取。

The block is the smallest indexed unit of data and is the smallest unit of data that can be read from disk.

block是数据最小的索引单元，也是可以从磁盘读取的最小单元。

The block size is configured per column family, and the default value is 64 KB. You may want to tweak this value larger or smaller depending on your use case. If you primarily perform random lookups, you likely want a more granular block index, so a smaller block size is preferred. Having smaller blocks creates a larger index and thereby consumes more memory. If you frequently perform sequential scans, reading many blocks at a time, you can afford a larger block size. This allows you to save on memory because larger blocks mean fewer index entries and thus a smaller index.

每个列簇可以单独设置block大小，缺省值是64KB。你可能想根据自己的情况调整该大小。如果你主要是随机查找，你想要一个更细粒度的block索引，那么较小的block尺寸更好。如果你经常进行顺序扫描，一次操作读取好多block，你可以把block大小设大点。这样可以节省内存，因为大的block意味着更少的索引项，索引也会因此变小。

Reading a row from HBase requires first checking the MemStore for any pending
modifications. Then the BlockCache is examined to see if the block containing this
row has been recently accessed. Finally, the relevant HFiles on disk are accessed. There
are more things going on under the hood, but this is the overall outline. Figure 2.2
illustrates the read path.

读取一行需要首先检查MemStore看是否有后续的修改（如果比较删除，则后面的反问就不需要了吧）。然后检查BlockCache看包含读取行的block是否最近被访问过（如果足够近，应该有缓存）。最后，读取相关的在磁盘上的HFiles（加复数的原因是一行包含多个列簇，不同的列簇保存在不同的HFile中）。底层还有许多事要做，但这是总体的轮廓。图2.2说明了读路径。

Note that HFiles contain a snapshot of the MemStore at the point when it was
flushed. Data for a complete row can be stored across multiple HFiles. In order to read a complete row, HBase must read across all HFiles that might contain information for
that row in order to compose the complete record.

注意，HFile包含它被刷写时MemStore的快照（快照是什么概念？如果只是时间点还好说，如果包含所有数据岂不是重复了？或者说，HFile就是快照？看来最后一种解释更可能）。一个完整行的数据可能包含在多个HFile中。为了读取一整行，HBase必须读取所有包含该行数据的HFile。