Bigtable: A Distributed Storage System for Structured Data : part4 Building Blocks

4 Building Blocks
Bigtable is built on several other pieces of Google infrastructure. 
Bigtable uses the distributed Google File System (GFS) to store log and data files. 
A Bigtable cluster typically operates in a shared pool of machines that run a wide variety of other distributed applications, and Bigtable processes often share the same machines with processes from other applications. 
Bigtable depends on a cluster management system for scheduling jobs, managing resources on shared machines, dealing with machine failures, and monitoring machine status.
The Google SSTable file format is used internally to store Bigtable data. 
An SSTable provides a persistent, ordered immutable map from keys to values, where both keys and values are arbitrary byte strings. 
Operations are provided to look up the value associated with a specified key, and to iterate over all key/value pairs in a specified key range. 
Internally, each SSTable contains a sequence of blocks (typically each block is 64KB in size, but this is configurable). 
A block index (stored at the end of the SSTable) is used to locate blocks; the index is loaded into memory when the SSTable is opened. 
A lookup can be performed with a single disk seek: 
we first find the appropriate block by performing a binary search in the in-memory index, and then reading the appropriate block from disk. 
Optionally, an SSTable can be completely mapped into memory, which allows us to perform lookups and scans without touching disk.


4创建块
Bigtable建立在其他几个Google基础架构上。
Bigtable使用分发的Google文件系统（GFS）来存储日志和数据文件。
BigTable集群通常在运行各种其他分布式应用程序的共享机器池中运行，而Bigtable过程通常会将来自其他应用程序的进程共享相同的机器。
Bigtable取决于用于调度作业的集群管理系统，在共享机器上管理资源，处理机器故障和监视机器状态。
Google SSTable文件格式在内部用于存储Bigtable数据。 SSTable提供了 从key到value 的持久的，有序的不可变映射，其中 key和value 都是任意字节字符串。
提供操作以查找与指定key相关联的value，并遍历指定键范围内的所有 key/value 对。
在内部，每个 SSTable 包含一个块序列（通常每个块大小为64KB，但是这是可配置的）。
块索引（存储在 SSTable 的末尾）用于定位块;当SSTable打开时，索引被加载到内存中。
可以使用单个磁盘查找来执行查找：
我们首先通过在内存索引中执行二进制搜索找到相应的块，然后从磁盘读取相应的块。

可选地，SSTable 可以完全映射到内存中，这样我们可以在不触摸磁盘的情况下执行查找和扫描。

Bigtable relies on a highly-available and persistent distributed lock service called Chubby. 
A Chubby service consists of five active replicas, one of which is elected to be the master and actively serve requests. 
The service is live when a majority of the replicas are running and can communicate with each other. 
Chubby uses the Paxos algorithm to keep its replicas consistent in the face of failure. 
Chubby provides a namespace that consists of directories and small files. 
Each directory or file can be used as a lock, and reads and writes to a file are atomic. 
The Chubby client library provides consistent caching of Chubby files. 
Each Chubby client maintains a session with a Chubby service. 
A client’s session expires if it is unable to renew its session lease within the lease expiration time. 
When a client’s session expires, it loses any locks and open handles. 
Chubby clients can also register callbacks on Chubby files and directories for notification of changes or session expiration.


Bigtable依赖于高可用性和持久性的分布式锁服务，称为Chubby。
胖子服务由五个活跃副本组成，其中一个被选为主人，并积极地提出要求。
当大多数副本正在运行并且可以相互通信时，该服务是生效的。
Chubby使用Paxos算法在面对失败时保持其副本一致。
Chubby提供了由目录和小文件组成的命名空间。
每个目录或文件都可以用作一个锁，对文件的读写是原子的。
Chubby客户端库提供了一致的缓存Chubby文件。
每个Chubby客户端都使用Chubby服务来维护会话。
如果客户端的会话在到期时间内无法续订其会话租约，则该会话将过期。
当客户端的会话过期时，它会失去任何锁和打开的句柄。
Chubby客户端还可以在Chubby文件和目录上注册回调，以通知更改或会话到期。

Bigtable uses Chubby for a variety of tasks: 
to ensure that there is at most one active master at any time; 
to store the bootstrap location of Bigtable data (see Section 5.1); 
to discover tablet servers and finalize tablet server deaths (see Section 5.2); 
to store Bigtable schema information (the column family information for each table); and to store access control lists. 
If Chubby becomes unavailable for an extended period of time, Bigtable becomes unavailable. 
We recently measured this effect in 14 Bigtable clusters spanning 11 Chubby instances.
The average percentage of Bigtable server hours during which some data stored in Bigtable was not available due to Chubby unavailability (caused by either Chubby outages or network issues) was 0.0047%. 
The percentage for the single cluster that was most affected by Chubby unavailability was 0.0326%.


Bigtable使用Chubby进行各种任务：
确保任何时候最多有一名主动掌握;
存储Bigtable数据的引导位置（见第5.1节）;
发现平板电脑服务器并完成平板电脑服务器死亡（参见第5.2节）;
存储Bigtable模式信息（每个表的列族信息）; 并存储访问控制列表。
如果Chubby长时间不可用，Bigtable将不可用。
我们最近在14个Bigtable集群中测量了这种效应，跨越11个Chubby实例。
由于Chubby不可用（由Chubby中断或网络问题引起），Bigtable中存储的一些数据不可用的Bigtable服务器小时的平均百分比为0.0047％。
受Chubby不可用性影响最大的单个群集的百分比为0.0326％。