Reading Ingestion —— Bigtable: A Distributed Storage System for Structured Data

Bigtable: A Distributed Storage System for Structured Data

  \space  

Acquirement 收获

1. Bigtable’s Data Model: is a sparse, distributed, persistent multidimensional sorted map. The map is indexed by a row key, column key, and a timestamp;
   $(Bigtable的数据模型：由行键、列键、时间戳构成的map。)$
2. Bigtable uses the distributed Google File System (GFS) to store log and data files.
   $(Bigtable用于存储数据和日志的GFS分布式文件系统。)$
3. Bigtable relies on a highly-available and persistent distributed lock service called Chubby.
   $(Chubby是Bigtable的高可用、持久分布式锁服务。)$
4. The Bigtable implementation has three major components: a library that is linked into every client, one master server, and many tablet servers.
   $(Bigtable实现的主要组成部分由：每个client都连接的library、一个master、多个tabletserver。)$
5. We use a three-level hierarchy analogous to that of a B± tree to store tablet location information:
   $(Bigtable的数据定位：用一个类似于B树的三层结构来存储tablet的位置信息。)$
6. The master keeps track of the set of live tablet servers by Chubby, and the current assignment of tablets to tablet servers, including which tablets are unassigned.
   $(Bigtable的数据分配：master通过Chubby来追踪存活tabletservers、目前已分配的tablets到)$
   $(tabletserver情况、未分配的tablet情况)$
7. The Bigtable 's tablet serving:recover tablet、write operation、read operation.
   $(Bigtable的tablet服务：tablet恢复、写操作、读操作。)$
8. The Bigtable 's compaction:minor compaction、merging compaction、major compaction.
   $(Bigtable的压缩：小压缩、合并压缩、大压缩。)$

  \space  

Introduction 引言

1. A distributed storage system for managing structured data at Google called Bigtable.
   $(Bigtable是用于管理结构化数据的分布式存储系统。)$
2. Bigtable has achieved several goals: wide applicability, scalability, high performance, and high availability.
   $(实现了广泛应用性、可扩展、高性能、高可用。)$
3. These products use Bigtable for a variety of demanding workloads, which range from throughput-oriented batch-processing jobs to latency-sensitive serving of data to end users.
   $(适用于面向高吞吐量的批处理任务、对延迟敏感的服务类产品。)$
4. Bigtable provides clients with a simple data model that supports dynamic control over data layout and format, and allows clients to reason about the locality properties of the data represented in the underlying storage.
   $(提供了支持动态控制数据布局及格式的简单数据模型。)$
5. Bigtable schema parameters let clients dynamically control whether to serve data out of memory or from disk.
   $(其模式参数使得Client动态控制是从内存还是磁盘中提供数据。)$

  \space  

Data Model 数据模型

  \space  

Map 映射

1. A Bigtable is a sparse, distributed, persistent multidimensional sorted map.
   $(稀疏、分布式、持久的多位排序映射。)$
2. The map is indexed by a row key, column key, and a timestamp; each value in the map is an uninterpreted array of bytes.
   $(由行键、列键、时间戳索引，每个值是一个无意义的字节数组。)$

$(row:string,column:string,time:int64)$ $\to$ $string$

  \space  

Webtable:a large collection of web pages and related information

1. In Webtable,The row name is a reversed URL.
   $(rowkey为URL。)$
2. The contents column family contains the page contents.
   $(contentscolumnfamily包含了页的内容。)$
3. The anchor column family contains the text of any anchors that reference the page. CNN’s home page is referenced by both the Sports Illustrated and the MY-look home pages, so the row contains columns named anchor:cnnsi.com and anchor:my.look.ca.
   $(anchorcolumnfamily包含了引用此页面的描点文本，包括了SportsIllustrated和MY-look)$
4. Each anchor cell has one version; the contents column has three versions, at timestamps t3, t5, and t6.
   $(每个anchorcell中只有一个版本，而每个contentscolumn有三个版本)$

  \space  

Rows 行

1. Every read or write of data under a single row key is atomic (regardless of the number of different columns being read or written in the row), a design decision that makes it easier for clients to reason about the system’s behavior in the presence of concurrent updates to the same row.
   $(单个rowkey下的每个数据的读写操作是原子性的，不管这一行有几个不同列组成。)$
   $(这是方便Client考虑如何处理对同一行并发更新时的行为。)$
2. Each row range is called a tablet, which is the unit of distribution and load balancing,and dynamically partitioned.
   $(一个rowrange内的rows称为tablet，是动态分布和负载均衡的单位。)$
3. Bigtable maintains data in lexicographic order by row key.As a result, reads of short row ranges are efficient and typically require communication with only a small number of machines.
   $(由于rowkeys是按照字典排序的，所以通过rowrange可以高效读取并使得机器间通讯减少。)$
4. For example, in Webtable, pages in the same domain are grouped together into contiguous rows by reversing the hostname components of the URLs：we store data for maps.google.com/index.html under the key com.google.maps/index.html.
   $(比如在上述的Webtable中，将相同领域的页通过将URL的主机名颠倒的方法放在连续的行中。)$
   $(比如maps.google.com/index.html与com.google.maps/index.html。)$

  \space  

Column Families 列族

1. Column keys are grouped into sets called column families, which form the basic unit of access control.
   $(一个columnfamiliy由一组相关的columnkey组成，是访问控制的单位。)$
2. A column family must be created before data can be stored under any column key in that family; after a family has
   been created, any column key within the family can be used.
   $(一个columnfamiliy创建之后，columnkey才可以被使用。)$
3. It is our intent that the number of distinct column families in a table be small (in the hundreds at most), and that families rarely change during operation. In contrast, a table may have an unbounded number of columns.
   $(独立的columnfamiliy往往只有$