关闭

Happy Apache Cassandra 2: File Store Format

标签: cassandrafileapacheimmutablefilterdelete
5245人阅读 评论(0) 收藏 举报
分类:

h3. Model Review

Cassandra data model

Column Family



Super Column Family



h3. File Format

h4. Keyspace

  • Each Keyspace(Eg. Lobs)in separated directory
  • Each ColumnFamily(Eg. object) in separated sstable files
    • ColumnFamilyName-version-#-Data.db
    • ColumnFamilyName-version-#-Index.db
    • ColumnFamilyName-version-#-Filter.db

h4. Index File 

  • Each entry is [Key, Position], order by key
  • Index Summary(pre-loaded into memory), every 1 entry in 128 entries
  • Key Cache, cache the entry after access it


h4. data file

  • Each entry is [Row], order by key
  • Each Row contains columns, order by column name
  • Each Row contains column index (in range)


h4. others

  • Bloom filter file
    • Given {an element}, answer {is contained in a set} .
    • Given a {key}, answer {is contained in current sstable file}
  • Commit log file
    • Used in write, avoid data lost 
  • Statistical file

h3. CRUD

h4. write

BigTable

  • First write to a disk commit log (sequential) 
  • Update to appropriate memtables
  • Memtables are flushed to disk into SSTable ( immutable)


h4. read

  • Check row cache
  • Cassandra will read all the SSTables for that Column Family
    • Bloom Filter for each SSTable to determine whether this SSTable contains the key
    • Use index in SSTable to locate the data (check key cache)
    • Read from data file

h4. update

  • Same sequence with write
    • SSTable is immutable
    • Update is write into new SSTable
  • Column Reconcile
    • Read columns from all the SSTables
    • Merge/Reduced columns with same name, CollationController
      • Use timestamp, return latest column

h4. delete

  • Same sequence with write
    • SSTable is immutable
    • Delete is write into new SSTable
  • Delete a column
    • Column flag is set to delete flag
  • Delete a Row
    • markedForDeleteAt  is set to delete timestamp
    • Column timestamp is compared with markedForDeleteAt when reducing columns in read

h4. drop

  • Drop a Column Family
    • Take snapshots (move to snapshot dir)
    • Remove column family definition
  • Drop a Keyspace
    • Take snapshots (move to snapshot dir)
    • Remove keyspace definition

h4. Compaction

Periodically data files are merged sorted into a new file (and creates new index)
  • Merge keys 
  • Combine columns 
  • Discard tombstones


References

http://jonathanhui.com/how-cassandra-read-persists-data-and-maintain-consistency

0
0

查看评论
* 以上用户言论只代表其个人观点,不代表CSDN网站的观点或立场
    个人资料
    • 访问:280312次
    • 积分:3755
    • 等级:
    • 排名:第8892名
    • 原创:96篇
    • 转载:12篇
    • 译文:2篇
    • 评论:29条
    文章分类
    最新评论