Bdb je的底层存储格式是Log-Structured File System,即就像写日志那样append only,并且其日志和数据是存在一起不分开的。
因此bdb的存储格式是面向entry的,一个entry或代表一个record,或代表一条日志等等。
从源码分析,bdb的entry总共有几十种之多,根据不同的配置使用不同的entry。代码结构是这样的:
je是以一个目录为一个Environment,而一个目录里面可以有多个Database,每个Database可以存储若干key-value,实际上有点像库和表的概念 对于一个Environmrnt来说,其B+tree格式如下:
通过Mapping IN和BIN将不同的Database映射分开。
逻辑上的B+tree node是这样的:
有些db支持duplicate record,因此,对于这种情况下,btree格式如下:
多种entry对应一个Java类,下面依次介绍:
Entry Types | Class | Feature |
LOG_LN_TRANSACTIONAL | LNLogEntry | Leaf Nodes In Transactional |
LOG_LN | LNLogEntry | Leaf Nodes without Transactional |
LOG_MAPLN_TRANSACTIONAL | LNLogEntry | MapLN Nodes In Transactional |
LOG_MAPLN | LNLogEntry | MapLN Nodes Without Transactional |
LOG_NAMELN_TRANSACTIONAL | NameLNLogEntry | Log NameLN when rename or delete database |
LOG_NAMELN | NameLNLogEntry | .. |
LOG_DEL_DUPLN_TRANSACTIONAL | DeletedDupLNLogEntry | Deleted Dup LNs |
LOG_DEL_DUPLN | DeletedDupLNLogEntry | … |
LOG_DUPCOUNTLN_TRANSACTIONAL | LNLogEntry | Count of a Dup tree |
LOG_DUPCOUNTLN | LNLogEntry | … |
LOG_FILESUMMARYLN | LNLogEntry | File summary like records count |
LOG_IN | INLogEntry | Internal Nodes |
LOG_BIN | INLogEntry | Bottom Internal Nodes |
LOG_DIN | INLogEntry | Duplicate Internal Nodes |
LOG_DBIN | INLogEntry | … |
LOG_ROOT | SingleItemEntry | Represent a DbTree |
LOG_TXN_COMMIT | SingleItemEntry | Transaction Commit |
LOG_TXN_ABORT | SingleItemEntry | Transaction Abort |
LOG_CKPT_START | SingleItemEntry | Check point start |
LOG_CKPT_END | SingleItemEntry |
|
LOG_IN_DELETE_INFO | SingleItemEntry | Remove a child from a IN |
LOG_BIN_DELTA | BINDeltaLogEntry | Delta log of BIN |
LOG_DUP_BIN_DELTA | BINDeltaLogEntry |
|
LOG_TRACE | SingleItemEntry | Log a messages like: “Close of environment bk1 started” |
LOG_FILE_HEADER | FileHeaderEntry | The most import data |
LOG_IN_DUPDELETE_INFO | SingleItemEntry |
|
LOG_TXN_PREPARE | SingleItemEntry | Transaction Prepare |
LOG_ROLLBACK_START | SingleItemEntry | Rollback Start |
LOG_ROLLBACK_END | SingleItemEntry | Rollback End |
LOG_MATCHPOINT | SingleItemEntry | For Replication Syncup |
LogManager不关心具体的entry,而是将相应的entry的内容写入磁盘,而具体是什么类型的entry将通过每个entry的头两个字节体现:
Log type(8 bits)
Provisional(2 bits) Replicated(1 bit) Invisible(1 bit) Version(5 bits)
第一个字节代表entry的类型,而第二个字节代表相关环境信息。
这里分析其中一个很重要的entry:LNLogEntry,请看代码:
/*
* Persistent fields in an LN entry.
*/
private LN ln;
private DatabaseId dbId;
private byte[] key;
private long abortLsn = DbLsn.NULL_LSN;
private boolean abortKnownDeleted;
private Txn txn; // conditional
其中的LN又包括以下信息:
private static final String BEGIN_TAG = "<ln>";
private static final String END_TAG = "</ln>";
private byte[] data;
/*
* Flags: bit fields
*
* -Dirty means that the in-memory version is not present on disk.
* -The last logged bits store the total size of the last logged entry.
*/
private static final int DIRTY_BIT = 0x80000000;
private static final int CLEAR_DIRTY_BIT = ~DIRTY_BIT;
private static final int LAST_LOGGED_SIZE_MASK = 0x7FFFFFFF;
private static final int CLEAR_LAST_LOGGED_SIZE = ~LAST_LOGGED_SIZE_MASK;
private int flags; // not persistent
这样最终log到文件中的entry是如下所示:
这里我dump了一个做为示例:
<entry lsn="0x0/0xdd5" type="LN/7" prev="0xdb5"> <ln><node>12</node><data>hidden</data></ln> <dbId id="3"/> <key v="50 50 54 55 48 52 53 56 "/> </entry>
具体访问时,将根据Lsn这个唯一的标识定位,读取entry返回相关的值。
参考文献
1.Berkeley DB Java Edition Architecture
2.Source Code