2021SC@SDUSC
Pager介绍
根据上图我们能够大致了解Pager模块在SQLite占据的地位,它是底层模块了,能够对数据库文件进行直接的读与写。在SQLite里面,BTree模块不会直接和磁盘上的数据文件打交道,而是通过Pager对象里面的各种资源来操作数据文件。上图表示,Pager模块包含四个模块,事务管理模块,锁管理模块,缓存模块和日志管理模块,其中事务管理模块的实现依赖于另外三个模块。
对于Lock manager:为了实验数据文件的并发控制,锁管理模块必须确保事务在访问数据页之前,一定先对数据文件上锁。
对于Page cache:当应用程序访问数据库文件时,pager模块会以块为单位进行缓存,每一个连接都有自己独有的pager模块,因此每个连接都有自己独有的缓存。
对于Log manager:是保证了事务的原子性,要么完整写入数据,要么不写入数据。日志记录了事务在更新数据库文件之前的数据,如果数据写入过程中发生异常中断,在下一次连接时会把日志里记录的原始数据还原回数据库文件。
SQLite对数据文件进行控制时,需要创建一个pager对象,通过pager对象内的资源来操作文件。对于同一个文件,一个进程可能有多个pager对象;这些对象之间都是相互独立的。对于共享缓存模式,每个数据文件只有一个pager对象,所有连接共享这个pager对象。
Pager对象
Pager对象是SQLite访问数据文件必不可少的元素,以下是pager对象具体实现代码
英文版:
struct Pager {
sqlite3_vfs *pVfs; /* OS functions to use for IO */
u8 exclusiveMode; /* Boolean. True if locking_mode==EXCLUSIVE */
u8 journalMode; /* One of the PAGER_JOURNALMODE_* values */
u8 useJournal; /* Use a rollback journal on this file */
u8 noSync; /* Do not sync the journal if true */
u8 fullSync; /* Do extra syncs of the journal for robustness */
u8 extraSync; /* sync directory after journal delete */
u8 syncFlags; /* SYNC_NORMAL or SYNC_FULL otherwise */
u8 walSyncFlags; /* See description above */
u8 tempFile; /* zFilename is a temporary or immutable file */
u8 noLock; /* Do not lock (except in WAL mode) */
u8 readOnly; /* True for a read-only database */
u8 memDb; /* True to inhibit all file I/O */
/**************************************************************************
** The following block contains those class members that change during
** routine operation. Class members not in this block are either fixed
** when the pager is first created or else only change when there is a
** significant mode change (such as changing the page_size, locking_mode,
** or the journal_mode). From another view, these class members describe
** the "state" of the pager, while other class members describe the
** "configuration" of the pager.
*/
u8 eState; /* Pager state (OPEN, READER, WRITER_LOCKED..) */
u8 eLock; /* Current lock held on database file */
u8 changeCountDone; /* Set after incrementing the change-counter */
u8 setSuper; /* Super-jrnl name is written into jrnl */
u8 doNotSpill; /* Do not spill the cache when non-zero */
u8 subjInMemory; /* True to use in-memory sub-journals */
u8 bUseFetch; /* True to use xFetch() */
u8 hasHeldSharedLock; /* True if a shared lock has ever been held */
Pgno dbSize; /* Number of pages in the database */
Pgno dbOrigSize; /* dbSize before the current transaction */
Pgno dbFileSize; /* Number of pages in the database file */
Pgno dbHintSize; /* Value passed to FCNTL_SIZE_HINT call */
int errCode; /* One of several kinds of errors */
int nRec; /* Pages journalled since last j-header written */
u32 cksumInit; /* Quasi-random value added to every checksum */
u32 nSubRec; /* Number of records written to sub-journal */
Bitvec *pInJournal; /* One bit for each page in the database file */
sqlite3_file *fd; /* File descriptor for database */
sqlite3_file *jfd; /* File descriptor for main journal */
sqlite3_file *sjfd; /* File descriptor for sub-journal */
i64 journalOff; /* Current write offset in the journal file */
i64 journalHdr; /* Byte offset to previous journal header */
sqlite3_backup *pBackup; /* Pointer to list of ongoing backup processes */
PagerSavepoint *aSavepoint; /* Array of active savepoints */
int nSavepoint; /* Number of elements in aSavepoint[] */
u32 iDataVersion; /* Changes whenever database content changes */
char dbFileVers[16]; /* Changes whenever database file changes */
int nMmapOut; /* Number of mmap pages currently outstanding */
sqlite3_int64 szMmap; /* Desired maximum mmap size */
PgHdr *pMmapFreelist; /* List of free mmap page headers (pDirty) */
/*
** End of the routinely-changing class members
***************************************************************************/
u16 nExtra; /* Add this many bytes to each in-memory page */
i16 nReserve; /* Number of unused bytes at end of each page */
u32 vfsFlags; /* Flags for sqlite3_vfs.xOpen() */
u32 sectorSize; /* Assumed sector size during rollback */
int pageSize; /* Number of bytes in a page */
Pgno mxPgno; /* Maximum allowed size of the database */
i64 journalSizeLimit; /* Size limit for persistent journal files */
char *zFilename; /* Name of the database file */
char *zJournal; /* Name of the journal file */
int (*xBusyHandler)(void*); /* Function to call when busy */
void *pBusyHandlerArg; /* Context argument for xBusyHandler */
int aStat[4]; /* Total cache hits, misses, writes, spills */
#ifdef SQLITE_TEST
int nRead; /* Database pages read */
#endif
void (*xReiniter)(DbPage*); /* Call this routine when reloading pages */
int (*xGet)(Pager*,Pgno,DbPage**,int); /* Routine to fetch a patch */
char *pTmpSpace; /* Pager.pageSize bytes of space for tmp use */
PCache *pPCache; /* Pointer to page cache object */
#ifndef SQLITE_OMIT_WAL
Wal *pWal; /* Write-ahead log used by "journal_mode=wal" */
char *zWal; /* File name for write-ahead log */
#endif
};
部分重要信息翻译后:
struct Pager {
sqlite3_vfs *pVfs; /* 访问磁盘IO所用的文件系统接口 */
u8 exclusiveMode; /* 是否在使用EXCLUSIVE锁 */
u8 journalMode; /* 日志模式 */
u8 useJournal; /* 是否使用日志 */
u8 noSync; /* 如果为真,不对日志写入磁盘,如数据库是内存模式的存储 */
u8 fullSync; /* 修改日志后是否立即刷盘 */
u8 extraSync; /* 删除日志后立即刷新磁盘 */
//以下3个参数最终都会传递给OsSync()决定是否立即刷盘
u8 ckptSyncFlags; /* SYNC_NORMAL or SYNC_FULL for checkpoint */
u8 walSyncFlags; /* SYNC_NORMAL or SYNC_FULL for wal writes */
u8 syncFlags; /* SYNC_NORMAL or SYNC_FULL otherwise */
u8 tempFile; /* 临时文件或只读文件 */
u8 noLock; /* Do not lock (except in WAL mode) */
u8 readOnly; /* True for a read-only database */
u8 memDb; /* 内存模式的数据库,不对磁盘操作 */
/**************************************************************************
在模块内的成员决定pager的state,其他成员决定pager的configuration
** The following block contains those class members that change during
** routine operation. Class members not in this block are either fixed
** when the pager is first created or else only change when there is a
** significant mode change (such as changing the page_size, locking_mode,
** or the journal_mode). From another view, these class members describe
** the "state" of the pager, while other class members describe the
** "configuration" of the pager.
*/
u8 eState; /* Pager state (OPEN, READER, WRITER_LOCKED..) */
u8 eLock; /* Current lock held on database file */
u8 changeCountDone; /* Set after incrementing the change-counter */
u8 setMaster; /* True if a m-j name has been written to jrnl */
u8 doNotSpill; /* Do not spill the cache when non-zero */
u8 subjInMemory; /* True to use in-memory sub-journals */
u8 bUseFetch; /* True to use xFetch() */
u8 hasHeldSharedLock; /* True if a shared lock has ever been held */
Pgno dbSize; /* Number of pages in the database */
Pgno dbOrigSize; /* dbSize before the current transaction */
Pgno dbFileSize; /* Number of pages in the database file */
Pgno dbHintSize; /* Value passed to FCNTL_SIZE_HINT call */
int errCode; /* One of several kinds of errors */
int nRec; /* Pages journalled since last j-header written */
u32 cksumInit; /* Quasi-random value added to every checksum */
u32 nSubRec; /* Number of records written to sub-journal */
Bitvec *pInJournal; /* One bit for each page in the database file */
sqlite3_file *fd; /* File descriptor for database */
sqlite3_file *jfd; /* File descriptor for main journal */
sqlite3_file *sjfd; /* File descriptor for sub-journal */
i64 journalOff; /* Current write offset in the journal file */
i64 journalHdr;