研究主备日志相关实现,那么xlog作为数据刷盘前首先将相关数据信息写入预写入日志(WAL),同时xlog作为Postgres数据库存储部分的重要组成部分,对齐的理解的重要性不言而喻。
首先,直观感受下xlog文件(默认在PGDATA目录pg_xlog子目录下,我们所生活的WAL segment file就是指下面的一个文件)
xlog文件名
xlog文件(常说的redolog)名字的命名方法是在XLogFileName宏里定义的,分别由时间线ID、日志ID、段ID的八位16进制数依次构成。
(1)时间线ID
即:TimeLineID(typedef uint32 TimeLineID;)TLI.用来表示数据库状态,每做一次恢复,都会增加。
(2)日志ID及segmentID
logSegNo源码定义:
#define XLByteToSeg(xlrp, logSegNo) \
logSegNo = (xlrp) / XLogSegSize
#define XLogSegSize ((uint32) XLOG_SEG_SIZE)
XLOG_SEG_SIZE 这个是在源码编译./configure 中定义的(默认16MB):./src/include/pg_config.h:829:#define XLOG_SEG_SIZE (16 * 1024 * 1024)
有关日志ID及segmentID源码中:
而XLogSegmentsPerXLogId的定义为:
#define XLogSegmentsPerXLogId (UINT64CONST(0x100000000) / XLOG_SEG_SIZE)
那么XLogSegmentsPerXLogId 经计算为256。可以看出xlog文件名日志ID最大为0xFFFFFFFFF, segmentID最大0x000000FF, 当xlog名字为0000000100000000000000FF后,下一个则为000000010000000100000000。实验如下:
通过pg_resetxlog -f -l 0000000100000002000000FE ./ 强制执行reset xlog
我们看下pg_xlog下最多能够有多少个xlog文件:
个数:2^8 * 2^32=2^40个,则大小为:2^40*16 MB=2^34 GB=2^24 TB=2^14 PB, 足够。
针对xlog文件个数,英文解释如下:
见postgres自带文档:There will always be at least one WAL segment file, and will normally not be more than (2 + checkpoint_completion_target) * checkpoint_segments + 1 or checkpoint_segments + wal_keep_segments + 1 files. Each segment file is normally 16 MB (though this size can be altered when building the server). You can use this to estimate space requirements for WAL. Ordinarily, when old log segment files are no longer needed, they are recycled (that is, renamed to become future segments in the numbered sequence). If, due to a short-term peak of log output rate, there are more than 3 * checkpoint_segments + 1 segment files, the unneeded segment files will be deleted instead of recycled until the system gets back under this limit.
有关pgxlog该文件对应的操作,请参考:http://blog.sina.com.cn/s/blog_067d069a90101qv54.html
介绍几个有关函数
函数 返回类型 描述
pg_current_xlog_insert_location() text 获取当前事务日志的插入位置
pg_xlogfile_name_offset(locationtext) text,integer 将事务日志的位置字符串转换为文件名并返回在文件中的字节偏移量
pg_xlogfile_name(locationtext) text 将事务日志的位置字符串转换为文件名
postgres=# select pg_current_xlog_location();
pg_current_xlog_location
--------------------------
3/201EBF0
(1 row)
postgres=# \! ls pg_xlog -l
总计 82024
-rw------- 1 wln wln 16777216 11-30 10:11 0000000100000002000000FE
-rw------- 1 wln wln 16777216 11-30 10:12 0000000100000002000000FF
-rw------- 1 wln wln 16777216 11-30 10:12 000000010000000300000000
-rw------- 1 wln wln 16777216 11-30 11:13 000000010000000300000001
-rw------- 1 wln wln 16777216 11-30 12:36 000000010000000300000002
drwx------ 2 wln wln 4096 11-29 22:37 archive_status
postgres=# select pg_xlogfile_name('3/201EBF0');
pg_xlogfile_name
--------------------------
000000010000000300000002
(1 row)
postgres=# select pg_xlogfile_name_offset('3/201EBF0');
pg_xlogfile_name_offset
-----------------------------------
(000000010000000300000002,125936)
(1 row)
参考:
http://www.highgo.com.cn/docs/docs90cn/functions-admin.html
http://blog.itpub.net/30088583/viewspace-1394057/
首先,直观感受下xlog文件(默认在PGDATA目录pg_xlog子目录下,我们所生活的WAL segment file就是指下面的一个文件)
xlog文件名
xlog文件(常说的redolog)名字的命名方法是在XLogFileName宏里定义的,分别由时间线ID、日志ID、段ID的八位16进制数依次构成。
(1)时间线ID
即:TimeLineID(typedef uint32 TimeLineID;)TLI.用来表示数据库状态,每做一次恢复,都会增加。
(2)日志ID及segmentID
logSegNo源码定义:
#define XLByteToSeg(xlrp, logSegNo) \
logSegNo = (xlrp) / XLogSegSize
#define XLogSegSize ((uint32) XLOG_SEG_SIZE)
XLOG_SEG_SIZE 这个是在源码编译./configure 中定义的(默认16MB):./src/include/pg_config.h:829:#define XLOG_SEG_SIZE (16 * 1024 * 1024)
有关日志ID及segmentID源码中:
而XLogSegmentsPerXLogId的定义为:
#define XLogSegmentsPerXLogId (UINT64CONST(0x100000000) / XLOG_SEG_SIZE)
那么XLogSegmentsPerXLogId 经计算为256。可以看出xlog文件名日志ID最大为0xFFFFFFFFF, segmentID最大0x000000FF, 当xlog名字为0000000100000000000000FF后,下一个则为000000010000000100000000。实验如下:
通过pg_resetxlog -f -l 0000000100000002000000FE ./ 强制执行reset xlog
我们看下pg_xlog下最多能够有多少个xlog文件:
个数:2^8 * 2^32=2^40个,则大小为:2^40*16 MB=2^34 GB=2^24 TB=2^14 PB, 足够。
针对xlog文件个数,英文解释如下:
见postgres自带文档:There will always be at least one WAL segment file, and will normally not be more than (2 + checkpoint_completion_target) * checkpoint_segments + 1 or checkpoint_segments + wal_keep_segments + 1 files. Each segment file is normally 16 MB (though this size can be altered when building the server). You can use this to estimate space requirements for WAL. Ordinarily, when old log segment files are no longer needed, they are recycled (that is, renamed to become future segments in the numbered sequence). If, due to a short-term peak of log output rate, there are more than 3 * checkpoint_segments + 1 segment files, the unneeded segment files will be deleted instead of recycled until the system gets back under this limit.
有关pgxlog该文件对应的操作,请参考:http://blog.sina.com.cn/s/blog_067d069a90101qv54.html
介绍几个有关函数
函数 返回类型 描述
pg_current_xlog_insert_location() text 获取当前事务日志的插入位置
pg_xlogfile_name_offset(locationtext) text,integer 将事务日志的位置字符串转换为文件名并返回在文件中的字节偏移量
pg_xlogfile_name(locationtext) text 将事务日志的位置字符串转换为文件名
postgres=# select pg_current_xlog_location();
pg_current_xlog_location
--------------------------
3/201EBF0
(1 row)
postgres=# \! ls pg_xlog -l
总计 82024
-rw------- 1 wln wln 16777216 11-30 10:11 0000000100000002000000FE
-rw------- 1 wln wln 16777216 11-30 10:12 0000000100000002000000FF
-rw------- 1 wln wln 16777216 11-30 10:12 000000010000000300000000
-rw------- 1 wln wln 16777216 11-30 11:13 000000010000000300000001
-rw------- 1 wln wln 16777216 11-30 12:36 000000010000000300000002
drwx------ 2 wln wln 4096 11-29 22:37 archive_status
postgres=# select pg_xlogfile_name('3/201EBF0');
pg_xlogfile_name
--------------------------
000000010000000300000002
(1 row)
postgres=# select pg_xlogfile_name_offset('3/201EBF0');
pg_xlogfile_name_offset
-----------------------------------
(000000010000000300000002,125936)
(1 row)
参考:
http://www.highgo.com.cn/docs/docs90cn/functions-admin.html
http://blog.itpub.net/30088583/viewspace-1394057/