Postgres2015全国用户大会将于11月20至21日在北京丽亭华苑酒店召开。本次大会嘉宾阵容强大,国内顶级PostgreSQL数据库专家将悉数到场,并特邀欧洲、俄罗斯、日本、美国等国家和地区的数据库方面专家助阵:
- Postgres-XC项目的发起人铃木市一(SUZUKI Koichi)
- Postgres-XL的项目发起人Mason Sharp
- pgpool的作者石井达夫(Tatsuo Ishii)
- PG-Strom的作者海外浩平(Kaigai Kohei)
- Greenplum研发总监姚延栋
- 周正中(德哥), PostgreSQL中国用户会创始人之一
- 汪洋,平安科技数据库技术部经理
- ……
|
|
# get snapshotsrc_curs.execute("select txid_current_snapshot()")snapshot = src_curs.fetchone()[0]src_db.commit()
# restore old behavioursrc_db.set_isolation_level(oldiso)src_db.commit()
tbl_stat.change_state(TABLE_CATCHING_UP)tbl_stat.change_snapshot(snapshot)self.save_table_state(dst_curs)
def change_snapshot(self, str_snapshot, tag_changed = 1):"""Set snapshot."""if self.str_snapshot == str_snapshot:returnself.log.debug("%s: change_snapshot to %s", self.name, str_snapshot)self.str_snapshot = str_snapshotif str_snapshot:self.from_snapshot = skytools.Snapshot(str_snapshot)else:self.from_snapshot = None
if tag_changed:self.ok_batch_count = 0self.last_tick = Noneself.changed = 1
## Support for PostgreSQL snapshot#
class Snapshot(object):"""Represents a PostgreSQL snapshot.
Example:>>> sn = Snapshot('11:20:11,12,15') // 最小未提交事务11, 最小未分配事务20, 当前未提交事务11,12,15>>> sn.contains(9) // 该snapshot是否包含事务9的内容, 是,因为最小未提交事务是11,9已经提交了。True>>> sn.contains(11) // 该snapshot是否包含事务11的内容, 否,因为11还未提交False>>> sn.contains(17) // 该snapshot是否包含事务9的内容, 是,已提交True>>> sn.contains(20) // 该snapshot是否包含事务9的内容, 否,未分配False"""
def __init__(self, str):"Create snapshot from string."
self.sn_str = strtmp = str.split(':')if len(tmp) != 3:raise Exception('Unknown format for snapshot')self.xmin = int(tmp[0])self.xmax = int(tmp[1])self.txid_list = []if tmp[2] != "":for s in tmp[2].split(','):self.txid_list.append(int(s))
def contains(self, txid):"Is txid visible in snapshot."
txid = int(txid)
if txid < self.xmin:return Trueif txid >= self.xmax:return Falseif txid in self.txid_list:return Falsereturn True
Name | Description |
---|---|
xmin | Earliest transaction ID (txid) that is still active. All earlier transactions will either be committed and visible, or rolled back and dead. |
xmax | First as-yet-unassigned txid. All txids greater than or equal to this are not yet started as of the time of the snapshot, and thus invisible. |
xip_list | Active txids at the time of the snapshot. The list includes only those active txids between xmin and xmax; there might be active txids higher than xmax. A txid that is xmin <= txid < xmax and not in this list was already completed at the time of the snapshot, and thus either visible or dead according to its commit status. The list does not include txids of subtransactions. |
/** SQL function for creating a new logical replication slot.*/Datumpg_create_logical_replication_slot(PG_FUNCTION_ARGS){....../* build initial snapshot, might take a while */DecodingContextFindStartpoint(ctx);......
/** Read from the decoding slot, until it is ready to start extracting changes.*/voidDecodingContextFindStartpoint(LogicalDecodingContext *ctx){....../* Wait for a consistent starting point */ 等待一致性点,所有事务结束for (;;){....../* only continue till we found a consistent spot */if (DecodingContextReady(ctx))break;......}ctx->slot->data.confirmed_flush = ctx->reader->EndRecPtr;......
/** Returns true if an consistent initial decoding snapshot has been built.*/boolDecodingContextReady(LogicalDecodingContext *ctx){return SnapBuildCurrentState(ctx->snapshot_builder) == SNAPBUILD_CONSISTENT;}
typedef enum{/** Initial state, we can't do much yet.*/SNAPBUILD_START,
/** We have collected enough information to decode tuples in transactions* that started after this.** Once we reached this we start to collect changes. We cannot apply them* yet because the might be based on transactions that were still running* when we reached them yet.*/SNAPBUILD_FULL_SNAPSHOT,
/** Found a point after hitting built_full_snapshot where all transactions* that were running at that point finished. Till we reach that we hold* off calling any commit callbacks.*/SNAPBUILD_CONSISTENT} SnapBuildState;
46.2.4. Exported Snapshots
When a new replication slot is created using the streaming replication interface, a snapshot is exported (see Section 9.26.5), which will show exactly the state of the database after which all changes will be included in the change stream. This can be used to create a new replica by using SET TRANSACTION SNAPSHOT to read the state of the database at the moment the slot was created. This transaction can then be used to dump the database's state at that point in time, which afterwards can be updated using the slot's contents without losing any changes.
/** Shared memory state of a single replication slot.*/typedef struct ReplicationSlot{/* lock, on same cacheline as effective_xmin */slock_t mutex;
/* is this slot defined */bool in_use;
/* is somebody streaming out changes for this slot */bool active;
/* any outstanding modifications? */bool just_dirtied;bool dirty;
/** For logical decoding, it's extremely important that we never remove any* data that's still needed for decoding purposes, even after a crash;* otherwise, decoding will produce wrong answers. Ordinary streaming* replication also needs to prevent old row versions from being removed* too soon, but the worst consequence we might encounter there is* unwanted query cancellations on the standby. Thus, for logical* decoding, this value represents the latest xmin that has actually been* written to disk, whereas for streaming replication, it's just the same* as the persistent value (data.xmin).*/TransactionId effective_xmin;TransactionId effective_catalog_xmin;
/* data surviving shutdowns and crashes */ReplicationSlotPersistentData data;
/* is somebody performing io on this slot? */LWLock *io_in_progress_lock;
/* all the remaining data is only used for logical slots */
/* ----* When the client has confirmed flushes >= candidate_xmin_lsn we can* advance the catalog xmin, when restart_valid has been passed,* restart_lsn can be increased.* ----*/TransactionId candidate_catalog_xmin;XLogRecPtr candidate_xmin_lsn;XLogRecPtr candidate_restart_valid;XLogRecPtr candidate_restart_lsn;} ReplicationSlot;