xstream-guide_19c 文档笔记

好长的文档，周末看了100多页，记录些概念

 

XStream 由DB组件和API接口组成，能让客户端应用接收源DB数据变化并发送至目标端，目标端可以是非oracle，例如sqlserver/pg/文件系统/第三方软件应用等。

XStream是一个程序接口，允许客户端应用连接到oracle数据库并且之间访问db capture process或apply process

XStream 有两个主要部分：XStream Out 和 XStream In

XStream Out在源端，从redolog中接收源端的DML,DDL语句，并将变化情况通过接口发送至客户端应用，也可以选择将变化情况存储到memory cache或指定文件中

 

XStream In在目标端，apply从客户端应用接收到的数据变化情况，使目标库同步源库数据。

 

实现基础：

1. 数据变化情况使用 logical change record (LCR) 格式存储

一条dml语句可能会操作多行，因此可能会对应产生多条row LCR记录；如果是ddl语句，则会对应产生一条DDL LCR记录.

1. 规则和规则集用于控制XStream行为，规则包括inclusion and exclusion rules，支持在DB级、schema级、表级、行/列级设置。
2. 基于规则的数据变化会被捕获
3. 支持DB中众多数据类型，包括LOBs, LONG, LONG RAW, and XMLType
4. 支持配置一对多、多对一等情况
5. 支持并行apply、sql生成、冲突检测和解决、错误处理、使用apply handlers定制apply

 

使用前提

有oracle DB相关知识

有分布式数据库相关知识

有sql和plsql知识

有应用编程知识，需要使用OCI或JAVA API创建客户端应用并与XStream交互

 

XStream安全模型

XStream支持两类安全模式：XStream Trusted User Model（可管理XStream配置，有更多DB权限，可通过DBA_视图监控XStream运行情况，更易实现）、XStream Untrusted User Model（只能管理XStream配置，可通过ALL_视图监控XStream运行情况）

 

 

• XStream相关概念

主要有以下几项

1. Logical Change Records (LCRs)
2. Rules and Rule Sets
3. Rule-Based Transformations
4. XStream and the Oracle Replication Performance Advisor
5. XStream and SQL Generation

 

Logical Change Records (LCRs)

LCR是一种有特定格式的用于模式DB变化情况的消息（An LCR is a message with a specific format that describes a database change.），它是XStream中描述数据变化的基本单位（ an LCR is the basic unit of information that describes a database change）

 

在XStream Out 配置中，capture process 会捕获LCR信息并将它们发送到 outbound server， outbound server再将LCR发送到客户端应用

 

在XStream In 配置中，客户端应用将LCR发送到inbound server，inbound server可以将LCR发送到DB直接进行apply，也可以对LCR进行客制化处理

 

 

LCR有三种类型： row LCRs, DDL LCRs, and sequence LCRs（包含序列值的row LCR）

LCR都有LCRID，用于标志和排序LCR，通过它可以确定哪些LCR和事务已被接收和应用

 

row LCR 的类型为LCR$_ROW_RECORD，有以下属性：

Attribute	Description
source_database_name	The name of the source database where the row change occurred. If the LCRs originated in a multitenant container database (CDB), then this attribute specifies the global name container where the row change occurred.
command_type	The type of DML statement that produced the change, either INSERT, UPDATE, DELETE, LOB ERASE, LOB WRITE, or LOB TRIM.
object_owner	The schema name that contains the table with the changed row.
object_name	The name of the table that contains the changed row.
tag	A raw tag that you can use to track the LCR.
transaction_id	The identifier of the transaction in which the DML statement was run.
scn	The system change number (SCN) at the time when the change was made.
old_values	The old column values related to the change. These are the column values for the row before the DML change. If the type of the DML statement is UPDATE or DELETE, then these old values include some or all of the columns in the changed row before the DML statement. If the type of the DML statement is INSERT, then there are no old values. For UPDATE and DELETE statements, row LCRs created by a capture process can include some or all of the old column values in the row.
new_values	The new column values related to the change. These are the column values for the row after the DML change. If the type of the DML statement is UPDATE or INSERT, then these new values include some or all of the columns in the changed row after the DML statement. If the type of the DML statement is DELETE, then there are no new values. For UPDATE and INSERT statements, row LCRs created by a capture process can include some or all of the new column values in the row.
position	A unique identifier of RAW data type for each LCR. The position is strictly increasing within a transaction and across transactions. LCR position is commonly used in XStream configurations. See "Position Order in an LCR Stream".
root_name	If the LCR originated in a CDB, then this attribute specifies the global name of the root in the CDB. If the LCR originated in a non-CDB, then this attribute is the same as the source_database_name attribute.

额外属性

Attribute	Description
commit_scn	The commit system change number (SCN) of the transaction to which the LCR belongs.
commit_scn_from_position	The commit system change number (SCN) of a transaction determined by the input position, which is generated by an XStream outbound server.
commit_time	The commit time of the transaction to which the LCR belongs.
compatible	The minimal database compatibility required to support the LCR.
instance_number	The instance number of the database instance that made the change that is encapsulated in the LCR. Typically, the instance number is relevant in an Oracle Real Application Clusters (Oracle RAC) configuration.
lob_information	The LOB information for the column, such as NOT_A_LOB or LOB_CHUNK.
lob_offset	The LOB offset for the specified column in the number of characters for CLOB columns and the number of bytes for BLOB columns.
lob_operation_size	The operation size for the LOB column in the number of characters for CLOB columns and the number of bytes for BLOB columns.
long_information	The LONG information for the column, such as NOT_A_LONG or LONG_CHUNK.
row_text	The SQL statement for the change that is encapsulated in the row LCR.
scn_from_position	The SCN of the LCR.
source_time	The time when the change in an LCR captured by a capture process was generated in the redo log of the source database, or the time when a persistent LCR was created.
xml_information	The XML information for the column, such as NOT_XML, XML_DOC, or XML_DIFF.

 

DDL LCR 的类型为LCR$_DDL_RECORD，有以下属性：

Table 2-3 Attributes Present in All DDL LCRs

Attribute	Description
source_database_name	The name of the source database where the DDL change occurred. If the LCRs originated in a CDB, then this attribute specifies the global name of the container where the DDL change occurred.
command_type	The type of DDL statement that produced the change, for example ALTER TABLE or CREATE INDEX.
object_owner	The schema name of the user who owns the database object on which the DDL statement was run.
object_name	The name of the database object on which the DDL statement was run.
object_type	The type of database object on which the DDL statement was run, for example TABLE or PACKAGE.
ddl_text	The text of the DDL statement.
logon_user	The logon user, which is the user whose session executed the DDL statement.
current_schema	The schema that is used if no schema is specified for an object in the DDL text.
base_table_owner	The base table owner. If the DDL statement is dependent on a table, then the base table owner is the owner of the table on which it is dependent.
base_table_name	The base table name. If the DDL statement is dependent on a table, then the base table name is the name of the table on which it is dependent.
tag	A raw tag that you can use to track the LCR.
transaction_id	The identifier of the transaction in which the DDL statement was run.
scn	The system change number (SCN) at the time when the change was made.
position	A unique identifier of RAW data type for each LCR. The position is strictly increasing within a transaction and across transactions. LCR position is commonly used in XStream configurations. See "Position Order in an LCR Stream".
edition_name	The name of the edition in which the DDL statement was executed.
root_name	If the LCR originated in a CDB, then this attribute specifies the global name of the root in the CDB. If the LCR originated in a non-CDB, then this attribute is the same as the source_database_name attribute.

 

额外属性

Table 2-4 Additional Attributes in DDL LCRs Captured by a Capture Process

Attribute	Description
commit_scn	The commit system change number (SCN) of the transaction to which the LCR belongs.
commit_scn_from_position	The commit SCN of a transaction determined by the input position, which is generated by an XStream outbound server.
commit_time	The commit time of the transaction to which the LCR belongs.
compatible	The minimal database compatibility required to support the LCR.
instance_number	The instance number of the database instance that made the change that is encapsulated in the LCR. Typically, the instance number is relevant in an Oracle Real Application Clusters (Oracle RAC) configuration.
scn_from_position	The SCN of the LCR.
source_time	The time when the change in an LCR captured by a capture process was generated in the redo log of the source database, or the time when a persistent LCR was created.

 

Extra Information in Row LCRs and DDL LCRs

In addition to the information discussed in the previous sections, row LCRs and DDL LCRs optionally can include extra information (or LCR attributes).

The extra attributes in LCRs are described in the following table.

Table 2-5 Extra Attributes in LCRs

Attribute	Description
row_id	The rowid of the row changed in a row LCR. This attribute is not included in DDL LCRs or row LCRs for index-organized tables.
serial#	The serial number of the session that performed the change captured in the LCR.
session#	The identifier of the session that performed the change captured in the LCR.
thread#	The thread number of the instance in which the change captured in the LCR was performed. Typically, the thread number is relevant only in an Oracle Real Application Clusters (Oracle RAC) environment.
tx_name	The name of the transaction that includes the LCR.
username	The name of the current user who performed the change captured in the LCR.

You can use the INCLUDE_EXTRA_ATTRIBUTE procedure in the DBMS_CAPTURE_ADM package to instruct a capture process to capture one or more extra attributes.

Sequence LCR是包含序列值的row LCR

可以通过如下方法捕获

DB层：将capture_sequence_nextval 和apply_sequence_nextval 参数设为y

OCI接口：use the OCILCRNew function and the OCILCRHeaderSet function with the OCI_ROWLCR_SEQ_LCR flag.

JAVA接口： use the DefaultRowLCR constructor and setSequenceLCRFlag method

 

 

如何定位LCR流顺序

 

每个LCR都有position属性，用于标志该LCR在LCR流和事务中的顺序

position属性有以下特点：

每个LCR的position属性是唯一的

position属性是RAW data type.

Position随LCR流和事务严格递增

• The position is byte-comparable, and the comparison results for multiple positions determines the ordering of the LCRs in the stream.
• The position of an LCR remains identical when the database, the client application, or an XStream component restarts.
• The position is not affected by any rule changes that might reduce or increase the number of LCRs in the stream.

 

XStream Out only sends committed data, and XStream In only receives committed data.

 

LCRID用于在XStream Out中唯一标志LCR的位置，它同样是严格递增的。

从12.2开始，the LCRID is versioned。当创建或新增outbound server时，可以选择使用的版本。要使用version 2，DB版本必须高于12.2。低于12.2版本默认会使用version 1,12.2开始默认使用version 2。如果源库版本为12.2或以上，目标库版本低于源库，可能需要手动设置该参数。

outbound server创建或添加完成后，不能再修改LCRID version，如果要改，只能drop该outbound server重建。如果outbound server正在发送LCR到inbound server，还需要drop该inbound server重建。

 

相同的DB变化在version 1和version 2中有不同的值，可以使用DBMS_XSTREAM_ADM 包可用于比较不同版本的LCRID（使用COMPARE_POSITION 函数）或将一个版本的LCRID转为另一个版本（使用CONVERT_POSITION 函数）

 

 

规则与规则集

 

Rules and Rule Sets Defined

规则是一个数据库对象，定义客户端执行某种动作的条件。在XStream配置中，规则标志哪些LCRs会被传送到另一个组件

A rule is a database object that enables a client to perform an action when an event occurs and a condition is satisfied. In an XStream configuration, rules identify which LCRs to stream from one component to another.

Capture processes, propagations, outbound servers and inbound servers都可以配置规则，规则适用于dml和ddl语言。

规则集是规则的集合，可以配置positive rule set and a negative rule set ，并且negative rule优先

 

Rule Sets and XStream Components

当database change符合规则时，XStream component会执行指定的任务

如果redolog中的change符合为capture process设置的规则，它会被捕获，如果不满足则被略过。

如果捕获到的LCR满足为propagations设置的规则，它会被发送，如果不满足则被略过。

如果被发送的LCR满足为outbound servers设置的规则，它会被发送到客户端应用，如果不满足则被略过。

如果inbound servers接收到的LCR满足为其设置的规则，它会被apply，如果不满足则被略过。

如果不设置规则，所有的database changes都会被捕获、发送、apply

 

System-Created Rules and XStream

A system-created rule is created by the DBMS_XSTREAM_ADM package.

Table 2-6 XStream System-Created Rule Procedures

Procedure	Capture Process	Propagation	Outbound Server	Inbound Server
CREATE_OUTBOUND	Yes	No	Yes	No
ADD_OUTBOUND	No	No	Yes	No
ALTER_OUTBOUND	Yes	No	Yes	No
ADD_GLOBAL_RULES	Yes	No	Yes	Yes
ADD_GLOBAL_PROPAGATION_RULES	No	Yes	No	No
ADD_SCHEMA_RULES	Yes	No	Yes	Yes
ADD_SCHEMA_PROPAGATION_RULES	No	Yes	No	No
ADD_GLOBAL_RULES	Yes	No	Yes	Yes
ADD_SUBSET_OUTBOUND_RULES	No	No	Yes	No
ADD_SUBSET_RULES	Yes	No	Yes	Yes
ADD_SUBSET_PROPAGATION_RULES	No	Yes	No	No
ADD_TABLE_RULES	Yes	No	Yes	Yes
ADD_TABLE_PROPAGATION_RULES	No	Yes	No	No

 

 

system-created rule可以设置为Subset 级、表级、schema级、global级（对整个DB生效）。Subset 级仅对表中部分行生效，类似where语句，需要开启Supplemental logging才能使用。

 

多租户环境中的system-created rule

Table 2-7 Key Procedure Parameters for System-Created Rules in a CDB

Parameter	Description
source_database	The global name of the source database. In a CDB, specify the global name of the container to which the rules pertain. The container can be the CDB root, a PDB, an application root, or an application PDB. The following are examples: mycdb.example.com or hrpdb.example.com.
source_root_name	The global name of the CDB root in the source CDB. The following are examples: mycdb.example.com.
source_container_name	The short name of the source container. The container can be the CDB root, a PDB, an application root, or an application PDB. The following are examples: CDB$ROOT or hrpdb.

Table 2-8 Local Capture and XStream Out Container Rule Conditions

source_database Parameter Setting	source_container_name Parameter Setting	Description
NULL	NULL	XStream Out captures and streams changes made in any container in the local CDB, including the CDB root, all PDBs, all application roots, and all application PDBs.
non-NULL	NULL	XStream Out captures and streams changes made in the specified source container of the local CDB. The source container can be the CDB root, a PDB, an application root, or an application PDB. The DBMS_XSTREAM_ADM procedure queries the CDB_PDBS view and CDB_PROPERTIES view to determine the source_container_name value.
NULL	non-NULL	XStream Out captures and streams changes made in the specified source container of the local CDB. The source container can be the CDB root, a PDB, an application root, or an application PDB. The DBMS_XSTREAM_ADM procedure queries the CDB_PDBS view and CDB_PROPERTIES view to determine the source_database value.
non-NULL	non-NULL	XStream Out captures and streams changes made in the specified source container of the local CDB. The source container can be the CDB root, a PDB, an application root, or an application PDB. If the prefix of the source_database value is different from the source_container_name value, then the resulting rules include a condition for the source_database value, and an internal table maps the source_database value to the source_container_name value.

Downstream Capture and XStream Out Container Rule Conditions

source_database Parameter Setting	source_container_name Parameter Setting	Description
NULL	NULL	XStream Out captures and streams changes made in any container in the remote source CDB, including the CDB root, all PDBs, all application roots, and all application PDBs.
non-NULL	NULL	XStream Out captures and streams changes made in the specified source container of the remote source CDB. The source container can be the CDB root, a PDB, an application root, or an application PDB. The DBMS_XSTREAM_ADM procedure derives the source_container_name value from the prefix of source_database value.
NULL	non-NULL	The DBMS_XSTREAM_ADM procedure raises an error.
non-NULL	non-NULL	XStream Out captures and streams changes made in the specified source container of the remote source CDB. The source container can be the CDB root, a PDB, an application root, or an application PDB. If the prefix of the source_database value is different from the source_container_name value, then the resulting rules include a condition for the source_database value, and an internal table maps the source_database value to the source_container_name value.

 

Rule-Based Transformations

rule-based transformation是对符合positive 规则的LCR进行指定的转换。这个转换最好是放在客户端应用处理，如果实在不愿意放，db端也可以对DML LCR进行些简单转换。

 

Declarative Rule-Based Transformations

You specify (or declare) such a transformation using one of the following procedures in the DBMS_XSTREAM_ADM package:

• ADD_COLUMN either adds or removes a declarative transformation that adds a column to a row LCR.
• DELETE_COLUMN either adds or removes a declarative transformation that deletes a column from a row LCR.
• KEEP_COLUMNS either adds or removes a declarative transformation that keeps a list of columns in a row LCR. The transformation removes columns that are not in the list from the row LCR.
• RENAME_COLUMN either adds or removes a declarative transformation that renames a column in a row LCR.
• RENAME_SCHEMA either adds or removes a declarative transformation that renames the schema in a row LCR.
• RENAME_TABLE either adds or removes a declarative transformation that renames the table in a row LCR.

 

By default, Oracle Database performs declarative transformations in the following order when the rule evaluates to TRUE:

1. Keep columns
2. Delete column
3. Rename column
4. Add column
5. Rename table
6. Rename schema

 

In addition to declarative rule-based transformations, a row migration is an internal transformation that takes place when a subset rule evaluates to TRUE.

You can use the DBMS_XSTREAM_ADM.ADD_SUBSET_RULES procedure to add subset rules. If both types of transformations are specified for a single rule, then Oracle Database performs the transformations in the following order when the rule evaluates to TRUE:

1. Row migration
2. Declarative rule-based transformation

 

User-Specified Declarative Transformation Ordering

If you do not want to use the default declarative rule-based transformation ordering for a particular rule, then you can specify step numbers for each declarative transformation specified for the rule.

 

使用 rule-based transformations时需考虑项

只会对符合positive rule的LCR生效

 rule-based transformations与使用DBMS_TRANSFORM 包进行转换是不同的

如果在XStream In中有大量row LCRs 需要转换，可以使用DML handlers，当然最好还是放在客户端应用模块进行处理

 

 

XStream and the Oracle Replication Performance Advisor

Oracle Replication Performance Advisor 由一系列数据字典视图组成，用于监控XStream性能

 

Performance Advisor tracks the following types of components in an XStream environment:

• QUEUE
• CAPTURE
• PROPAGATION SENDER
• PROPAGATION RECEIVER
• APPLY

 

Performance Advisor 能帮忙寻找组件瓶颈并找到最忙的组件，可以通过DBA_STREAMS_TP_PATH_BOTTLENECK 视图ACTION_NAME 列查看，一般最忙的会是客户端应用，对应ACTION_NAME为EXTERNAL 

 

XStream Out Apply Subcomponents

There are several XStream Out apply subcomponents types.

The following subcomponent types are possible:

• PROPAGATION SENDER+RECEIVER for sending LCRs from a capture process to an outbound server where the capture process and outbound server are in different databases.
• APPLY READER for a reader server. APPLY READER receives LCRs from the capture process, organizes them into transactions, does dependency calculations, and passes the LCRs to the apply coordinator.
• APPLY COORDINATOR for a coordinator process. It takes the transactions from the capture process, uses the dependency information to determine how to schedule the transactions and sends the LCRs to the apply server.
• APPLY SERVER for an apply server. It delivers the LCRs to the client application.

 

XStream In Apply Subcomponents

There are several XStream In apply subcomponents types.

The following subcomponent types are possible:

• APPLY READER for a reader server. It takes the LCRs from client application converts them into transactions, checks the transactional order and does dependency calculations.
• APPLY COORDINATOR for a coordinator process. It takes the transactions from the reader server, uses the dependency information to determine how to schedule the transactions and sends the LCRs to the apply server.
• APPLY SERVER for an apply server. It applies the LCRs to an apply handler. If the LCR cannot be applied, it is placed into an error queue.

 

Performance Advisor可以跟踪组件级统计信息

The Performance Advisor tracks the following component-level statistics:

• The MESSAGE APPLY RATE is the average number of LCRs applied each second by the apply process, outbound server, or inbound server.
• The TRANSACTION APPLY RATE is the average number of transactions applied by the apply process, outbound server, or inbound server each second. Transactions typically include multiple LCRs.

 

Performance Advisor还能跟踪 LATENCY component-level statistics

LATENCY is defined in the following ways:

• For apply processes, the LATENCY is the amount of time between when the LCR was created at a source database and when the LCR was applied by the apply process at the destination database.
• For outbound servers, the apply LATENCY is amount of time between when the LCR was created at a source database and when the LCR was sent to the XStream client application.
• For inbound servers, the apply LATENCY is amount of time between when the LCR was created by the XStream client application and when the LCR was applied by the apply process.

 

You can collect XStream statistics with the UTL_RPADV package

Run the utlrpadv.sql script in the rdbms/admin directory in ORACLE_HOME to load the UTL_RPADV package

exec UTL_RPADV.COLLECT_STATS

exec UTL_RPADV.START_MONITORING

The SHOW_STATS procedure in the UTL_RPADV package displays the statistics that the Performance Advisor gathered and stored.

Use the path_stat_table parameter to specify the table that contains the statistics.

 

SELECT SHOW_STATS_TABLE FROM STREAMS$_PA_MONITORING;

或

SET SERVEROUTPUT ON SIZE 50000

BEGIN

  UTL_RPADV.SHOW_STATS(

    path_stat_table => 'STREAMS$_PA_SHOW_PATH_STAT');

END;

/

 

The SHOW_STATS_HTML procedure in the UTL_RPADV package creates an HTML report that contains the statistics that the Performance Advisor gathered and stored.

XStream and SQL Generation

SQL generation is the ability to generate the SQL statement required to perform the change encapsulated in a row LCR.

XStream outbound servers and XStream inbound servers can use SQL generation to generate the SQL statement necessary to perform the insert, update, or delete operation in a row LCR.

SQL generation能利用row LCR反向解析出sql语句，可以为 inline values格式和带绑定变量的格式，支持大部分数据类型

 

You can use the following interfaces to perform SQL generation:

• The PL/SQL interface, which uses the GET_ROW_TEXT and GET_WHERE_CLAUSE member procedures for row LCRs
• The OCI for XStream
• The Java interface for XStream

 

Part II XStream Out

主要有以下五部分

• XStream Out Concepts
  Become familiar with concepts related to XStream Out.
• Configuring XStream Out
  You can configure the Oracle Database components that are used by XStream Out.
• Managing XStream Out
  You can manage XStream Out components and their rules.
• Monitoring XStream Out
  You can monitor an XStream Out configuration.
• Troubleshooting XStream Out
  You can diagnose and correct problems with an XStream Out configuration.

 

XStream Out Concepts

XStream Out can capture transactions from the redo log of an Oracle database and send them efficiently to a client application.

 

Capture Processes

A capture process is an optional Oracle background process that scans the database redo log to capture DML and DDL changes made to database objects.

When a capture process captures a database change, it converts it into a specific message format called a logical change record (LCR). In an XStream Out configuration, the capture process sends these LCRs to an outbound server.

 

可以通过V$XSTREAM_CAPTURE的STATE 列查看其状态。

When a capture process runs on its source database, the capture process is a local capture process.

You can also capture changes for the source database by running the capture process on different server. When a capture process runs on a remote database, the capture process is called a downstream capture process, and the remote database is called the downstream database

 

3.2.2.1 ID Key LCRs

An ID key LCR is a special type of row LCR. ID key LCRs enable an XStream client application to process changes to rows that include unsupported data types.

XStream Out does not fully support the following data types in row LCRs:

• ROWID
• Nested tables
• The following Oracle-supplied types: ANYTYPE, ANYDATASET, URI types, SDO_TOPO_GEOMETRY, SDO_GEORASTER, and Expression.

 

Local Capture and Downstream Capture

You can configure a capture process to run locally on a source database or remotely on a downstream database.

 

Local Capture

Local capture means that a capture process runs on the source database.

Configuration and administration of the capture process is simpler than when downstream capture is used.

A local capture process can scan changes in the online redo log before the database writes these changes to an archived redo log file. 

 

Downstream Capture

Downstream capture means that a capture process runs on a database other than the source database.

有两种类型: real-time downstream capture and archived-log downstream capture.由downstream_real_time_mine 参数设置

real-time 类型优点在于实时性高，不需要等db change写入redolog和archivelog即可捕获。

 

archived-log类型优点在于允许downstream database有多个源库。它需要将归档日志copy到downstream database，可以用自带的DBMS_FILE_TRANSFER 包，也可以直接用ftp传

 

A real-time downstream capture process and one or more archived-log downstream capture processes can coexist（共存） at a downstream database. With downstream capture, the redo log files of the source database must be available at the downstream database.

The following are the advantages of using downstream capture:

捕获change占用源库资源更少，因为基本是在downstream库中完成的

使得多源库的数据同步更简单

将归档数据copy到downstream库相当于给源库多做了一份归档备份，可用于数据恢复

提高了灵活性和可扩展性

 

The following are operational requirements for using downstream capture:

• The source database must be running at least Oracle Database 10g Release 2 (10.2).
• The XStream Out downstream capture database must be running Oracle Database 11g Release 2 (11.2.0.3) or later and the source database must be running Oracle Database 10g Release 2 (10.2) or later.
• The operating system on the source and downstream capture sites must be the same, but the operating system release does not need to be the same. In addition, the downstream sites can use a directory structure that is different from the source site.
• The hardware architecture on the source and downstream capture sites must be the same. For example, a downstream capture configuration with a source database on a 64-bit Sun system must have a downstream database that is configured on a 64-bit Sun system. Other hardware elements, such as the number of CPUs, memory size, and storage configuration, can differ in the source and downstream sites.

 

 Capture Process Subcomponents

The capture process subcomponents are a reader server, one or more preparer servers, and a builder server.

A capture process is an optional Oracle background process whose process name is CPnn, where nn can include letters and numbers. A capture process captures changes from the redo log by using the infrastructure of LogMiner. XStream configures LogMiner automatically. You can create, alter, start, stop, and drop a capture process, and you can define capture process rules that control which changes a capture process captures.

 

When capture process parallelism is greater than 0, the capture process consists of the following subcomponents:

• One reader server that reads the redo log and divides the redo log into regions.
• One or more preparer servers that scan the regions defined by the reader server in parallel and perform prefiltering of changes found in the redo log. Prefiltering involves sending partial information about changes, such as schema and object name for a change, to the rules engine for evaluation, and receiving the results of the evaluation. You can control the number of preparer servers using the parallelism capture process parameter.
• One builder server that merges redo records from the preparer servers. These redo records either evaluated to TRUE during partial evaluation or partial evaluation was inconclusive for them. The builder server preserves the system change number (SCN) order of these redo records and passes the merged redo records to the capture process.

The capture process (CPnn) performs the following actions for each change when it receives merged redo records from the builder server:

• Formats the change into an LCR
• If the partial evaluation performed by a preparer server was inconclusive for the change in the LCR, then sends the LCR to the rules engine for full evaluation
• Receives the results of the full evaluation of the LCR if it was performed
• Discards the LCR if it satisfies the rules in the negative rule set for the capture process or if it does not satisfy the rules in the positive rule set
• Enqueues the LCR into the queue associated with the capture process if the LCR satisfies the rules in the positive rule set for the capture process

 

Capture Process Checkpoints and XStream Out

A capture process tries to record a checkpoint at regular intervals called checkpoint intervals.

• Required Checkpoint SCN
  The system change number (SCN) that corresponds to the lowest checkpoint for which a capture process requires redo data is the required checkpoint SCN.
• Maximum Checkpoint SCN
  The SCN that corresponds to the last physical checkpoint recorded by a capture process is the maximum checkpoint SCN.
• Checkpoint Retention Time
  The checkpoint retention time is the amount of time, in number of days, that a capture process retains checkpoints before purging them automatically.

 

SCN Values Related to a Capture Process

Specific system change number (SCN) values are important for a capture process.

You can query the ALL_CAPTURE data dictionary view to display these values for one or more capture processes.

 

• Captured SCN and Applied SCN
  The captured SCN is the SCN that corresponds to the most recent change scanned in the redo log by a capture process. The applied SCN for a capture process is the SCN of the most recent LCR processed by the relevant outbound server.
• First SCN and Start SCN
  The first SCN and start SCN are important for a capture process.

 

Outbound Servers

An outbound server is an optional Oracle background process that sends database changes to a client application.

Specifically, a client application can attach to an outbound server and extract database changes from LCRs. A client application attaches to the outbound server using OCI or Java interfaces.

 

A client application can create multiple sessions. Each session can attach to only one outbound server, and each outbound server can serve only one session at a time. However, different client application sessions can connect to different outbound servers or inbound servers.

 

Change capture can be performed on the same database as the outbound server or on a different database. When change capture is performed on a different database from the one that contains the outbound server, a propagation sends the changes from the change capture database to the outbound server database. Downstream capture is also a supported mode to reduce the load on the source database.

 

An outbound server consists of a reader server, a coordinator process, and an apply server.

A reader server that receives LCRs from the outbound server's capture process. The reader server then returns the assembled transactions to the coordinator process.视图V$XSTREAM_APPLY_READER 

A coordinator process that gets transactions from the reader server and passes them to apply servers. The coordinator process name is APnn  视图V$XSTREAM_APPLY_COORDINATOR

An apply server that sends LCRs to an XStream client application. The apply server is a process. If the apply server encounters an error, then it then it records information about the error in the ALL_APPLY view.视图 V$XSTREAM_APPLY_SERVER 

The reader server and the apply server process names are ASnn

 

未完待续

参考

https://docs.oracle.com/en/database/oracle/oracle-database/19/xstrm/xstream-out-concepts.html#GUID-67059465-CB08-442F-A587-B677227110E6