GoldenGate学习笔记(2)_体系结构

GoldenGate学习笔记之体系结构

参考文档 <>P24

 

1     Overview of Architecture

GoldenGate is composed of the following components:

l         Extract

l         Data Pump

l         Replicat

l         Trails or extract files

l         Checkponts

l         Manager

l         Collector

 

2     GoldenGate logical architecture

 

 

3     Overview of Extract

3.1 Extract can be configured in two ways:

1)      Initial loads: Extract Processes extracts a current set of data directly from source tables. ( 数据初始化的一种方式，但平时更多的是使用其它工具来做数据初始化的动作，如Exp/Imp.)

2)      Change synchronization: Extract Processes extracts transactional changes made to table data after the initial synchronization has taken place. DDL changes are also extracted in some supported platform. (实现基于事务变化的数据同步(Online log or Archived Log.)．数据同步的方式是异步的,DDL在某些平台上也支持，但对于一个正常运行的系统，DDL的变化是比较少的.)

 

3.2 Extract extracts all of the data changes that are made to tables that you configure for synchronization, but it only sends the data from committed transactions to the target system. (虽然提取所有的变化，但只发送已提交的事务数据)

 

Extract Process运行在Source system,主要负责数据的抽取和投递到另一端

4     Overview of Data Pump

4.1 A data pump is a secondary Extract group within the source GoldenGate configuration. If a data pump is not used, Extract must send data to remote trail on the target. In a typical configuration that includes a data pump, however, the primary Extract group writes to a trail on the source system. The data pump reads this trail and sends the data across the network to a remote trail on the target. The data pump adds storage flexibility and also serves to isolate the primary Extract process from TCP/IP activity.(Data Pump是可选的，它的引入分离了数据抽取与数据发送)

 

4.2 It can perform. data filtering, mapping, and conversion, or it can be configured in pass-through mode, where data is passively transferred as-is, without manipulation. Pass-through mode increases the throughput of the data pump, because all of the functionality that looks up object definitions is bypassed. (data pump可以用来过虑，转换数据,或配置成Pass-through模式，即不做任何的动作，只是投递数据)

 

4.3 data pump的好处

1) Protection against network and target failures.

2) Data filtering or transformation

3) Consolidating data from many source to a central target

4) Synchronizing one source with multiple targets

 

5     Overview of Replicat

The Replica process runs on the target system.

You can configure Replicat in one of the following ways:

1)      Initial loads: Replicat can apply data to target tables or else route them to a high-speed bulk-load utility.

2)      Change synchronization: Replicat applies extracted transactional changes to target tables using native database calls, statement caches, and local database access. Replicate DDL operations are also applied, if supported. To ensure data and referential integrity, Replicat applies the replicated changes in the same order as those changes were committed to the source database.(数据应用的顺序与提交的顺序一致，从而确保约束的完整性)

 

6    Overview of Trails

5.1 To support the continuous extraction and replication of data changes, GG stores those changes temporarily on disk in a series of files called a TRAIL. A trail can exist on the source or target system, depending on how you configure GG (Local Trail and Remote Trail).Only one Extract process can write to a trail. (Trail的工作形式类似缓存．带来的好处是Separate extraction and replication activities, support data accuracy and fault tolerance)

 

5.2 Extracted data is sent into and out of a trail in large blocks. By default, GG writes data to the trail in UNIVERESAL DATA FORMAT, a proprietary format which allows it to be exchanged rapidly and accurately among heterogeneous database. However, data can be written in other formats that are compatible with different applications.

 

5.3 The records in GoldenGate trails contain header and data areas. The header contains information about the transaction environment, and the data area contains the actual data values that were extracted. (Trail file的内容，可通过Logdump, 或者直接用Strings –af xxx 来查看)

 

5.4 By default, each file in a trail is 10 MB in size. All file names in a trail begin with the same two characters, which you assign when you create the trail. As files are created, each name is appended with a unique six-digit serial (sequence) number from 000000 through 999999, for example \ggs\dirdat\tr000001. By default, trails are stored in the dirdat sub-directory of the GoldenGate directory. (Trail file的命名格式与存储路径)

 

6     Overview of Extract files

When processing a none-time run, such as an initial load or a batch run that synchronizes transactional changes, GoldenGate stores the extracted changes in an extract file instead of a trail. It is similar to a trail, except that checkpoints are not recorded. (类似Trail,除了不记CHCEKPOINTS)

 

7     Overview of checkpoints

7.1 Checkpoints store the current read and write position of a process to disk for recovery purpose. Checkpoints ensure that database changes marked for synchronization are extracted by Extract and replicated by Replicat.

7.2 Extract creates checkpoints for its positions in the data source and in the trail. Replicat creates checkpoints for its position in the trail.

 

7.3 Checkpoint information is maintained in checkpoint files within the dirchk sub-directory of the GG directory. Optionally, Replicat’s checkpoints can be maintained in a checkpoint table within the target database, in addition to a standard checkpoint file.

 

Checkpoint file用来记录Extract Process and Replicate Process 进程的读写位置，保证一致性．

如何查看Checkpoint 信息?

 

8     Overview of Manager

l         Monitor and restart GoldenGate processes.

l         Issue threshold reports, for example when throughput slows down or when synchronization latency increases.

l         Maintain trail files and logs.

l         Allocate data storage space.

l         Report errors and events.

l         Receive and route user requests from the user interface

 

9     Overview of Collector

Collector is a process that runs in the background on the target system. Collector receives extracted database changes that are sent across the TCP/IP network, and it writes them to a trail or extract file.(在目标端接收数据，由Manager进程自动启动)

 

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/10248702/viewspace-624590/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/10248702/viewspace-624590/