1.Oracle GoldenGate概述

目录

1、本文背景

2、关于官网抛出的几个关于学习OGG的问题

3、Oracle GoldenGate Product Family

4、OGG的两种架构

5、Oracle GoldenGate Processes and Key Terms

Overview of Target-Initiated Paths

Oracle GoldenGate Key Terms and Concepts

Overview of Process Types

Overview of Groups

Overview of the Commit Sequence Number (CSN)

---

1、本文背景

接触OGG也有好几年的时间了。
OGG适用的异构、同构实时数据同步很多，不过目前在生产实践中只有用到Mysql 实时同步至ORACLE。所以对于其他的同步案例很少用到，关于OGG的知识点也是用到的才比较熟悉，很少用到的知识和一些关键名词也是比较陌生。
趁着现在有时间，想全面的学习下。就记录下来。

2、关于官网抛出的几个关于学习OGG的问题

本文学习记录，官网入口
Introduction to Oracle GoldenGate

Learn aboutOracle GoldenGate concepts, why and when should you use it, and get familiar with some of the basic terminology and keywords associated with Oracle GoldenGate.

Topics:

• What is Oracle GoldenGate?
  Oracle GoldenGate is a comprehensive software package for real-time data integration and replication. It enables high availability solutions, real-time data integration, transactional change data capture, data replication, transformations, and verification between operational and analytical enterprise systems.
• Why Do You Need Oracle GoldenGate?
  Enterprise data is typically distributed across the enterprise in heterogeneous databases. To get data between different data sources, you can use Oracle GoldenGate to load, distribute, and filter transactions within your enterprise in real-time and enable migrations between different databases in near zero-downtime.
• When Do You Use Oracle GoldenGate?
  Oracle GoldenGate meets almost any data movement requirements you might have. Some of the most common use cases are described in this section.
• Topologies for Oracle GoldenGate
  After installation, Oracle GoldenGate can be configured to meet your organization's business needs.
• Oracle GoldenGate Product Family
  There are numerous products in the Oracle GoldenGate product family.

这几个问题都是比较概念、概括性。如果有感兴趣的网友，可以自己阅读官网。
下文就选OGG相关产品介绍下。其他的不记录于此

3、Oracle GoldenGate Product Family

There are numerous products in the Oracle GoldenGate product family.#Oracle GoldenGate产品系列中有许多产品。

• Oracle GoldenGate Veridata : Oracle GoldenGate Veridata compares one set of data to another and identifies data that is out-of-sync, and allows you to repair any data that is found out-of-sync. #Oracle GoldenGate Veridata将一组数据与另一组数据进行比较，识别不同步的数据，并提供修复发现的任何不同步数据的功能。

• Oracle GoldenGate Plug-in for EMCC: The Enterprise Manager Plug-in for Oracle GoldenGate extends the Oracle Enterprise Manager Cloud Control and provides visual support for monitoring and managing Oracle GoldenGateprocesses.#EMCC的Oracle GoldenGate插件。用于Oracle GoldenGate的Enterprise Manager插件扩展了Oracle Enterprise Manager云控制，并为监视和管理Oracle GoldenGateway流程提供了可视化支持。

• Oracle GoldenGate Monitor: Oracle GoldenGate Monitor is a real-time, Web-based monitoring console that delivers an at-a-glance, graphical view of all of the Oracle GoldenGate instances and their associated databases within your enterprise. #Oracle GoldenGate Monitor是一个实时的、基于Web的监控控制台，它提供了企业内所有Oracle GoldenGateway实例及其相关数据库的一目了然的图形视图。

• Oracle GoldenGate for Big Data: Oracle GoldenGate for Big Data contains built-in support to write operation data from Oracle GoldenGate trail records into various Big Data targets (such as, HDFS, HBase, Kafka, Flume, JDBC, Cassandra, and MongoDB). #Oracle GoldenGate for Big数据包含内置支持，可将Oracle GoldenGateway跟踪记录中的操作数据写入各种大数据目标（如HDFS、HBase、Kafka、Flume、JDBC、Cassandra和MongoDB）

• Oracle GoldenGate Application Adapters: Oracle GoldenGate Application Adapters integrate with installations of the Oracle GoldenGatecore product to bring in Java Message Service (JMS) information or to deliver information as JMS messages or files. #Oracle GoldenGate应用程序适配器与Oracle GoldenGatecore产品的安装集成，以引入Java消息服务（JMS）信息或以JMS消息或文件的形式传递信息。

• Oracle GoldenGate for HP NonStop (Guardian): Oracle GoldenGate for HP NonStop enables you to manage business data at a transactional level by extracting and replicating selected data records and transactional changes across a variety of heterogeneous applications and platforms.  #Oracle GoldenGate for HP NonStop使您能够通过在各种异构应用程序和平台上提取和复制选定的数据记录和事务更改，在事务级别管理业务数据。

• Oracle GoldenGate Studio: Oracle GoldenGate Studio enables you to design and deploy high-volume, real-time replication by automatically handling table and column mappings, allowing drag and drop custom mappings, generating best practice configurations from templates, and contains context sensitive help. #Oracle GoldenGate Studio使您能够通过自动处理表和列映射、允许拖放自定义映射、从模板生成最佳实践配置来设计和部署高容量实时复制，并包含上下文相关帮助

4、OGG的两种架构

Oracle GoldenGate supports two architectures, the Classic Architecture and the Microservices Architecture (MA). #Oracle GoldenGate支持两种体系结构，即经典体系结构和微服务体系结构（MA）

Oracle GoldenGate can be configured for the following purposes: #Oracle GoldenGate可以配置用于以下目的：

• A static extraction of data records from one database and the loading of those records to another database. #从一个数据库中静态提取数据记录，并将这些记录加载到另一个数据库。

• Continuous extraction and replication of transactional Data Manipulation Language (DML) operations and data definition language (DDL) changes (for supported databases) to keep source and target data consistent. #连续提取和复制事务性数据操作语言（DML）操作和数据定义语言（DDL）更改（对于支持的数据库），以保持源数据和目标数据的一致性

• Data extraction from supported database sources and replication to Big Data and file targets using Oracle GoldenGate for Big Data. #使用Oracle GoldenGate for Big Data从受支持的数据库源提取数据，并复制到大数据和文件目标。

Oracle GoldenGate Architectures Overview   #Oracle GoldenGate体系结构概述

The following table describes the two Oracle GoldenGate architectures and when you should use each of the architectures.   #下表介绍了两种Oracle GoldenGate体系结构，以及何时应该使用每种体系结构。

X	Classic Architecture	Microservices Architecture
What is it?	Oracle GoldenGate Classic Architecture is the original architecture for enterprise replication. This architecture provides the processes and files required to effectively transfer transactional data across a variety of topologies. These processes and files form the main components of the classic architecture and was the main product installation method until the Oracle GoldenGate 12c (12.3.0.1) release. #Oracle GoldenGate经典体系结构是用于企业复制的原始体系结构。此体系结构提供了跨各种拓扑有效传输事务数据所需的流程和文件。这些过程和文件构成了经典体系结构的主要组件，并且在Oracle GoldenGate 12c（12.3.0.1）发布之前一直是主要的产品安装方法	Oracle GoldenGate Microservices Architecture is a microservices architecture that enables REST services as part of the Oracle GoldenGate environment. The REST-enabled services provide API end-points that can be leveraged for remote configuration, administration, and monitoring through web-based consoles, an enhanced command line interface, PL/SQL and scripting languages. #Oracle GoldenGate微服务架构是一种微服务架构，可将REST服务作为Oracle GoldenGate环境的一部分。REST-enabled服务提供了API端点，可以通过基于web的控制台、增强的命令行界面、PL/SQL和脚本语言来利用这些端点进行远程配置、管理和监控。
When should I use it?	Oracle GoldenGate can be installed and configured to use the Oracle GoldenGate Classic Architecture only if MA release is not available for that platform, mentioned in the following scenarios: #经典架构可用于以下微服务架构不适用的场景 。个人备注：这段官网的原文，翻译起来有点拗口而且并没有他说的只能用于以下场景，当前推出的版本下，除非强制不支持经典结构，不然在生产实践中，经典结构和微架构实现功能上没啥大的区别。也有可能是给人们强推微架构 A static extraction of data records from one database and the loading of those records to another database. #从一个数据库中静态提取数据记录，并将这些记录加载到另一个数据库。 Continuous extraction and replication of transactional Data Manipulation Language (DML) operations and Data Definition Language (DDL) changes (for supported databases) to keep source and target data consistent. #连续提取和复制事务性数据操作语言（DML）操作和数据定义语言（DDL）更改（针对支持的数据库），以保持源数据和目标数据的一致性 Extraction from a database and replication to a file outside the database. #从数据库中提取并复制到数据库外部的文件。 Capture from heterogeneous database sources. #从异构数据库源捕获。	Oracle GoldenGate can be installed and configured to use the Oracle GoldenGate Microservices Architecture for the following purposes: #Oracle GoldenGate可以安装和配置为将Oracle GoldenGate微服务体系结构用于以下目的 Large scale and cloud deployments with fully-secure HTTPS interfaces and Secure WebSockets for streaming data. #大规模和云部署，具有完全安全的HTTPS接口和用于流式数据的安全WebSockets Simpler management of multiple implementations of Oracle GoldenGate environments and control user access for the different aspects of Oracle GoldenGate setup and monitoring. #更简单地管理Oracle GoldenGate环境的多个实施，并控制Oracle GoldenGate设置和监控的不同方面的用户访问 Support system managed database sharding to deliver fine-grained, multi-master replication where all shards are writable, and each shard can be partially replicated to other shards within a shardgroup. #支持系统管理的数据库分片，以提供细粒度的多主复制，其中所有分片都是可写的，并且每个分片可以部分复制到分片组中的其他分片。 Support the following features: #支持以下特性 Thin and browser-based clients #简化和使用基于web的客户端 Network security #网络安全 User Authorization #用户授权 Distributed deployments #分布式部署 Remote administration #远程管理 Performance monitoring and orchestration #性能监控和协调 Coordination with other systems and services in an Oracle Database environment. #与Oracle数据库环境中的其他系统和服务进行协调。 Custom embedding of Oracle GoldenGate into applications or to use secure, remote HTML5 applications. #将Oracle GoldenGate自定义嵌入到应用程序中或使用安全的远程HTML5应用程序
Which databases are supported?	Classic Architecture supports all supported databases as per the certification matrix.	MA only supports the Oracle database for an end-to-end MA-only topology. However, it is possible for a source Oracle GoldenGate Classic associated with heterogeneous databases to replicate to a target Oracle GoldenGate MA with Oracle, or a source Oracle GoldenGate MA with Oracle to replicate to a target Oracle GoldenGate legacy with heterogeneous databases.

Topics:  #以下的内容为支持列表、经典结构、微架构 架构性的介绍，在此就不展开，有兴趣的。自己阅读官网

• Oracle GoldenGate Supported Processing Methods and Databases #Oracle GoldenGate支持的处理方法和数据库
• Components of Oracle GoldenGate Microservices Architecture
  You can use Oracle GoldenGate Microservices Architecture to configure and manage your data replication using an HTML user interface.
• Components of Oracle GoldenGate Classic Architecture
  You can use the Oracle GoldenGate Classic Architecture to configure and manage your data replications from the command line.
• Common Data Replication Processes
  There are a number of data replication processes that are common to both Oracle GoldenGate architectures.

下文重点介绍，经典和微架构常见或者通用的关键术语和概念

5、Oracle GoldenGate Processes and Key Terms

Oracle GoldenGate has common data replication processes and architecture-specific processes as well.

Specific components of Classic Architecture and Microservices Architecture are discussed in Components of Classic Architecture and Components of Microservices Architecture. However, there are various processes and key terms that are common to both the architectures of Oracle GoldenGate. #经典体系结构和微服务体系结构的具体组件在经典体系结构的组件和微服务架构的组件中进行了讨论。然而，Oracle GoldenGate的两种体系结构都有各种通用的过程和关键术语

• Overview of Target-Initiated Paths
• Oracle GoldenGate Key Terms and Concepts
  Apart from the two architectures and their components, there are some key terms that you should get familiar with.

Overview of Target-Initiated Paths

这部分略过。

Target-initiated paths for microservices enable the Receiver Server to initiate a path to the Distribution Service on the target deployment and pull trail files. This feature allows the Receiver Server to create a target initiated path for environments such as Demilitarized Zone Paths (DMZ) or Cloud to on-premise, where the Distribution Server in the source Oracle GoldenGate deployment cannot open network connections in the target environment to the Receiver Server due to network security policies.

If the Distribution Server cannot initiate connections to the Receiver Server, but Receiver Server can initiate a connection to the machine running the Distribution Server, then the Receiver Server establishes a secure or non-secure target initiated path to the Distribution Server through a firewall or Demilitarized (DMZ) zone using Oracle GoldenGate and pull the requested trail files.

The Receiver Server end-points display that the retrieval of the trail files was initiated by the Receiver Server, see Quick Tour of the Receiver Server Home Page.

You can enable this option from the Configuration Assistant wizard Security options, see How to Create Deployments. For steps to create a target-initiated distribution path, see How to Add a Target-Initiated Distrbution Path in Using the Oracle GoldenGate Microservices Architecture.

Oracle GoldenGate Key Terms and Concepts

#OGG关键术语和概念

Apart from the two architectures and their components, there are some key terms that you should get familiar with.

Topics:

• Overview of Process Types
• Overview of Groups
• Overview of the Commit Sequence Number (CSN)

Parent topic: Oracle GoldenGate Processes and Key Terms

Overview of Process Types

#流程类型概述

Depending on the requirement, Oracle GoldenGate can be configured with the following processing types.

• An online Extract or Replicat process that runs until stopped by a user. Online processes maintain recovery checkpoints so that processing can resume after interruptions. You use online processes to continuously extract and replicate DML and DDL operations (where supported) to keep source and target objects synchronized. The EXTRACT and REPLICAT parameters apply to this process type.

• A source-is-table (also known as in initial-load Extract) Extract process extracts a current set of static data directly from the source objects in preparation for an initial load to another database. This process type does not use checkpoints. The SOURCEISTABLE parameter applies to this process type.

• A special-run Replicat process applies data within known begin and end points. You use a special-run Replicat for initial data loads, and it also can be used with an online Extract to apply data changes from the trail in batches, such as once a day rather than continuously. This process type does not maintain checkpoints because the run can be started over with the same begin and end points. The SPECIALRUN parameter applies to this process type.

• A remote task is a special type of initial-load process in which Extract communicates directly with Replicat over TCP/IP. Neither a Collector process nor temporary disk storage in a trail or file is used. The task is defined in the Extract parameter file with the RMTTASK parameter.

Parent topic: Oracle GoldenGate Key Terms and Concepts

Overview of Groups

#组概要

To differentiate among multiple Extract or Replicat processes on a system, you define processing groups. For example, to replicate different sets of data in parallel, you would create two Replicat groups. #要区分系统上的多个提取或复制进程，需要定义处理组。例如，要并行复制不同的数据集，需要创建两个Replicat组

A processing group consists of a process (either Extract or Replicat), its parameter file, its checkpoint file, and any other files associated with the process. For Replicat, a group may also include an associated checkpoint table. You define groups by using the ADD EXTRACT and ADD REPLICAT commands in the Oracle GoldenGate command interface, GGSCI. #处理组由进程（Extract或Replicat）、其参数文件、检查点文件以及与该进程关联的任何其他文件组成。对于Replicat，组还可以包括相关联的检查点表。您可以使用Oracle GoldenGate命令界面GGSCI中的ADD EXTRACT和ADD REPLICAT命令来定义组

All files and checkpoints relating to a group share the name that is assigned to the group itself. Any time that you issue a command to control or view processing, you supply a group name or multiple group names by means of a wildcard. #与组相关的所有文件和检查点共享分配给组本身的名称。任何时候发出控制或查看处理的命令时，都可以通过通配符提供一个或多个组名。

Parent topic: Oracle GoldenGate Key Terms and Concepts

Overview of the Commit Sequence Number (CSN)

When working with Oracle GoldenGate, you might need to refer to a Commit Sequence Number, or CSN. A CSN is an identifier that Oracle GoldenGate constructs to identify a transaction for the purpose of maintaining transactional consistency and data integrity. It uniquely identifies a point in time in which a transaction commits to the database. #使用Oracle GoldenGate时，可能需要参考提交序列号（CSN）。CSN是Oracle GoldenGate为维护事务一致性和数据完整性而构建的用于识别事务的标识符。它唯一地标识事务提交到数据库（即OGG抽取的源库）的时间点

The CSN can be required to position Extract in the transaction log, to reposition Replicat in the trail, or for other purposes. It is returned by some conversion functions and is included in reports and certain GGSCI output. #CSN可能被要求在事务日志中定位Extract、在踪迹中重新定位Replicat或用于其他目的。它由一些转换函数返回，并包含在报告和某些GGSCI输出中

A CSN is a monotonically increasing identifier generated by Oracle GoldenGate that uniquely identifies a point in time when a transaction commits to the database. It purpose is to ensure transactional consistency and data integrity as transactions are replicated from source to target. Each kind of database management system generates some kind of unique serial number of its own at the completion of each transaction, which uniquely identifies the commit of that transaction. For example, the Oracle RDBMS generates a System Change Number, which is a monotonically increasing sequence number assigned to every event by Oracle RDBMS. The CSN captures this same identifying information and represents it internally as a series of bytes, but the CSN is processed in a platform-independent manner. A comparison of any two CSN numbers, each of which is bound to a transaction-commit record in the same log stream, reliably indicates the order in which the two transactions completed.  #CSN是由Oracle GoldenGate生成的单调递增标识符，它唯一标识事务提交到数据库的时间点。其目的是确保事务从源复制到目标时的事务一致性和数据完整性。每种数据库管理系统在每个事务完成时都会生成自己的某种唯一序列号，该序列号唯一地标识该事务的提交。例如，Oracle RDBMS生成一个系统更改号，这是Oracle RDBMS分配给每个事件的单调递增的序列号。CSN捕获相同的标识信息，并在内部将其表示为一系列字节，但CSN是以独立于平台的方式处理的。任何两个CSN编号的比较，每个编号都绑定到同一日志流中的事务提交记录，可以可靠地指示两个事务完成的顺序

The CSN is cross-checked with the transaction ID (displayed as XID in Oracle GoldenGate informational output). The XID-CSN combination uniquely identifies a transaction even in cases where there are multiple transactions that commit at the same time, and thus have the same CSN. For example, this can happen in an Oracle RAC environment, where there is parallelism and high transaction concurrency. #CSN与事务ID（在Oracle GoldenGate信息输出中显示为XID）进行交叉检查。XID-CSN组合唯一地标识一个事务，即使在同时提交多个事务从而具有相同CSN的情况下也是如此。例如，这可能发生在Oracle RAC环境中，该环境具有并行性和高事务并发性

The CSN value is stored as a token in any trail record that identifies the commit of a transaction. This value can be retrieved with the @GETENV column conversion function and viewed with the Logdump utility. #CSN值作为令牌存储在标识事务提交的任何跟踪记录中。可以使用@GETENV列转换函数检索该值，并使用Logdump实用程序查看该值

See About the Commit Sequence Number in Administering Oracle GoldenGate for more information about the CSN and a list of CSN values per database. #有关CSN和每个数据库的CSN值列表的详细信息，请参阅管理Oracle GoldenGate中的关于提交序列号

This appendix contains information about using the Oracle GoldenGate Commit Sequence Number (CSN) with Oracle and non-Oracle databases. #本附录包含有关在Oracle和非Oracle数据库中使用Oracle GoldenGate提交序列号（CSN）的信息。

All database platforms except Oracle, DB2 LUW, and DB2 z/OS have fixed-length CSNs, which are padded with leading zeroes as required to fill the fixed length. CSNs that contain multiple fields can be padded within each field. For more information on CSN, see Overview of CSN in Understanding Oracle GoldenGate #除了Oracle、DB2 LUW和DB2 z/OS之外，所有数据库平台都有固定长度的CSN，这些CSN根据需要填充前导零以填充固定长度。可以在每个字段中填充包含多个字段的CSN。有关CSN的更多信息，请参阅了解Oracle GoldenGate中的CSN概述

MySQL does not create a transaction ID as part of its event data, so Oracle GoldenGate considers a unique transaction identifier to be a combination of the following: #MySQL不会创建事务ID作为其事件数据的一部分，因此Oracle GoldenGate认为唯一的事务标识符是以下各项的组合：

• the log file number of the log file that contains the START TRANSACTION record for the transaction that is being identified #包含要标识事务的START TRANSACTION记录的日志文件号 （即binlog文件的文件号）

• the record offset of that record  #该记录的记录偏移量

下表是每种数据库的CSN来源：

Table D-1 Oracle GoldenGate CSN Values Per Database

Database	CSN Value
DB2 for i	sequence_number Where: sequence_number is the fixed-length, 20 digit, decimal-based DB2 for i system sequence number. Example: 12345678901234567890
DB2 LUW	LRI Where: For version 10.1 and later, LRI is a period-separated pair of numbers for the DB2 log record identifier. Example: 123455.34645
DB2 z/OS	RBA where: RBA is the 6-byte relative byte address of the commit record within the transaction log. Example: 1274565892
MySQL	LogNum:LogPosition Where: LogNum is the the name of the log file that contains the START TRANSACTION record for the transaction that is being identified. LogPosition is the event offset value of that record. Event offset values are stored in the record header section of a log record. For example, if the log number is 12 and the log position is 121, the CSN is: 000012:000000000000121
Oracle	system_change_number Where: system_change number is the Oracle SCN value. Example: 6488359
SQL Server	Can be any of these, depending on how the database returns it: Colon separated hex string (8:8:4) padded with leading zeroes and 0X prefix Colon separated decimal string (10:10:5) padded with leading zeroes Colon separated hex string with 0X prefix and without leading zeroes Colon separated decimal string without leading zeroes Decimal string Where: The first value is the virtual log file number, the second is the segment number within the virtual log, and the third is the entry number. Examples: 0X00000d7e:0000036b:01bd 0000003454:0000000875:00445 0Xd7e:36b:1bd 3454:875:445 3454000000087500445

概述就介绍到这里，知识粗略的作笔记。想要很精细而且专业的记录，需要很多的精力和时间，本人精力和时间有限还需处理工作事项等。所以有兴趣的网友顺着上面给到的官方文档链接，自行研究即可。