GoldenGate配置实例：RHEL 4.7下的Oracle 10g RAC到单实例的单向同步(二)

安装DDL OBJECTS

在源端，以oracle用户登录sqlplus，执行以下脚本

执行marker_setup

确保goldengate的相关进程 关闭状态，任何使用oracle的应用程序都已关闭，且不会有新的会话产生。然后执行下面命令

[goldengate@gg1 ~]$ cd /opt/gg/goldengate/
[goldengate@gg1 goldengate]$ sqlplus / as sysdba

SQL> @marker_setup

Marker setup script

You will be prompted for the name of a schema for the GoldenGate database objects.
NOTE: The schema must be created prior to running this script.
NOTE: Stop all DDL replication before starting this installation.

Enter GoldenGate schema name: goldengate

……

Script complete.

SQL>

执行ddl_setup

确保所有的会话都已关闭

SQL> @ddl_setup

GoldenGate DDL Replication setup script

Verifying that current user has privileges to install DDL Replication...
Checking user sessions...

Check complete.

You will be prompted for the name of a schema for the GoldenGate database objects.
NOTE: For an Oracle 10g source, the system recycle bin must be disabled. For Oracle 11g and later, it can be enabled.
NOTE: The schema must be created prior to running this script.
NOTE: Stop all DDL replication before starting this installation.

Enter GoldenGate schema name: goldengate

You will be prompted for the mode of installation.
To install or reinstall DDL replication, enter INITIALSETUP
To upgrade DDL replication, enter NORMAL
Enter mode of installation: INITIALSETUP

Working, please wait ...
Spooling to file ddl_setup_spool.txt

Using GOLDENGATE as a GoldenGate schema name, INITIALSETUP as a mode of installation.

Working, please wait ...

RECYCLEBIN must be empty.
This installation will purge RECYCLEBIN for all users.
To proceed, enter yes. To stop installation, enter no.

Enter yes or no: yes

……

Script complete.
SQL>

执行role_setup

创建一个名为ggs_ggsuser_role的角色，包含了ddl objects需要的权限
SQL> @role_setup

GGS Role setup script

This script will drop and recreate the role GGS_GGSUSER_ROLE
To use a different role name, quit this script and then edit the params.sql script to change the gg_role parameter to the preferred name. (Do not run the script.)

You will be prompted for the name of a schema for the GoldenGate database objects.
NOTE: The schema must be created prior to running this script.
NOTE: Stop all DDL replication before starting this installation.

Enter GoldenGate schema name:goldengate
Wrote file role_setup_set.txt

PL/SQL procedure successfully completed.

Role setup script complete

Grant this role to each user assigned to the Extract, GGSCI, and Manager processes, by using the following SQL command:

GRANT GGS_GGSUSER_ROLE TO <loggedUser>

where <loggedUser> is the user assigned to the GoldenGate processes.
SQL>

将创建的角色授权给goldengate用户

SQL> grant ggs_ggsuser_role to goldengate;

Grant succeeded.

SQL>

启用DDL触发器

SQL> @ddl_enable

Trigger altered.

SQL>

安装可选的性能工具

安装dbms_shared_pool包

如果系统中不存在dbms_shared_pool包，则手动执行脚本安装。如下

SQL> select object_name,object_type from dba_objects where object_name='DBMS_SHARED_POOL';

no rows selected

SQL> @?/rdbms/admin/dbmspool

Package created.

Grant succeeded.

View created.

Package body created.

SQL> select object_name,object_type from all_objects where object_name='DBMS_SHARED_POOL';

OBJECT_NAME OBJECT_TYPE
------------------------------ -------------------
DBMS_SHARED_POOL PACKAGE
DBMS_SHARED_POOL PACKAGE BODY

2 rows selected.

ddl_pin

ddl_pin将触发器用到的plsql包放进内存中

SQL> @ddl_pin goldengate

PL/SQL procedure successfully completed.

PL/SQL procedure successfully completed.

PL/SQL procedure successfully completed.

SQL>

配置goldengate

创建goldengate工作目录

源端和目标端：

[goldengate@rac1 goldengate]$ cd $GG_HOME
[goldengate@rac1 goldengate]$ ./ggsci

GGSCI (rac1) 1> create subdirs

Creating subdirectories under current directory /opt/gg/goldengate

Parameter files /opt/gg/goldengate/dirprm: created
Report files /opt/gg/goldengate/dirrpt: created
Checkpoint files /opt/gg/goldengate/dirchk: created
……

GGSCI (gg1) 2> exit

创建trail文件存放目录

源和目标端：

[goldengate@rac1 ~]$ mkdir /opt/gg/trails
[goldengate@rac1 ~]$ ls -l /opt/gg | grep trails
drwxr-xr-x 2 goldengate oinstall 4096 Nov 27 14:57 trails
[goldengate@rac1 ~]$

配置MANAGER

源端和目标端：

DYNAMICPORTLIST中配置了GoldenGate(extract和replicat)进程使用的端口范围
PORT参数指定MANAGER使用的端口
AUTORESTART参数使抽取/复制进程失败后自动重启
配置MANAGER的参数，PURGEOLDEXTRACTS参数指定：当根据checkpoint发现已经完成抽取和复制的trail文件将被自动删除，但保留最近10个。
PURGEDDLHISTORY和PURGEMARKERHISTORY分别删除DDL历史表和marker表中的过期数据，以控制它们不会变得过于庞大。

GGSCI (gg1) 1> edit params mgr

DYNAMICPORTLIST 7840-7914
PORT 5898
PURGEOLDEXTRACTS /opt/gg/trails/w1*, USECHECKPOINTS, MINKEEPFILES 10
PURGEOLDEXTRACTS /opt/gg/trails/w2*, USECHECKPOINTS, MINKEEPFILES 10
AUTORESTART ER *, RETRIES 3, WAITMINUTES 5
PURGEDDLHISTORY MINKEEPDAYS 3, MAXKEEPDAYS 5, FREQUENCYMINUTES 30
PURGEMARKERHISTORY MINKEEPDAYS 3, MAXKEEPDAYS 5, FREQUENCYMINUTES 30
~
~

全局参数设置

源端：

GGSCI (rac1) 2> edit params ./globals
GGSCHEMA goldengate
~
~
~

目标端：

创建一个checkpoint表
replicat通过这个表来维护trail文件中的read position。这不是个必须的操作，如果没有这个表，则通过一个磁盘文件来维护

GGSCI (ggdb) 2> dblogin userid goldengate,password goldengate
Successfully logged into database.

GGSCI (ggdb) 3> add checkpointtable goldengate.chkpoint

Successfully created checkpoint table GOLDENGATE.CHKPOINT.

GGSCI (ggdb) 4> edit params ./globals

GGSCHEMA goldengate
CHECKPOINTTABLE goldengate.chkpoint
~
~
~
~
"./globals" 2L, 44C written

配置源端extract

为了避免primary extract受到网络的影响，我们在源端和目标端之间增加一个data pump，这样的话，primary extract负责将数据从源数据中抽取出来，存在本地的trail文件中，然后data pump进程负责将本地trail文件中的数据传输到目标端的trail文件里。这样能提高更高的灵活性和可用性(当源和目标端之间的网络出现故障时，primary extract会继续抽取数据存到本地的trail中)，图解如下：

我们这里要同步4个用户下的所有表：dycommondatabase20、dyulcentermanage、dyacdb34、dyulcenterm_bak
这里我们将它们分成两部分来测试，两个primary extract，一个负责同步dycommondatabase20和dyulcentermanage下的数据，另一个负责同步dyacdb34和dyulcenterm_bak

首先添加两个primary extract group，因为这是一个两节点的RAC，我们需要指定threads 2选项

GGSCI (rac1) 4> add extract w1ext,tranlog,threads 2,begin now

EXTRACT added.

GGSCI (rac1) 5> info all

Program Status Group Lag Time Since Chkpt

MANAGER STOPPED
EXTRACT STOPPED DYEXT 00:00:00 00:00:07

编辑w1ext的参数文件：

1. EXTTRAIL参数指定该抽取进程对应的exttrail
2. DISCARDFILE参数指定一个文件，用来记录不能正常处理的记录，这里使用追加方式，最大为5MB
3. TRANLOGOPTIONS ALTARCHIVELOGDEST指定源数据库归档所在的路径。如果不确定，使用该SQL*Plus命令：show parameter log_archive_dest_1
4. TRANLOGOPTIONS ALTARCHIVEDLOGFORMAT指定了归档文件的命名格式，可以通过SQL*Plus命令show parameter log_archive_format 来确定该格式
5. 由于源数据使用了ASM，这里通过tranlogoptions asmuser来提供登录ASM实例的用户名和密码(用户名必须是SYS)
6. DDL INCLUDE MAPPED表示只捕获MAPPED范围内的DDL操作
7. DDLOPTIONS ADDTRANDATA ：当创建新的表时，自动为其启用追加日志
8. FETCHOPTIONS：MISSINGROW REPORT表示当extract需要获取的行在源库中无法定位时，extract进程继续运行，相关的错误信息会保存在discardfile参数指定的文件中；USESNAPSHOT表示extract使用flashback查询来从undo从获取一些数据，比如无法从redo中直接获取的UDT、嵌套表、XMLtype以及9i中的LOB；NOUSELATESTVERSION使得extract当无法从undo中获取数据时，忽略该条件而不是从源表中获取当前值。
9. STATOPTIONS REPORTFETCH：使用ggsci命令stats时，显示获取的行的统计信息
10. WARNLONGTRANS 1H, CHECKINTERVAL 5M：当发现超过1个小时的长事务时，会在错误日志中产生一条warning，5分钟检测一次

GGSCI (rac1) 6> edit params w1ext

EXTRACT w1ext
USERID goldengate, PASSWORD goldengate
EXTTRAIL /opt/gg/trails/w1
DISCARDFILE w1extdsc,APPEND,MEGABYTES 5
TRANLOGOPTIONS ALTARCHIVELOGDEST +RECOVERY_DG
TRANLOGOPTIONS ALTARCHIVEDLOGFORMAT %t_%s_%r.dbf
TRANLOGOPTIONS ASMUSER SYS@ORADB_ASM,ASMPASSWORD BBB
DDL INCLUDE MAPPED
DDLOPTIONS ADDTRANDATA
FETCHOPTIONS, USESNAPSHOT, NOUSELATESTVERSION, MISSINGROW REPORT
STATOPTIONS REPORTFETCH
WARNLONGTRANS 1H, CHECKINTERVAL 5M
TABLE dycommondatabase20.*;
TABLE dyulcentermanage.*;


添加该extract对应的trail文件，用来存储抽取的数据。单个文件大小设置为100MB
GGSCI (rac1) 7> add exttrail /opt/gg/trails/w1,extract w1ext, MEGABYTES 100
EXTTRAIL added.

添加secondary extract group，即data pump：

GGSCI (rac1) 8> ADD EXTRACT w1extdp, EXTTRAILSOURCE /opt/gg/trails/w1, BEGIN now
EXTRACT added.

编辑参数文件
RMTHOST后面跟目标端的ip或主机名(需在hosts文件中有对应ip解析)和manager进程的端口号；RMTTRAIL指定目标端的trail文件所在位置

GGSCI (rac1) 9> edit params w1extdp

EXTRACT w1extdp
USERID GOLDENGATE, PASSWORD GOLDENGATE
RMTHOST 192.168.47.211, MGRPORT 5898
RMTTRAIL /opt/gg/trails/w1
DISCARDFILE w1extdpdsc,APPEND,MEGABYTES 5
TABLE dycommondatabase20.*;
TABLE dyulcentermanage.*;
~
~
添加该rmttrail

GGSCI (rac1) 10> add rmttrail /opt/gg/trails/w1,extract w1extdp, megabytes 100
RMTTRAIL added.


类似的，我们添加第二组primary&secondary extract以及exttrail&rmttrail
GGSCI (rac1) 11> add extract w2ext,tranlog,threads 2,begin now
EXTRACT added.

GGSCI (rac1) 12> edit params w2ext

EXTRACT w2ext
USERID goldengate, PASSWORD goldengate
EXTTRAIL /opt/gg/trails/w2
DISCARDFILE w2extdsc,APPEND,MEGABYTES 5
TRANLOGOPTIONS ALTARCHIVELOGDEST +RECOVERY_DG
TRANLOGOPTIONS ALTARCHIVEDLOGFORMAT %t_%s_%r.dbf
TRANLOGOPTIONS ASMUSER SYS@ORADB_ASM,ASMPASSWORD BBB
DDL INCLUDE MAPPED
DDLOPTIONS ADDTRANDATA
FETCHOPTIONS, USESNAPSHOT, NOUSELATESTVERSION, MISSINGROW REPORT
STATOPTIONS REPORTFETCH
WARNLONGTRANS 1H, CHECKINTERVAL 5M
TABLE dyacdb34.*;
TABLE dyulcenterm_bak.*;

GGSCI (rac1) 13> add exttrail /opt/gg/trails/w2,extract w2ext, MEGABYTES 100
EXTTRAIL added.

GGSCI (rac1) 14> ADD EXTRACT w2extdp, EXTTRAILSOURCE /opt/gg/trails/w2, BEGIN now
EXTRACT added.

GGSCI (rac1) 15> edit params w2extdp

EXTRACT w2extdp
USERID GOLDENGATE, PASSWORD GOLDENGATE
RMTHOST 192.168.47.211, MGRPORT 5898
RMTTRAIL /opt/gg/trails/w2
DISCARDFILE w1extdpdsc,APPEND,MEGABYTES 5
TABLE dyacdb34.*;
TABLE dyulcenterm_bak.*;
~
~

GGSCI (rac1) 16> add rmttrail /opt/gg/trails/w2,extract w2extdp, megabytes 100
RMTTRAIL added.

配置目标端replicat

回到目标端，使用goldengate用户登录到ggsci命令行中，对应前面的两组extract和data pump增加两个replicat

GGSCI (ggdb) 6> add replicat w1rep,exttrail /opt/gg/trails/w1,checkpointtable goldengate.chkpoint
REPLICAT added.

这里的handlecollisions参数在目标端数据初始化并同步之后去掉

编辑参数文件：
1. assumetargetdefs:由于在这里我们源端和目标端的表结构是完全一致的，因此使用这个参数来使replicat不用去查看相关的定义文件，从而提高效率
2. DDLOPTIONS REPORT：将ddl的具体信息写入到报告文件中
3. BATCHSQL：将相似的SQL语句放到一个数组中以加快执行速度。在normal模式下，repliat同一时间只应用一条sql语句。
4. DBOPTIONS DEFERREFCONST：将完整性约束推迟到replicat事务提交以后再检测
5. 如果数据库版本在10.2.0.5或11.2.0.2以后，可以使用DBOPTIONS SUPPRESSTRIGGERS在replicat会话中禁用触发器。如果不是，应该在目标端数据库中禁用触发器(触发器产生的DML操作会从源端同步到目标端)
6. DBOPTIONS LOBWRITESIZE ：将要写入目标库的LOB数据缓存在内存中，当达到参数中指定的大小时写入数据，以减少I/O。这个值的范围是2KB到1MB，默认为32KB
7. DDLERROR DEFAULT DISCARD RETRYOP MAXRETRIES 5 RETRYDELAY 20：当出现DDL错误，重试5次，时间间隔为20秒。如果失败，replicat会继续运行，但相关信息会记录在discardfile中。
8. HANDLECOLLISIONS：当replicat往表中插入一条记录，而该记录已经存在，则进行覆盖；当replicat在表中试图更新或删除一条记录，而该记录不存在，则该操作被丢弃。这个参数一般在initial-data load中使用，在源和目标端的数据同步之后应该将该参数删除
9. MAP TARGET：源表和目标表之间的映射，可以使用通配符

GGSCI (ggdb) 7> edit params w1rep

REPLICAT w1rep
ASSUMETARGETDEFS
USERID GOLDENGATE, PASSWORD GOLDENGATE
DISCARDFILE w1repdsc,APPEND,MEGABYTES 5
DDL INCLUDE MAPPED
DDLOPTIONS REPORT
BATCHSQL
DBOPTIONS DEFERREFCONST
DBOPTIONS LOBWRITESIZE 102400
HANDLECOLLISIONS
DDLERROR DEFAULT DISCARD RETRYOP MAXRETRIES 5 RETRYDELAY 20
MAP dycommondatabase20.* , TARGET dycommondatabase20.* ;
MAP dyulcentermanage.* , TARGET dyulcentermanage.* ;
~
~
~

同样地再配置一个replicat进程 w2rep
GGSCI (ggdb) 8> add replicat w2rep,exttrail /opt/gg/trails/w2,checkpointtable goldengate.chkpoint
REPLICAT added.

GGSCI (ggdb) 9> edit params w2rep

REPLICAT w2rep
ASSUMETARGETDEFS
USERID GOLDENGATE, PASSWORD GOLDENGATE
DISCARDFILE w2repdsc,APPEND,MEGABYTES 5
DDL INCLUDE MAPPED
DDLOPTIONS REPORT
BATCHSQL
DBOPTIONS DEFERREFCONST
DBOPTIONS LOBWRITESIZE 102400
HANDLECOLLISIONS
DDLERROR DEFAULT RETRYOP MAXRETRIES 5 RETRYDELAY 20
MAP dyacdb34.* , TARGET dyacdb34.* ;
MAP dyulcenterm_bak.* , TARGET dyulcenterm_bak.* ;
~

ADD TRANDATA

在启动goldengate相关进程之前，在目标端使用trandata对源数据库中需要同步的表启用对象级别的追加日志

GGSCI (rac1) 17> dblogin userid goldengate,password goldengate
Successfully logged into database.

GGSCI (rac1) 18> add trandata dycommondatabase20.*
Logging of supplemental redo data enabled for table DYCOMMONDATABASE20.AUDITCONFIG.

Logging of supplemental redo data enabled for table DYCOMMONDATABASE20.AUDITHISTORY.
……

如果表中没有主键和唯一键，会产生一条警告，例如：
2011-12-08 22:53:11 WARNING OGG-00869 No unique key is defined for table UDPPACKPORTSET. All viable columns will be used to represent the key, but may not guarantee uniqueness. KEYCOLS may be used to define the key.

GGSCI (rac1) 19> add trandata dyulcentermanage.*
……
GGSCI (rac1) 20> add trandata dyacdb34.*
……
GGSCI (rac1) 21> add trandata dyulcenterm_bak.*
……

业务表中没有主键和唯一键，这是个很糟糕的习惯，这里就不批判了...

GoldenGate配置实例：RHEL 4.7下的Oracle 10g RAC到单实例的单向同步(一)

http://blog.csdn.net/wildwave/article/details/7053768

GoldenGate配置实例：RHEL 4.7下的Oracle 10g RAC到单实例的单向同步(二)

http://blog.csdn.net/wildwave/article/details/7056362

GoldenGate配置实例：RHEL 4.7下的Oracle 10g RAC到单实例的单向同步(三)

http://blog.csdn.net/wildwave/article/details/7056451

GoldenGate配置实例：RHEL 4.7下的Oracle 10g RAC到单实例的单向同步(四)
http://blog.csdn.net/wildwave/article/details/7056500